FPGA-based Dynamic Deep Learning Acceleration for Real-time Video Analytics

Deep neural networks (DNNs) are a key technique in modern artificial intelligence that has provided state-of-the-art accuracy on many applications, and they have received significant interest. The requirements for ubiquity of smart devices and autonomous robot systems are placing heavy demands on DNNs-inference hardware, with high requirement for energy and computing efficiencies, along with the rapid development of AI techniques. The high energy efficiency, computing capabilities, and reconfigurability of FPGAs make these a promising platform for hardware acceleration of such computing tasks. This paper primarily addresses this challenge and proposes a new flexible hardware accelerator framework to enable adaptive support for various DL algorithms on an FPGA-based edge computing platform. This framework allows run-time reconfiguration to increase power and computing efficiency of both DNN model/software and hardware, to meet the requirements of dedicated application specifications and operating environments. The achieved results show that with the proposed framework is capable to reduce energy consumption and processing time up to 53.8% and 36.5% respectively by switching to a smaller model. In addition, the time and energy consumption are further elaborated with a benchmark test set, which shows that how input data in each frame and size of a model can affect the performance of the system.