A reconfigurable processor for mix-precision CNNs on FPGA

面向混合量化CNNs的可重构处理器设计

¹ School of Computer Science, Northwestern Polytechnical University, Xi’an 710072, China
² School of Electronic Engineering, Xi’an University of Posts and Telecommunication, Xi’an 710121, China

Received: 15 July 2021

Abstract

To solve the problem of low computing efficiency of existing accelerators for convolutional neural network (CNNs), which caused by the inability to adapt to the characteristics of computing mode and caching for the mixed-precision quantized CNNs model, we propose a reconfigurable CNN processor in this paper, which consists of the reconfigurable adaptable computing unit, flexible on-chip cache unit and macro-instruction set. The multi-core CNN processor can be reconstructed according to the structure of CNN models and constraints of reconfigurable resources, to improve the utilization of computing resources. The elastic on-chip buffer and the data access approach by dynamically configuring an address to better utilization of on-chip memory. Then, the macroinstruction set architecture (mISA) can fully express the characteristics of the mixed-precision CNN models and reconfigurable processors, to reduce the complexity of mapping CNNs with different network structures and computing modes to reconfigurable the CNNs processors. For the well-known CNNs-VGG16 and ResNet-50, the proposed CNN processor has been implemented using Ultra96-V2 and ZCU102 FPGA, showing the throughput of 216.6 GOPS, and 214 GOPS, the computing efficiency of 0.63 GOPS/DSP and 0.64 GOPS/DSP on Ultra96-V2, respectively, achieving a better efficiency than the CNN accelerator based on fixed bit-width. Meanwhile, for ResNet-50, the throughput and the computing efficiency are up to 931.8 GOPS, 0.40 GOPS/DSP on ZCU102, respectively. In addition, these achieve up to 55.4% higher throughput than state-of-the-art CNN accelerators.

摘要

为了解决已有卷积神经网络(convolution neural networks，CNNs)加速器，因无法适应混合量化CNN模型的计算模式和访存特性而引起加速器效率低的问题，设计了可适应混合量化模型的可重构计算单元、弹性片上缓存单元和宏数据流指令集。其中，采用了可根据CNN模型结构的重构多核结构以提高计算资源利用率，采用弹性存储结构以及基于Tile的动态缓存划分策略以提高片上数据复用率，采用可有效表达混合精度CNN模型计算和可重构处理器特性的宏数据流指令集以降低映射策略的复杂度。在Ultra96-V2平台上实现VGG-16和ResNet-50的计算性能达到216.6和214 GOPS，计算效率达到0.63和0.64 GOPS/DSP。同时，在ZCU102平台上实现ResNet-50的计算性能可达931.8 GOPS，计算效率可达0.40 GOPS/DSP，相较于其他类似CNN加速器，计算性能和计算效率分别提高了55.4% 和100%。

1. Zhao R, Hu Y, Dotzel J, et al. Improving neural network quantization without retraining using outlier channel splitting[C]//International Conference on Machine Learning, 2019 [Google Scholar]
2. Wang K, Liu Z, Lin Y, et al. HAQ: hardware-aware automated quantization with mixed precision[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019 [Google Scholar]
3. Ma Y, Cao Y, Vrudhula S, et al. Performance modeling for CNN inference accelerators on FPGAIEEE Trans on Computer-Aided Design of Integrated Circuits and Systems 2019394843856[Article] [Google Scholar]
4. Azizimazreah A, Chen L. Shortcut mining: exploiting cross-layer shortcut reuse in DCNN accelerators[C]//2019 IEEE International Symposium on High Performance Computer Architecture, 2019 [Google Scholar]
5. Hennessy J, Patterson D. A new golden age for computer architecture: domain-specific hardware/software co-design, enhanced[C]//ACM/IEEE 45th Annual International Symposium on Computer Architecture, 2018 [Google Scholar]
6. Judd P, Albericio J, Hetherington T, et al. Stripes: bit-serial deep neural network computing[C]//2016 49th Annual IEEE/ACM International Symposium on Microarchitecture, 2016 [Google Scholar]
7. Lee J, Kim C, Kang S, et al. UNPU: an energy-efficient deep neural network accelerator with fully variable weight bit precision[J]. IEEE Journal of Solid-State Circuits, 2018, 54(1): 173–185 [Article] [Google Scholar]
8. Sharify S, Lascorz A D, Siu K, et al. Loom: exploiting weight and activation precisions to accelerate convolutional neural networks[C]//2018 55th ACM/ESDA/IEEE Design Automation Conference, 2018: 1–6 [Google Scholar]
9. Sharma H, Park J, Suda N, et al. Bit fusion: bit-level dynamically composable architecture for accelerating deep neural network[C]//2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture, 2018: 764–775 [Google Scholar]
10. Yin S, Tang S, Lin X, et al. A high throughput acceleration for hybrid neural networks with efficient resource management on FPGA[J]. IEEE Trans on Computer-Aided Design of Integrated Circuits and Systems, 2018, 38(4): 678–691 [Article] [Google Scholar]
11. Ma Y, Cao Y, Vrudhula S, et al. Automatic compilation of diverse CNNs onto high-performance FPGA accelerators[J]. IEEE Transations on Computer-Aided Design of Integrated Circuits and Systems, 2018, 39(2): 424–437 [Article] [Google Scholar]
12. Wei X, Yu C H, Zhang P, et al. Automated systolic array architecture synthesis for high throughput CNN inference on FPGAs[C]//Proceedings of the 54th Annual Design Automation Conference, 2017 [Google Scholar]
13. Guo K, Sui L, Qiu J, et al. Angel-eye: a complete design flow for mapping CNN onto embedded FPGA[J]. IEEE Trans on Computer-Aided Design of Integrated Circuits and Systems, 2017, 37(1): 35–47 [Article] [Google Scholar]
14. Azizimazreah A, Chen L. Polymorphic accelerators for deep neural networks[J]. IEEE Trans on Computers, 2022, 71(3): 534–546 [Article] [CrossRef] [Google Scholar]
15. Dong Z, Yao Z, Arfeen D, et al. HAWQ-v2:hessian aware trace-weighted quantization of neural networks[J]. Advances in Neural Information Processing Systems, 2020, 33: 18518–18529 [Article] [Google Scholar]