With the increasing size of models in Convolutional Neural Networks (CNNs) recently, the demand for memory size, bandwidth and computational resources has gradually become a central issue. Quantification has a pivotal role in dramatically reducing the computation of CNN models and bandwidth. However, quantization technology is difficult to improve the throughput and power efficiency of accurately fixed accelerators. Different applications have different requirements for accelerators in all aspects, and accurately fixing accelerators lacks the flexibility to meet these requirements. In this paper, a precision configurable processing unit (PE) is proposed, which not only simplifies the computing unit and the external complex configurable logic, but also introduces the concept of approximate calculation, while ensuring a certain precision of CNN. For the first time, approximate computation is introduced in a configurable computational unit, which allows the architecture to further reduce power consumption based on bit-level flexibility and to accommodate parameters from different quantization methods of the network. The design of this paper is implemented in SMIC 40nm process library. Compared with Bit Fusion [1], this method achieves the lowest accuracy of 98.49% in Lenet, ensuring that the area and power consumption are reduced by 53.2% and 19.8% respectively.
both activations and weights to be scaled in proportion. Compared with the traditional data gating approach, DVAFS, with a shortened critical path and dynamic adjustment of the clock frequency, adopts the sparsity of convolution in a dedicated processor architecture during the chip implementation and achieves variable voltage and frequency with accuracy.
However, with the increase of performance, the lower precision components require complex
configurable logic. The decrease in precision also requires more activations and weights to perform the
computation of precision-configurable units such as Bit-Fusion [
The main contributions of this paper are as follows: (1) A precision-configurable computational unit
is proposed to simplify the computational unit and the complex external configurable logic under the
premise of a certain accuracy of the neural network; (2) For the first time, an approximation is
introduced in the configurable unit, which enables the architecture to further reduce power
consumption on the basis of bit-level flexibility and to adapt to parameters from multiple quantization
methods of the network; (3) The design of this paper is implemented in SMIC 40nm process library.
Compared with Bit Fusion [
The remainder of this paper is organized as follows. The second section introduces the background work of Quantization Compression and precision scalability. The third section analyzes the hardware design content and main innovations, and the fourth section evaluates the performance of the whole design. Finally, the fifth section summarizes the full text.
When CNNs are applied in embedded devices, it is necessary to consider not only the demand on
memory size, bandwidth and computational resources brought by the huge computational volume, but
also the problem of limited energy supply. Quantization [
MAC operations account for 99% of the total operations in CNN. 97.3% of MAC operations can be
performed at less than 4 bits without affecting the accuracy, and even most of the operations can be
done at 1 bit. [
Accuracy-scaling MACs can adapt to the input parameters of different quantified network, which
makes the hardware much more flexible. Precision-scaling MACs are efficiently parallelized or
serialized. Data gating is first proposed in configurable arithmetic circuits, after which Subword
Parallelism, Divide and Conquer, and Bit-serial architectures were proposed, respectively. Bit-Fusion
[
Precision-scaling units are generally composed of adders, multipliers and external configuration units. The research on adders and multipliers is very mature, and we fuse the external configuration unit with the MAC unit in order to reduce the overall area and power consumption. The basic principle of calculating multiplication in this design is the same as that of Bit-Fusion. However, this design takes into account the fact that the partial sums of high bits and low bits do not affect each other and can be added by bit splicing. Thus, the use of bit splicing in our design reduces the use of accumulator and thus reduces the area overhead. In addition, in this design, there is no need to make up 1-bit sign bit after splitting into lower bits. The smallest unit of this design implements a 2-bit multiplication operation, which further saves area overhead compared to Bit-Fusion which implements a 3-bit multiplication operation.
In addition to the above optimization methods, this paper also introduces approximate means in configurable computing units for the first time. We use the LOA adder to approximate optimize the configurable computing unit, and further reduce the area and power consumption on the premise of ensuring accuracy.
The main purpose of this design is to make the architecture have bit level flexibility on the basis of reducing power consumption, and finally be able to adapt to the parameters from various quantization methods of the network. The core of this design is the dynamic implementation of operand bit-width adjustment with a multiplexer to select the bit-width mode. The configurable MAC architecture is able to dynamically implement calculations for three cases--8×8, 4×4, and 2×2, which is sufficient for application to neural networks and avoids fine-grained calculations.
As the variety of computations supported by MAC increases, the complexity of the hardware design increases. Because fine-grained computations can lead to complex architectures, the required granularity needs to be carefully chosen. If two computation cases have similar accuracy, the one with better throughput can be used instead of the other, which reduces the variety of computations and simplifies the hardware design.
In our design, the configurable MAC is applied to the LeNet5 network with the activation and weight quantified to 8 bit, 4 bit and 2 bit, respectively. Since the quantization of activation affects the accuracy more than the quantization of weight, only the six computation cases-- 8×8, 8×4, 8×2, 4×4, 4×2, and 2×2 --shown in Table II are considered. As shown in Table I, there is little difference in accuracy between these six computation cases.
Res34 73.6 73.1 71.5 N/A N/A N/A 8×4
VGG16 ResNe‐t152 2×2
The effect of simultaneous quantization of weights and activation on the training results was investigated on the PyTorch platform to verify the feasibility of the simplified computational cases. It is evident from Fig.1 that the quantization of weights and activation has little effect on the output accuracy of the trained model. The activation is more sensitive to changes in the number of quantization bits due to a larger range of activation quantization errors. Therefore, the accuracy configurable MAC unit supports 8×8, 4×4, and 2×2 computations, which can greatly reduce the complexity of hardware computation within the accuracy loss allowed.
1)Using a 2-bit multiplier based on multiplexer: The smallest computation unit of this design is the bit-level processing element, which is capable of 2-bit multiplication. When this design perform the 2bit multiplication, the multiplexer determines the operand with or without sign, avoiding the addition of sign bits in Bit-Fusion.
A and B are two signed/unsigned numbers. The inputs and outputs of the multipliers are in the form of the complement of signed numbers. Cond(1) represents the case of "unsigned A × unsigned B"; cond(2) represents the "signed A × unsigned B"
A,B
A<<2
0
（1）
（3）
（2）
sel
4 A[
(3) 2)Adopting bit-splicing: Bit-splicing method directly merges partial products that do not interfere with each other. Taking the 4×4 shown in Fig. 3 as an example, the 4-bit A and B are first split into 2bit A[3:2], A[1:0], B[3:2] and B[1:0]. The product A[1:0]×B[1:0] of the lower 2-bit and the product A[3:2]×B[3:2] of the higher 2-bit do not interfere with each other and can be directly bit-spliced without going through accumulation. In the 4-bit multiplication, a bit-splicer is used instead of an adder and a shifter to avoid shifting and accumulation of the higher four bits of the partial sum. 8×8 16 9 8
As shown in Table III, the number of multipliers, shifters, and adders required by the bit-splicingbased approach proposed in this paper is significantly less than that required in Bit-Fusion, and this advantage becomes more and more obvious as the number of bits of the computed multipliers becomes larger. In comparison, this design have smaller area and lower power consumption, so the overall area and power consumption of the configurable MAC are smaller.
2×2 4×4 8×8 2×2 4×4
multiplier
multipliers
3)Building configurable multipliers: The 2BM which perform 2-bit multiplication are arranged in the space. As shown in Fig. 4, a complete configurable MAC is composed of 16 2BMs and is capable of accommodating MAC operations of 2bit, 4bit, and 8bit DNN layers.
M 16 15 15
<<2 A0,1 B2,3 A2,3 B0,1 A0,1 B0,1 A2,3 B2,3 2BM 2BM 2BM 2BM <<2 BA00,,11 2BM BA40,,51 2BM BA20,,31 2BM A0,1 B6,7 2BM BA24,,35 2BM BA04,,15 2BM BA04,,15 2BM A4,5 B2,3 2BM
M M M M 4 3 3 <<2 <<2
M M 1 0 0 2BM BA02,,13
A2,3 2BM B4,5 2BM BA22,,33
A2,3 2BM B6,7 2BM BA06,,17 <<2 2BM BA46,,57 2BM BA26,,37
A6,7 <<2 2BM B6,7 4 1 2
In this experiment, we perform a hardware implementation of the above proposed model and a simple analysis of the whole implementation is performed in Design Compiler. The area of the whole design and the percentage of the adder module, 2BMs and the external configuration module are counted in Table V, and it is found that the adder area accounts for a larger percentage. Due to the fault tolerance of CNNs, the adder is approximated to operate with a certain accuracy. 100
796.41 54.1 2BM 511.9 34.8
163.52 11.1
Approximate adders are mainly classified as Accuracy Configurable Adder (ACA), Speculative Carry Select Addition (SCSA), Carry-Skip Adder (CSA), Error-Tolerant Adder (ETA) and Low or Adder (LOA). The comparison of various approximate adders is as shown in Table VI. It is found that LOA has the smallest area and power consumption due to the complete use of logic or gates for low-bit operation, but it has the highest error rate because the accuracy is not considered. Since adders have different requirements for different bit widths, we finally choose LOA.
adders (um2) (ns) (uW)
LOA 53.2 0.39 65.9 ETAII 71.6 0.55 80.6 ACA 73.8 0.25 118.4 SCSA 109.2 0.32 134.5 CSA 142.5 0.39 97.8
Error rate
(%) 89.99 5.85/16.94 16.66/16.34
5.85 0.18/0.91
Mean Relative Error (um2) 1.0 2.6 18.9 2.6 0.15
The approximate bit widths required for different bit-width adders are different. We modeled each adder in Matlab, selected certain equal intervals, and tested its MRED using Monte Carlo method. Because the MRED of the computational unit is usually required to be less than 5%[18], we finally determined the approximate bit-width to be 2/3/4/6 for bit-width 6/8/10/16 bits. We list three cases for different bit widths in Table VII for comparison.
After modeling the selected low approximation bits corresponding to different bit-width adders, the hardware implementation of each adder will be performed and the implemented hardware will be tested in Design Compiler, and the final test results are shown in Table VII. Finally, we use case 2 as the final bit width selection.
wBiditt‐h Number (Aurmea2) To(tuaml a2r)e a Area of A(pupmr2o)x imation 6 6 30.88 185.28 23.46 8 4 40.93 163.72 29.45 10 4 50.99 203.96 35.43
Approximation (um2) 140.76 117.80 141.72 wBiditt‐h Number (Aurmea2) To(tuaml a2r)e a Area of A(pupmr2o)x imation ApprTooxtiaml aatrieoan o (fu m2) 16 3 81.15 243.45 57.45 172.35 As shown in Figure VIII, we count the number of adders, the area of a single accurate and approximate adder, and the total area of a fixed bit width adder. The sum of the statistical exact adder area and the sum of the approximate post adder area are compared to achieve a gain of 39.41% in area, which is a huge gain for the entire computational unit.
Bit-Fusion [
1)Comparison of 2-bit minimum multiplication: Bit-Fusion uses four full adders(FAs) and three half adders(HAs) while this design uses only one FAs, four HAs and two data selectors. The benefits of using this design are considerable as it reduces the area by 35.5% and the power consumption by 47.5% compared to the Bit-Fusion design.
2)Comparison of 8-bit multiplication: The 8-bit multiplier is compatible with sixteen multipliers with 2-bit input or four multipliers with 4-bit input. Due to the bit-selective design and the use of bitsplicing, the benefits of this design are considerable as it reduces the area by 53.1% and the power consumption by 40% compared to the Bit-Fusion design.
8‐Bit
Area(um2) 1471.83 1247.99 17.9%
Power(mW) 1.59 1.46 8.9%
The final approximate design solution is compared with the comparative design Bit-Fusion, and the results are shown in Table Ⅹ. The accuracy configurable unit is greater than 53.2% in power consumption and greater than 19.8% in area reduction.
8‐Bit
Area(um2) 1912.5 1247.99 53.2 Power(uW) 1.75 1.46 19.8
2 3 4 6
4 8
0.019
Based on the fault-tolerance of CNNs, the accuracy-configurable unit enables the circuit to accept a
variety of network parameters by means of adding additional configuration units. In this paper, from
the perspective of improving the flexibility of accelerators, the precision- scaling MAC is designed to
adapt to multiple network structures while ensuring low power consumption. The hardware
performance of the accelerator will be improved, and the worst accuracy in Lenet will reach more than
98.49%. The precision-configurable cell carried out in SMIC 40nm process has a power gain of more
than 53.2% and an area reduction gain of more than 19.8% compared to Bit-Fusion [
[14] T Luo, S Liu, L Li, et al. DaDianNao: A Neural Network Supercomputer[J]. IEEE
TRANSACTIONS ON COMPUTERS, 2017, 66(1): 73-88. [15] Y-H Chen, T Krishna, J S Emer, et al. Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks[J]. IEEE JOURNAL OF SOLID-STATE CIRCUITS, 2017, 52(1): 127-138. [16] S Ryu, H Kim, W Yi, et al. BitBlade: Area and Energy-Efficient Precision-Scalable Neural Network Accelerator with Bitwise Summation; proceedings of the 2019 56th ACM/IEEE Design Automation Conference (DAC), F 2-6 June 2019, 2019 [C]. [17] Jiang,H.,Liu,C., Liu,L. et al.(2017) A Review. Classification, and Comparative Evaluation of Approximate Arit-metic Circuits.ACMJournal onEmerging Technologies in Computing Systems(JETC),13,1-34. [18] Q. Li, X. Fan, J. Chen, H. Li and H. Liu, "A Hardware Efficient Approximate Shift Multiplier with High Accuracy," 2021 IEEE 14th International Conference on ASIC (ASICON), 2021, pp. 1-5, doi: 10.1109/ASICON52560.2021.9620363.