Results of the summarizing process for the arXiv paper: 2303.17954v1

The field of Machine Learning (ML) and Deep Learning (DL) has seen a recent surge in the deployment of models at the extreme edge of the Internet-of-Things (IoT), commonly referred to as Tiny Machine Learning (TinyML). To meet the strict latency, throughput, accuracy, and flexibility requirements for on-chip DNN inference and training at this level, heterogeneous clusters have emerged as promising solutions. These clusters combine the flexibility of DSP-enhanced cores with the performance and energy boost of dedicated accelerators. In this context, DARKSIDE is presented as a System-on-Chip with a heterogeneous cluster consisting of 8 RISC-V cores enhanced with 2-b to 32-b mixed-precision integer arithmetic. To enhance performance and efficiency on key compute-intensive DNN kernels, three digital accelerators are added to the cluster: a specialized engine for low-data-reuse depthwise convolution kernels capable of up to 30 MAC/cycle; a minimal overhead datamover that marshals 1-b to 32-b data on-the-fly; and a 16-b floating-point Tensor Product Engine (TPE) for tiled matrix multiplication acceleration. DARKSIDE is implemented using 65nm CMOS technology and achieves peak integer performance of 65 GOPS and peak efficiency of 835 GOPS/W when working on 2-b integer DNN kernels. When targeting floating point tensor operations, the TPE provides up to 18.2 GFLOPS of performance or 300 GFLOPS/W of efficiency - enough to enable on chip floating point training at competitive speed coupled with ultra low power quantized inference. The authors argue that while catch instructions can improve energy efficiency by reducing memory accesses in certain scenarios, they may not be sufficient for all use cases. Instead they propose using heterogeneous clusters like DARKSIDE that offer both flexibility and performance boosts through dedicated accelerators. Overall, DARKSIDE presents a promising solution for on chip DNN inference and training at the extreme edge of TinyML, with its heterogeneous cluster offering both flexibility and performance boosts through dedicated accelerators. Its implementation in 65nm CMOS technology demonstrates impressive peak integer performance and efficiency when working on 2 b integer DNN kernels while its TPE provides sufficient performance and efficiency to enable on chip floating point training coupled with ultra low power quantized inference.

• - Machine Learning (ML) and Deep Learning (DL) models are being deployed at the extreme edge of the Internet-of-Things (IoT), known as Tiny Machine Learning (TinyML).
• - Heterogeneous clusters have emerged as promising solutions to meet the strict latency, throughput, accuracy, and flexibility requirements for on-chip DNN inference and training at this level.
• - DARKSIDE is presented as a System-on-Chip with a heterogeneous cluster consisting of 8 RISC-V cores enhanced with 2-b to 32-b mixed-precision integer arithmetic.
• - Three digital accelerators are added to the cluster: a specialized engine for low-data-reuse depthwise convolution kernels capable of up to 30 MAC/cycle; a minimal overhead datamover that marshals 1-b to 32-b data on-the-fly; and a 16-b floating-point Tensor Product Engine (TPE) for tiled matrix multiplication acceleration.
• - DARKSIDE achieves peak integer performance of 65 GOPS and peak efficiency of 835 GOPS/W when working on 2-b integer DNN kernels.
• - The TPE provides up to 18.2 GFLOPS of performance or 300 GFLOPS/W of efficiency when targeting floating point tensor operations.
• - The authors argue that using heterogeneous clusters like DARKSIDE that offer both flexibility and performance boosts through dedicated accelerators is better than catch instructions in certain scenarios.
• - Overall, DARKSIDE presents a promising solution for on chip DNN inference and training at the extreme edge of TinyML, with its heterogeneous cluster offering both flexibility and performance boosts through dedicated accelerators.

Machine Learning (ML) and Deep Learning (DL) are ways computers can learn to do things on their own. They are being used in very small devices called Tiny Machine Learning (TinyML). DARKSIDE is a special computer chip that has 8 small processors and 3 special helpers to make it work better. It can do math really fast, which helps it learn things quickly. The authors of the article think that using chips like DARKSIDE is better than other ways of teaching computers in some cases. Overall, DARKSIDE is a good way to teach computers at the smallest level. Definitions- Machine Learning: A way for computers to learn how to do things on their own. - Deep Learning: A type of machine learning that uses complex algorithms to help computers learn. - Internet-of-Things (IoT): Devices that are connected to the internet, like smart watches or thermostats. - Heterogeneous clusters: Groups of different types of processors working together. - Inference: Using what a computer has learned to make decisions or predictions.

The Promise of TinyML: Introducing DARKSIDE

In recent years, the field of Machine Learning (ML) and Deep Learning (DL) has seen a surge in the deployment of models at the extreme edge of the Internet-of-Things (IoT), commonly referred to as Tiny Machine Learning (TinyML). To meet the strict latency, throughput, accuracy, and flexibility requirements for on-chip DNN inference and training at this level, heterogeneous clusters have emerged as promising solutions. In this context, DARKSIDE is presented as a System-on-Chip with a heterogeneous cluster consisting of 8 RISC-V cores enhanced with 2-b to 32-b mixed precision integer arithmetic.

DARKSIDE’s Heterogeneous Cluster

To enhance performance and efficiency on key compute intensive DNN kernels, three digital accelerators are added to the cluster:

• A specialized engine for low data reuse depthwise convolution kernels capable of up to 30 MAC/cycle;
• A minimal overhead datamover that marshals 1 b to 32 b data on the fly;
• And a 16 b floating point Tensor Product Engine (TPE) for tiled matrix multiplication acceleration.

DARKSIDE is implemented using 65nm CMOS technology and achieves peak integer performance of 65 GOPS and peak efficiency of 835 GOPS/W when working on 2 b integer DNN kernels. When targeting floating point tensor operations, its TPE provides up to 18.2 GFLOPS performance or 300 GFLOPS/W efficiency - enough to enable on chip floating point training at competitive speed coupled with ultra low power quantized inference.

Catch Instructions vs Heterogeneous Clusters

The authors argue that while catch instructions can improve energy efficiency by reducing memory accesses in certain scenarios, they may not be sufficient for all use cases. Instead they propose using heterogeneous clusters like DARKSIDE that offer both flexibility and performance boosts through dedicated accelerators. This approach allows users to take advantage of specific hardware features such as vectorization or parallelism which can significantly reduce execution time compared with catch instructions alone. Furthermore it also allows developers more freedom in terms of design choices since they don't need to rely solely on catch instructions but can instead tailor their designs according their needs without sacrificing too much in terms of energy consumption or performance gains.

Conclusion

Overall , DARKSIDE presents a promising solution for on chip DNN inference and training at the extreme edge of TinyML , with its heterogeneous cluster offering both flexibility and performance boosts through dedicated accelerators . Its implementation in 65nm CMOS technology demonstrates impressive peak integer performance and efficiency when working on 2 b integer DNN kernels while its TPE provides sufficient performance and efficiency to enable on chip floating point training coupled with ultra low power quantized inference .