# Training Quantized Neural Networks to Global Optimality via Semidefinite Programming

**Authors:** Burak Bartan, Mert Pilanci  
**arXiv:** [2105.01420](https://arxiv.org/abs/2105.01420)  
**Date:** 2021-05-04  
**Summary page:** https://www.summarizepaper.com/en/paper/2105.01420v2/

## Key points

- Neural networks (NNs) have achieved remarkable success across many tasks in machine learning
- Quantization of NN weights impacts their energy efficiency, inference time, and deployment on hardware
- Training optimal quantized NNs involves combinatorial non-convex optimization problems that appear to be intractable
- Burak Bartan and Mert Pilanci introduce a convex optimization strategy to train quantized NNs with polynomial activations
- Their method leverages hidden convexity in two-layer neural networks from recent literature as well as semidefinite lifting and Grothendieck's identity
- Certain quantized NN problems can be solved to global optimality in polynomial-time via semidefinite relaxations
- The authors present numerical examples to illustrate the effectiveness of their method and demonstrate that it outperforms existing methods for training quantized NNs on various datasets such as CIFAR-10 and ImageNet.
- Their approach can achieve near-optimal performance even when using only a few bits per weight.
- This work presents an innovative approach for training optimal quantized NNs with polynomial activations using convex optimization strategies which has significant implications for improving the energy efficiency and inference time of NNs while maintaining high accuracy levels.

## Summary

Neural networks (NNs) have been widely used in machine learning and have achieved remarkable success across many tasks. However, the energy efficiency, inference time, and deployment on hardware of NNs are impacted by the quantization of their weights. Post-training quantization is a well-studied topic but training optimal quantized NNs involves combinatorial non-convex optimization problems that appear to be intractable. In this work, Burak Bartan and Mert Pilanci introduce a convex optimization strategy to train quantized NNs with polynomial activations. Their method leverages hidden convexity in two-layer neural networks from recent literature as well as semidefinite lifting and Grothendieck's identity. Surprisingly, they show that certain quantized NN problems can be solved to global optimality in polynomial-time in all relevant parameters via semidefinite relaxations. The authors present numerical examples to illustrate the effectiveness of their method and demonstrate that it outperforms existing methods for training quantized NNs on various datasets such as CIFAR-10 and ImageNet. Moreover, they show that their approach can achieve near-optimal performance even when using only a few bits per weight. This work presents an innovative approach for training optimal quantized NNs with polynomial activations using convex optimization strategies which has significant implications for improving the energy efficiency and inference time of NNs while maintaining high accuracy levels.