# Efficacy of Neural Prediction-Based NAS for Zero-Shot NAS Paradigm

**Authors:** Minh Le, Nhan Nguyen, Ngoc Hoang Luong  
**arXiv:** [2308.16775](https://arxiv.org/abs/2308.16775)  
**Date:** 2023-08-31  
**Summary page:** https://www.summarizepaper.com/en/paper/2308.16775v1/

## Key points

- Prediction-based approaches in Neural Architecture Search (NAS) have been successful in leveraging performance indicators from graph convolutional networks through one-hot encoding of feed-forward structures as component graphs.
- Zero-shot NAS paradigm utilizes handcrafted performance indicators with random initialization, demonstrating remarkable generalization abilities across diverse search spaces.
- A novel approach has been proposed to enhance zero-shot NAS by incorporating Fourier sum of sines encoding for convolutional kernels, enabling the construction of a computational feed-forward graph that closely mirrors the architecture under evaluation.
- Learnable encodings provide a comprehensive representation of the architecture's topological information, and a multi-layer perceptron (MLP) is used to rank architectures based on their encoded features.
- Experimental results show that this new approach outperforms previous methods using graph convolutional networks in terms of correlation on the NAS-Bench-201 dataset and exhibits a higher convergence rate.
- Feature representations extracted and trained on each NAS-Benchmark demonstrate transferability to other benchmarks, indicating promising generalizability across various search spaces.

## Summary

In the realm of Neural Architecture Search (NAS), the use of prediction-based approaches has garnered attention for their success in leveraging performance indicators derived from graph convolutional networks. These indicators are obtained through one-hot encoding of feed-forward structures as component graphs and have shown significant efficacy. However, a notable limitation arises from their inability to effectively evaluate architecture performance across diverse search spaces. In contrast, the zero-shot NAS paradigm employs handcrafted performance indicators using the same architecture with random initialization and showcases remarkable generalization abilities across multiple search spaces. To address this limitation and enhance the capabilities of zero-shot NAS, a novel approach utilizing deep learning techniques has been proposed. This innovative method incorporates Fourier sum of sines encoding for convolutional kernels, enabling the construction of a computational feed-forward graph that closely mirrors the structure of the architecture under evaluation. These learnable encodings provide a comprehensive representation of the architecture's topological information. Subsequently, a multi-layer perceptron (MLP) is employed to rank these architectures based on their encoded features. Experimental results have demonstrated that this new approach outperforms previous methods utilizing graph convolutional networks in terms of correlation on the NAS-Bench-201 dataset and exhibits a higher convergence rate. Furthermore, feature representations extracted and trained on each NAS-Benchmark showcase transferability to other NAS-Benchmarks, indicating promising generalizability across various search spaces. The study conducted by Minh Le, Nhan Nguyen, and Ngoc Hoang Luong sheds light on the efficacy of neural prediction-based NAS for zero-shot NAS paradigms. The research delves into innovative methodologies that not only push boundaries in architectural evaluation but also pave the way for enhanced generalizability and efficiency in neural architecture search tasks. For further insights and access to the code repository associated with this study, interested readers can refer to https://github.com/minh1409/DFT-NPZS-NAS.