What is neural architecture search (NAS)?

Neural Architecture Search (NAS) automates the design of neural network architectures. Instead of hand-designing networks, NAS uses algorithms to search over a design space of possible architectures to find the optimal one for a given task and resource budget. Approaches: Reinforcement Learning (train an RL controller to generate architectures), Evolutionary algorithms, and Differentiable NAS (DARTS) (relaxes the discrete architecture space to be continuous, enabling gradient-based optimization). NAS discovered EfficientNet and NASNet architectures that outperformed hand-designed networks. Main challenge: computational cost.