# Non-linear Functional Modeling using Neural Networks

**Authors:** Aniruddha Rajendra Rao, Matthew Reimherr  
**arXiv:** [2104.09371](https://arxiv.org/abs/2104.09371)  
**Date:** 2021-04-19  
**Summary page:** https://www.summarizepaper.com/en/paper/2104.09371v1/

## Key points

- The paper proposes a new class of non-linear models for functional data based on neural networks
- The approach utilizes continuous hidden layers with continuous neurons to capture temporal dependency and complicated relationships in the data
- Two variations of the framework are introduced: Functional Direct Neural Network (FDNN) and Functional Basis Neural Network (FBNN)
- A functional gradient-based optimization algorithm is derived to fit these models
- FDNN and FBNN outperform other existing methods in terms of RMSE and classification error on various datasets, including Fat Spectrum, Growth, and Phonemes
- The proposed approach is flexible as it does not assume a particular structure that limits the kinds of non-linear patterns learned, making it suitable for various applications such as speech recognition, image processing, and finance
- The proposed FDNN and FBNN models provide an effective solution for modeling complex non-linear relationships in functional data with scalar response
- Future work could explore extending this approach to multivariate or high-dimensional functional data settings to further improve its performance.

## Summary

This paper proposes a new class of non-linear models for functional data based on neural networks, which is an extension of deep learning to functional data. The proposed approach utilizes continuous hidden layers with continuous neurons to capture temporal dependency and complicated relationships in the data. Two variations of the framework are introduced: the Functional Direct Neural Network (FDNN) and the Functional Basis Neural Network (FBNN), both designed explicitly to exploit the structure inherent in functional data. To fit these models, a functional gradient-based optimization algorithm is derived. The effectiveness of the proposed methods in handling complex functional models is demonstrated through comprehensive simulation studies and real data examples. The results show that FDNN and FBNN outperform other existing methods in terms of root mean square error (RMSE) and classification error on various datasets, including Fat Spectrum, Growth, and Phonemes. The proposed approach is flexible as it does not assume a particular structure that limits the kinds of non-linear patterns learned, especially when dealing with complex functional data. This makes it suitable for various applications such as speech recognition, image processing, and finance. In conclusion, this paper presents a novel approach for complex functional regression with scalar response using neural networks. The proposed FDNN and FBNN models provide an effective solution for modeling complex non-linear relationships in functional data. Future work could explore extending this approach to multivariate or high-dimensional functional data settings to further improve its performance.