OS Numerical Optimization: Adaptive Deep Neural Network Architectures: Time-adaptive pruning and sensitivity-based layer insertion
Time
Thursday, 13. June 2024
15:15 - 16:45
Location
F423
Organizer
B. Azmi & S. Volkwein
Speaker:
Dr. Evelyn Herberg
On 13th June 2024 at 15:15, Dr. Evelyn Herberg from the Scientific Computing and Optimization (University of Heidelberg) will give a talk.
Abstract: We propose adaptive neural network architecture approaches for Feedforward Neural Networks (FNNs) and Residual Neural Networks (ResNets). On the one hand we propose an adaptive pruning approach, and on the other hand we provide a sensitivity-based layer insertion technique. In both approaches, the hyperparameter search is simplified and network complexity can be reduced without compromising expressiveness, while simultaneously decreasing training time. The results are illustrated by numerical examples.