AI & Fundamentals
Recent Advances in Neural Architecture Search - Frank Hutter, Professor, University of Freiburg
DATE: Mon, December 16, 2019 - 10:30 am
LOCATION: ICCS - X836, ICICS Computer Science, 2366 Main Mall, Vancouver, BC
Deep learning has removed the need for manual feature engineering but still requires a lot of manual work on architecture design. Neural architecture search (NAS) can be seen as the logical next step in representation learning, by also learning the architecture used to learn the representation. Correspondingly, the young field of NAS is currently exploding, and I will give an overview of some of the works in the field. No previous knowledge of NAS is required! I'll cover blackbox optimization approaches and various ways to speed them up, including weight inheritance, multi-fidelity optimization, meta-learning, and weight sharing methods. I'll also discuss various issues concerning reproducibility and benchmarking in the field, as well as failure modes of the prominent DARTS approach, and how to overcome these.
My students Arber Zela, Julien Siems, Jörg Franke, and Raghu Rajan will also be joining for the talk and we'll all be available for follow-up questions during the networking lunch.
Frank Hutter is a Full Professor for Machine Learning at the Computer Science Department of the University of Freiburg (Germany), where he has been a faculty member since 2013. Before that, he was at the University of British Columbia (UBC) for eight years, for his PhD and postdoc. Frank is best known for his work on AutoML. For his 2009 PhD thesis on algorithm configuration, he received the CAIAC doctoral dissertation award for the best thesis in AI in Canada that year, and with his coauthors, he received several best paper awards and prizes in international competitions on machine learning, SAT solving, and AI planning. Since 2016 he holds an ERC Starting Grant for a project on automating deep learning based on Bayesian optimization, Bayesian neural networks, and deep reinforcement learning. He is also a Fellow of ELLIS and consults for the Bosch Center for Artificial Intelligence as Chief Expert AutoML.
Associate Professor Mark Schmidt, Computer Science, UBC