Seminar talk Prof. Gharesifard "Universal approximation power of deep neural networks through the lens of control theory"

16/11/2021
 

Speaker

Bahman Gharesifard (UCLA)

Title

Universal approximation power of deep neural networks through the lens of control theory

Time and Place

November 16, 4pm

Seminar room 333, 3rd floor in our ICT cubes

Attendance

Notice that the 3G regulation (''3G rule'') applies.

Abstract

We explain the universal approximation capabilities of deep residual neural networks using tools from control theory. In particular, we show that residual networks with a large class of activation functions have the power of universal approximation. The first contribution consists of relating the universal approximation problem to the controllability of an ensemble of control systems corresponding to a residual network, and to leverage classical Lie algebraic techniques to characterize controllability. The second contribution is to identify monotonicity as the bridge between controllability of finite ensembles and uniform approximability on compact sets. We describe how these results can play a key role in developing stability guarantees for neural networks in the loop. This is joint work with Paulo Tabuada (UCLA).

Biography

Bahman Gharesifard is currently a Professor with the Electrical & Computer Engineering Department at the University of California, Los Angeles. He was an Associate Professor, from 2019 to 2021, and an Assistant Professor, from 2013 to 2019, with the Department of Mathematics and Statistics at Queen's University. He was an Alexander von Humboldt research fellow with the Institute for Systems Theory and Automatic Control at the University of Stuttgart in 2019-2020. He held postdoctoral positions with the Department of Mechanical and Aerospace Engineering at University of California, San Diego 2009-2012 and with the Coordinated Science Laboratory at the University of Illinois at Urbana-Champaign from 2012- 2013. He received the 2019 CAIMS-PIMS Early Career Award, jointly awarded by the Canadian Applied & Industrial Math Society and the Pacific Institute for the Mathematical Sciences, a Humboldt research fellowship for experienced researchers from the Alexander von Humboldt Foundation in 2019, an NSERC Discovery Accelerator Supplement in 2019, and the SIAG/CST Best SICON Paper Prize 2021. He was a finalist (as an advisor) for the Best Student Paper Award at the American Control Conference in 2017. He received the Engineering and Applied Science First Year Instructor Teaching Award in 2015 and 2017. He has served on the Conference Editorial Board of the IEEE Control Systems Society 2016-2020, and is currently an Associate Editor for the IEEE Control System Letters and IEEE Transactions on Network Control Systems. His research interests include systems and control, distributed control, distributed optimization, machine learning, social and economic networks, game theory, geometric control theory, geometric mechanics, and applied Riemannian geometry.