Events


 
<< August 2020 >>
 
 >>
 >>
 >>
 >>
 >>
 >>
Sun Mon Tue Wed Thu Fri Sat
   1 
 2   3   4   5   6   7   8 
 9   10   11   12   13   14   15 
 16   17   18   19   20   21   22 
 23   24   25   26   27   28   29 
 30   31   
 
Add an Event Edit This Event

Events at Physics

<< Summer 2020 Fall 2020 Spring 2021 >>
Subscribe your calendar or receive email announcements of events
Physics ∩ ML Seminar
The large learning rate phase of deep learning
Date: Wednesday, August 26th
Time: 11:00 am
Place: Please register for this online event: http://physicsmeetsml.org
Speaker: Yasaman Bahri, Google Brain
Abstract: Recent investigations of infinitely-wide deep neural networks have given rise to foundational connections between deep nets, kernels, and Gaussian processes. Nonetheless, there is still a gap to characterizing the dynamics of finite-width neural networks in common optimization settings. I’ll discuss how the choice of learning rate is a crucial factor to be considered and naturally classifies gradient descent dynamics of deep nets into two classes (a ‘lazy’ regime and a ‘catapult’ regime) which are separated by a sharp transition as networks become wider. I’ll discuss the distinct phenomenological signatures of the two phases and how they are elucidated in a class of solvable simple models we analyze.
Host: Shiu
Add this event to your calendar