Events

Physics ∩ ML Seminars

<< Summer 2020 Fall 2020 Spring 2021 >>
Subscribe your calendar or receive email announcements of events

Events During the Week of August 23rd through August 30th, 2020

Monday, August 24th, 2020

No events scheduled

Tuesday, August 25th, 2020

No events scheduled

Wednesday, August 26th, 2020

The large learning rate phase of deep learning
Time: 11:00 am - 12:00 pm
Place: Please register for this online event:
Speaker: Yasaman Bahri, Google Brain
Abstract: Recent investigations of infinitely-wide deep neural networks have given rise to foundational connections between deep nets, kernels, and Gaussian processes. Nonetheless, there is still a gap to characterizing the dynamics of finite-width neural networks in common optimization settings. I’ll discuss how the choice of learning rate is a crucial factor to be considered and naturally classifies gradient descent dynamics of deep nets into two classes (a ‘lazy’ regime and a ‘catapult’ regime) which are separated by a sharp transition as networks become wider. I’ll discuss the distinct phenomenological signatures of the two phases and how they are elucidated in a class of solvable simple models we analyze.
Host: Shiu
Add this event to your calendar

Thursday, August 27th, 2020

No events scheduled

Friday, August 28th, 2020

No events scheduled