Events at Physics |
Events on Wednesday, August 26th, 2020
- Physics ∩ ML Seminar
- The large learning rate phase of deep learning
- Time: 11:00 am - 12:00 pm
- Place: Please register for this online event:
- Speaker: Yasaman Bahri, Google Brain
- Abstract: Recent investigations of infinitely-wide deep neural networks have given rise to foundational connections between deep nets, kernels, and Gaussian processes. Nonetheless, there is still a gap to characterizing the dynamics of finite-width neural networks in common optimization settings. I’ll discuss how the choice of learning rate is a crucial factor to be considered and naturally classifies gradient descent dynamics of deep nets into two classes (a ‘lazy’ regime and a ‘catapult’ regime) which are separated by a sharp transition as networks become wider. I’ll discuss the distinct phenomenological signatures of the two phases and how they are elucidated in a class of solvable simple models we analyze.
- Host: Shiu
- Department Meeting
- Time: 12:15 pm - 1:30 pm
- Place: Virtual see "abstract" for connection info
- Speaker: Sridhara Dasu, Department Chair, UW-Madison
- Meeting Coordinates: Meeting number: 120 392 9242 Password: Q5EjaTz3Pk3 (75352893 from phones) Join by video system Dial 1203929242@uwmadison.webex.com You can also dial 173.243.2.68 and enter your meeting number. Join by phone +1-415-655-0001 US Toll +1-312-535-8110 United States Toll (Chicago) Access code: 120 392 9242
- Host: Sridhara Dasu, Department Chair