BEGIN:VCALENDAR
VERSION:2.0
CALSCALE:GREGORIAN
PRODID:UW-Madison-Physics-Events
BEGIN:VEVENT
SEQUENCE:0
UID:UW-Physics-Event-6760
DTSTART:20220105T170000Z
DTEND:20220105T181500Z
DTSTAMP:20260314T131445Z
LAST-MODIFIED:20211217T042904Z
LOCATION:Online Seminar: Please sign up for our mailing list at www.ph
 ysicsmeetsml.org for zoom link
SUMMARY:A self-consistent GP framework far from the GP limit\, Physics
  ∩ ML Seminar\, Gadi Naveh\, Racah Institute of Physics\, Hebrew Uni
 versity
DESCRIPTION:Recently\, the infinite-width limit of deep neural network
 s (DNNs) has garnered much attention\, since it provides a clear theor
 etical window into deep learning via mappings to Gaussian processes (G
 Ps). In spite of its theoretical appeal\, this perspective lacks a key
  component of finite DNNs\, that is at the core of their success - fea
 ture learning. Here we consider DNNs trained with noisy gradient desce
 nt on a large training set and derive a self-consistent Gaussian Proce
 ss theory accounting for strong finite-DNN and feature learning effect
 s. We apply this theory to two toy models and find excellent agreement
  with experiments. We further identify\, both analytically and numeric
 ally\, a sharp transition between a feature learning regime and a lazy
  learning regime in one of these models. We have numerical evidence de
 monstrating that the assumptions required for our theory hold true in 
 more realistic settings (Myrtle5 CNN trained on CIFAR-10).  <br>
\nLin
 k to paper: https://openreview.net/forum?id=vBYwwBxVcsE
URL:https://www.physics.wisc.edu/events/?id=6760
END:VEVENT
END:VCALENDAR