Join us for talks from new faculty on emerging AI topics, including optimizing neural nets, large language models, and generative AI for 3D scenes.
- 10:00-10:30am Check-in
- 10:30-10:35am Opening Remarks
- 10:35-11:00am "Explorations of Optimization and Generalization for Neural Nets"
- 11:00-11:25am "An Adventure Inside a Large Language Model"
- 11:25-11:50am "Generative Models in Three Dimensions"
- 11:50-12:15pm Q&A panel with all speakers
Micah Goldblum:
"Explorations of Optimization and Generalization for Neural Nets"
The fundamental building blocks of machine learning are (1) optimization or fitting the training data and (2) generalization or extrapolating from the training data to unseen samples. In this talk, we will explore both of these topics in the context of neural networks. First, we will discuss recent work on stabilizing optimization for language models. We show that vanilla SGD without momentum is nearly as fast as Adam in the small-batch regime. Then, we will discuss work on how to build neural networks that can generalize to far more complex problems than they were trained on by “thinking” for longer.
John Hewitt:
"An Adventure Inside a Large Language Model"
The behavior of large language models---like those underlying ChatGPT---is not programmed; it's the emergent result of centuries of computation and an internet's worth of text. Understanding what goes on inside these language models is both one of the most exciting scientific opportunities, and one of the most pressing engineering challenges of our time. In this work, I'll provide an overview of this problem, and then discuss work my colleagues and I have done to discover what goes on inside these systems: do they learn the structure of human languages? How can we discover what concepts or ideas they have arrived at, which humans might not have thought of yet? And how can we make precision fixes for their problems?
Aleksander Holynski:
"Generative Models in Three Dimensions"
Generative models aren’t magic; they’re learned from massive datasets. For language and images, this data is abundant; but what happens when no such dataset exists, like for the 3D world around us? Unlike the sea of pixels on the internet, datasets capturing the geometry and physics of 3D space are incredibly scarce. However, the capabilities and models that this data could enable are a critical component for the next generation of AI. So, how can we build spatial generative models without an internet-scale dataset of the physical world? I'll explore several novel strategies to overcome this data bottleneck, discussing work on converting videos to 3D, learning from implicit 3D signals, and training on synthetic data; as well as applications, like immersive virtual reality to complex problem-solving, that highlight the critical need for models that can reason about 3D space.