Title: How much is your training FLOP worth?

Speaker: Dr. Karl Stratos @ Apple

Time : 13:00 - 14:00, Apr 14th, 2026

Location: 

Online: https://hli.skku.edu/InvitedTalk260414 

Language: English speech & English slides

Abstract: 

Training a foundation model relies on predictable scaling laws. We expect X% performance improvement when we increase our compute (i.e., FLOP) budget by Y%. Thus every research effort is, and should be, measured by how much it increases the "FLOP efficiency" across scales. This relatively new (but now completely standard in industry) paradigm is what drives the explosive growth of AI, and is a fundamental departure from the classical (largely academic) mode of research where we solve specific problems at constant scale. In this talk, I will give a high-level overview of the scaling paradigm and its challenges, and give my recent work on second-order optimizers as a concrete example of effort to improve scaling efficiency.

Bio:

Karl Stratos is a researcher at Apple Foundation Models. He obtained a PhD in computer science at Columbia University in 2016 advised by Dr. Michael Collins. He previously worked at Bloomberg LP (2016-2017), Columbia University (2017), Toyota Technological Institute at Chicago (2017-2019), Rutgers University (2019-2024), and Essential AI (2024-2025).