Today we're talking with Nick Erickson from AutoGluon.
We discuss AutoGluon's fascinating origin story, its unique point of view, the science and engineering behind some of its unique contributions, Amazon's Machine Learning University, AutoGluon's multi-layer stack ensembler in all its detail, their feature preprocessing pipeline, their feature type inference, their adaptive approach to early stopping, controlling for inference speeds, the different multi-modal architectures, the ML culture at Amazon, the unique challenges of time series, the role of competitions, the decision to reject hyperparameter optimization, benchmarking in AutoML, what the research community can do to help industry along, AutoGluon's relationship with pre-trained tabular models like Tab-PFN, whether the rise of LLMs is likely to affect AutoGluon, what's stopping more people from adopting AutoML solutions, AutoGluon Cloud, the dream and reality of an auto-benchmarking tool, how to contribute to their project, and many, many other topics.
This was one of my favorite episodes. Nick, thank you for joining!
You can follow Nick on Twitter here: @innixma.
And you can follow AutoGluon on GitHub here: https://github.com/autogluon.
Some more resources on AutoGluon:
- The original AutoGluon Paper: "AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data": https://arxiv.org/abs/2003.06505
- AutoML Fall School 2022 AutoGluon presentation, a good way to understand the philosophy behind AutoGluon: https://www.youtube.com/watch?v=VAAITEds-28
- AutoGluon multi-modal paper: https://dl.acm.org/doi/abs/10.1145/3534678.3542616
Related Episodes
Discovering Temporally-Aware Reinforcement Learning Algorithms
Designing algorithms by hand is hard, so Chris Lu and Matthew Jackson talk about how to meta-learn them for reinforcement learning. Many of the concepts in this episode are interesting to meta-learning approaches as a whole, though: "how expressive can we be and still perform well?", "how can we get...
X Hacking: The Threat of Misguided AutoML
AutoML can be a tool for good, but there are pitfalls along the way. Rahul Sharma and David Selby tell us about how AutoML systems can be used to give us false impressions about explainability metrics of ML systems - maliciously, but also on accident. While this episode isn't talking about a new exc...
Introduction To New Co-Host, Theresa Eimer
In today's episode, we're introducing the very special Theresa Eimer to the show. Theresa will be taking over the hosting of many of the future episodes. Theresa has already recorded multiple episodes and we are stoked to air those shortly.We also spend a few moments explaining my relative absence i...