Today Ankush Garg is talking to Mehdi Bahrami about his recent project: BERT-Sort.
BERT-Sort is an example of how large language models can add useful context to tabular datasets, and to AutoML systems.
Mehdi is a Member of Research Staff at Fujitsu and, as he describes, he began using AutoML systems for his research, yet he came across some crucial limitations of existing solutions. The modifications he made highlight a promising future for the relationship between language models and AutoML. This is a direction we're going to continue to explore on the show.
References:
BERT-Sort: A Zero-shot MLM Semantic Encoder on Ordinal Features for AutoML - https://proceedings.mlr.press/v188/bahrami22a.html
PyTorrent: A Python Library Corpus for Large-scale Language Models: https://arxiv.org/abs/2110.01710
AugmentedCode: Examining the Effects of Natural Language Resources in Code Retrieval Models: https://arxiv.org/abs/2110.08512
Related Episodes
Discovering Temporally-Aware Reinforcement Learning Algorithms
Designing algorithms by hand is hard, so Chris Lu and Matthew Jackson talk about how to meta-learn them for reinforcement learning. Many of the concepts in this episode are interesting to meta-learning approaches as a whole, though: "how expressive can we be and still perform well?", "how can we get...
X Hacking: The Threat of Misguided AutoML
AutoML can be a tool for good, but there are pitfalls along the way. Rahul Sharma and David Selby tell us about how AutoML systems can be used to give us false impressions about explainability metrics of ML systems - maliciously, but also on accident. While this episode isn't talking about a new exc...
Introduction To New Co-Host, Theresa Eimer
In today's episode, we're introducing the very special Theresa Eimer to the show. Theresa will be taking over the hosting of many of the future episodes. Theresa has already recorded multiple episodes and we are stoked to air those shortly.We also spend a few moments explaining my relative absence i...