History
What object did Yann LeCun use in his NIPS 2016 keynote to describe the 'layers' of intelligence?
Cake
Proposed by @GokuMohandas
Card 1 of 259. Answer: Cake
Cake
Proposed by @GokuMohandas
Card 1 of 259. Answer: Cake
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
A cake. In his opening keynote at NIPS 2016 in Barcelona, "Predictive Learning", Yann LeCun said that if intelligence is a cake, unsupervised learning is the bulk of the cake, supervised learning the icing, and reinforcement learning the cherry. Each layer's size reflects its share of what a machine learns.
The argument was about information per example, and his slides put rough numbers on it. A reinforcement learning agent gets an occasional scalar reward, a few bits for some samples. A supervised model gets a label, 10 to 10,000 bits per sample. A model that predicts hidden or future parts of its input, such as the next frames of a video, gets millions of bits per sample.
At ISSCC in February 2019, LeCun relabeled the cake self-supervised learning, where the data supplies its own targets. Language models fit the picture: next-token pretraining is the cake, supervised fine-tuning the icing, RLHF the cherry. RL researchers disliked the ranking, and reasoning models trained with large-scale RL have revived the debate.