The in-context learner of the "beautiful @GoogleResearch paper" is a meta learner like @HochreiterSepp's 2001 meta LSTM [1] which learned by gradient descent (GD) a learning algorithm that outperformed GD - no test time weight changes! Since 1992, GD can learn learning algorithms https://t.co/53whqQRqW4