The Chinchilla scaling paper by Hoffmann et al. has been highly influential in the language modeling community. We tried to replicate a key part of their work and discovered discrepancies. Here's what we found. (1/9)

I'm working on replicating a widely-cited ML paper and I'm finding serious issues. If you have experience with dealing with these types of things/know what best-practices are, please DM me.

If it's possible to continue to trade off inference and training compute, we should expect similar amounts of compute to be spent on training large models and on running inference with them. epochai.org/blog/opt…

A recent paper asseses whether AI could cause explosive growth and suggests no. It's good to have other economists seriously engage with the arguments that suggest that AI that substitutes for humans could accelerate growth, right?

The survey of thousands of AI experts shows that it is believed that the falling cost of computation was most important for AI progress in the past ten years. Increased funding and progress in AI algorithms were about on par. This seems mistaken.