To better understand how neural networks learn to simulate writing, researchers trained simpler versions on synthetic children’s stories. The post Tiny Language Models Come of Age first appeared on Quanta Magazine
Learning English is no easy task, as countless students well know. But when the student is a computer, one approach works surprisingly well: Simply feed mountains of text from the internet to a giant mathematical model called a neural network. That’s the operating principle behind generative language models like OpenAI’s ChatGPT, whose ability to converse coherently (if not always truthfully) on a…