Navigating The Future Of AI Training: Addressing The Data Shortage Challenge

We can all agree that training data is essential to any machine learning model, but what happens when that data runs out? According to a recent article in The New Scientist, the high-quality language data used to train models such as ChatGPT could run out as soon as 2026.

High-quality language data includes books and scientific papers but is slow and costly to generate. Lower-quality data includes posts on blogs, forums and social media and is plentiful, but machine learning models based on lower-quality data may struggle to make the paradigm-shifting developments seen in machine leaning models recently. Not only is this data shortage likely to slow development, it could also see the cost of training data rocket.