Artificial intelligence has read everything on the internet, but remains hungry for more data » TwistedSifter

Those in power have done their best to convince us that AI technology is nothing like what we have seen in science fiction films and that there is nothing to fear.

When you hear that it has eaten everything we have and is still hungry for more, well…the parallels draw themselves.

AI companies worry that as they build bigger and better models, there won’t be enough data available on the internet to train them.

Some companies are looking for alternative sources of data training, with things like video transcriptions and “synthetic data” on the list.

The latter is generated by AI, and no one knows what will happen if we let it train itself.

I have to think it’s nothing good.

Early research agrees that training an AI model on generated AI data would eventually lead to ‘model collapse’.

Some companies have claimed they can create higher quality synthetic data, but haven’t disclosed what that would actually look like.

One company, Dataology (founded by ex-Meta and Google DeepMind researcher Ari Morcos), is among those looking for ways to train bigger and smarter models with less data.

These are largely controversial methods of data training, such as transcriptions of public YouTube videos.

Researchers have been eyeing the specter of a data shortage caused by AI for some time now. Pablo Villalobos estimates that AI will run out of useful data within a year or two, but he doesn’t seem concerned.

“The biggest uncertainty is what breakthrough you will see.”

Or, you know, companies could stop creating those bigger and better models because of the training data storage issue – along with other issues, like excessive energy consumption.

But I really don’t see that happening.

If you liked that story, check out what happened when a guy gave ChatGPT $100 to make as much money as possible, and it turned out exactly as you’d expect.