Looking forward to this short workshop just getting started now:

Reflecting on Power and AI: The Case of GPT-3


OpenAI started in 2015 as a non-profit to help insure that there is viable open AI tech.

But in 2019 OpenAI needed more compute power and staff so they incorporated as part of Microsoft.

Here is a screen cap of the pricing model for their closed API (shared as part of this talk).

Ffrom the GPT-3 paper papers.nips.cc/paper/2020/file

"In collecting training data for GPT-3, we used the unfiltered distribution of languages reflected in internet text datasets (primarily Common Crawl)"

For those that are interested in why web archives matter, this is very significant.

Maybe someone has made something of this already, but it is strange to consider how models like GPT-3 will be trained on their own output, as they are used to generate text on the web, and then collected by projects like commoncrawl.


@edsu This won't be (isn't) a problem for models that have become self-aware.

Sign in to participate in the conversation

chaos.social – a Fediverse instance for & by the Chaos community