The problem is that training a free and open source model costs just as much as training a closed one, but has even fewer potential avenues for recouping that investment. The money still has to come from somewhere.

I'm not sure if open weights are immune to being compromised by ads anyway, they can't serve pay-per-impression ads on the output side, but there's nothing stopping the creator from accepting funding in exchange for biasing the training one way or another.

Coming soon: Foobar-600B, a new SOTA open weight model kindly sponsored by Coca Cola, Exxon Mobil and the Heritage Foundation. Please pay no attention to the men behind the curtain.

I'm not sure about that. Reports have shown that models from China or Mistral can achieve 80% or more of OpenAI's performance for a fraction of the cost.

If you're tucked in right behind the absolute frontier models, the economics change completely

I would laugh my ass off if Coca Cola Company ends up being the company that solves alignment - so that it can align an "open weight" AI with its corporate interests.

Without that though? Our ability to manipulate LLMs is so shaky I would be really surprised if anyone managed to pull off this kind of model manipulation and have it remain undetected.

I almost believed that they just did, they aren't without their share of quirky and unusual projects and sponsorships.

Just wait until someone leaks an internal SOTA model. Would be deeply ironic given how much AI robber barons ‘respect’ others’ copyright and trade secrets.

What is a free model worth if it’s running on another company’s server farm, trained with data you do not have access to?

That is literally the thing the parent poster wants to avoid by running open models.

[edit] I was a little unfair -- lack of access to training data is a bit of an issue (perhaps moreso for analysis than for for actual use, considering what it takes to train these models). I'm thankful that some of them are also distributed as base models, which should be relatively unbiased compared to what happens later during finetuning.

Run them on what though?

Three power supplies, an old server, a grocery cart and a box fan, and every 3090 you and your friends can get your hands on.

I want models I can run on my machine.

I agree, but what about the training data that goes into it (intentional poisoning of the training data, for a variety of reasons, $, power, etc.)

To run your own chatgpt level model would require half a million bucks in infrastructure.

I’m wondering how long it will be until they are also “sponsored” to have ad content trained in. I personally despise advertising but nobody is building these things out of the goodness of their heart. There needs to be some ongoing incentive to train and release open models.

Similarly, I’m wondering when huggingface is going to need to start showing returns and starts putting ads into transformers etc.