This is the fundamental problem with where the AI race is heading, IMHO. Broadly, there are two possible legal interpretations (to my layman's mind):
* A model is derived work of its training data. This seems sane to me. Open, but copyrighted, works (like FOSS) remain protected from abuse. There's some legal moat around AI models. But on the other hand it seems unlikely that there's enough liberally licensed (or public domain) training data to go around. The little guy's status quo remains, the frontier labs' work slows down massively.
* A model is not derived work of its training data. This seems to me insane, but a lot of the world seems to hold this view (including the frontier labs). Stuff like FOSS or indie art is under huge threat of copyrightwashing. But on the other hand, there's also zero legal moat around the models. The little guy is eviscerated, but so are the frontier labs.
Neither interpretation seems, to me, to be capable of sustaining the last couple of years' developments. But what do I know.
> A model is not derived work of its training data. This seems to me insane, but a lot of the world seems to hold this view
Why insane? Models don't take the content as-is, they take measurements. I don't owe you royalties just because I used your photo to get the proportions and coloring of a duck right. Go watch artist streams, you'll often see people to go Google Images for references. I've never seen that result in credit or payments.
The alternative is that we hand out lots of money to a few large companies specializing in content archives, and there's really no benefit to anyone else anyway. On the long term I would expect a few fat cats to get fatter, the small guy to get nothing, and AI still work but get there slowly. I don't see the point or the benefit.
> Models don't take the content as-is, they take measurements.
At some point, enough measurements constitute a copy. If I redistribute the average value of all the pixels in your photo, I'm obviously not in violation of your copyright. If I measure and redistribute 90% of its DCT coefficients (i.e. make a slightly compressed JPEG), I am.
The interesting stuff happens between those extremes. We cannot just take as a given that all LLMs always are on the safe side. It is not at all obvious.
Yeah but here is the thing. Say you train a Lora for a model. You then merge those weights into the open weight model.
Now prompt it for an original image, it will pretty much be able to reproduce that exact image.
You can say it is just measurements but at some point, it can just reproduce with high enough accuracy to just be seen as a copy
Given that corporations buy up any valuable IP, I personally think the answer is to abolish copyright because right now it is really only protecting the rich and corporations . Individuals have the illusion of protection but if Disney steals your shit, good luck with the pain and suffering you experience trying to win a court case against them
Abolishing copyright would make it much harder for smaller players to protect their interests. Their original work would be gobbled up by the big players, and made easily available under the umbrella of a large corporations pre-existing market share. Not a fun situation at all to be in. Especially for the little guys.
Lemme fix that for you in the style of Accelerando....
This is the fundamentally great thing about where the AI spend is heading, looks like there is no moat and will be a democratization of cognition simply by dint of the way the tech works. Which is awesome.
I shed 1x tear for the private investors and large existing monopolists who poured money into this, only because they might not make the same mistake with the next great technology, but given their greed, that's unlikely. They also might get government backing to mitigate their loss, which would be horrible and inflationary, but given the state of open-weights, that seems unlikely outside of the USA.
If a model is derived work of its training data, surely all the existing frontier models that have been training on copyrighted work have a big legal issue, and therefore so does anything produced with them?
Yes. Of course. They'd have to be rebuilt with acceptably licensed training data. And since there might not be a enough of it, the model owners are screwed.
My point is that they're also screwed in the opposite scenario, because they rely on the same legal protection (against deriving works) as the works they trained on.
That's why I don't understand how any of this can be sustained.
Yes, only a small number of companies produce memory because of capital requirements and complexity. When prices go up both of those things become less of a problem for investors who can see better return for lower risk on that high capital investment. They will then invest in production to capture those returns. This is literally the exact reason Samsung are doubling supply of this particular type of memory.
Memory factories cost billions to tens of billions, "investors" aren't going to spend that kind of money if they are not reasonably certain that they are going to make a profit, and the market is very uncertain right now with everybody bracing for a bubble burst at any moment. It's not the first time there has been such a shortage in the computer parts market, and the oligopoly, be it RAM or hard drives manufacturers, has always managed to keep prices high and production low enough to avoid a price crash after recovery.
No, completely wrong. Supply and demand will definitely fix this.
More HBM4 will drive the price of it down. That will make it less profitable to produce. That will mean firms shift some production back to more profitable "consumer" RAM. Increased supply will lead to lower prices.
What specifically did that say that you think is wrong?
Supply is currently moving the wrong way. Supply and demand for consumer DRAM will get even worse, even if it will "definitely fix this" in the long run.
And the fact that production is moving the wrong way suggests that in the short term supply and demand is actually doing more harm than good. HBM is so profitable it's radioactive.
I don't know where you got the idea in another comment that all memory will get "very cheap quite soon". They're not increasing total memory production. (much/yet)
China is about to massively compete. And doesn't Micron and someone else have new fabs in NY coming online ? I can't imagine they're going to exclusively make HBM.
Ram always goes through these boom and bust cycles. They're going to increase supply but not so much that they're eventually left with huge overproduction as usual.
Of course if Google researchers come up with more ram reduction tricks for newer AI architectures that could make a huge difference as well
I don't expect China to be a particularly large competitor in the next year or two affected by the decision in the article, but they'll help a lot in the long run.
Micron's NY fab had a press release two months ago that they started pouring the first concrete, so that's far from coming online.
If AI demand sticks around and buildouts stay cautious it could take much much longer than usual to build up enough capacity to push prices back down below $3/GB.
More efficient AI memory use risks even more demand for memory because you're now able to get even more performance per dollar; the benefit of extra memory has to be sufficiently far into diminishing returns for demand to drop overall.
> What specifically did that say that you think is wrong?
You said:
> the market doesn't fit the issue.
I presume you meant "fix" not "fit". You are wrong that the market will not fix the issue. There are literally millions of observed cases across many markets of increased demand (or decreased supply) leading to higher prices leading to increased supply leading to lower prices, which is why it's a fundamental law of economics.
I feel like everyone here forgets that the DRAM industry is operated by one big cartel. The rules of supply and demand don't apply here. Now that prices are up, they will keep it up, cutting supply as needed in order to do so. Every big DRAM manufacturer in the industry has been charged with price fixing, among other things: https://en.wikipedia.org/wiki/DRAM_industry_price_fixing
There are so few players in this space they can coordinate and collude pretty easily.
DRAM isn't nearly as hard to get into as compute. The price fixing stuff is going to bite them in the ass someday soon by massively increasing the incentives for new players to enter the market and compete.
Also they're talking about the short to medium term. On that time scale they're not wrong. The market is so far from fixing it right now, it's going in the opposite direction. The market is chasing HBM money which is outbidding everything else.
> The price of RAM will fall.
And if that might take 5+ years, that's not much comfort.
> higher prices leading to increased supply leading to lower prices
So far higher prices are causing barely any supply increases. (Note that the article we're commenting on is about a supply decrease.) Companies are barely trying to expand, and even if they do start trying hard it'll take a very long time.
Yes. Non-HBM Samsung supply going from 60% to 20% will cause prices to increase, which will cause increased production of non-HBM, and then prices will fall (below where they currently are).
From a design and manufacturing point of view you want everything to be identical. Differences are expensive.
From a profit maximization standpoint you want identical phones that can't easily be sold across market borders. Usually that's done in software (e.g. region locked video games), but that can usually be cracked, so a single design difference is the best solution. A great example is Nintendo's cartridge slot shapes that were different in each region despite the hardware being the same.
Apple had lightning everywhere; they were forced to change to USB-C. Therefore they changed to USB-C everywhere. The market protection design change remains the sim slot.
We see this more locally in the US as well. There's something called the California effect. California emission standards were more strict than the rest of the country, but it didn't make sense to make California only vehicles and then vehicles for the rest of the country. So automakers built to the California standards and residents in all of the other states got to benefit from cleaner air as a result.
> From a design and manufacturing point of view you want everything to be identical.
OK, so why not ship a SIM card slot in the US? eSIMs work perfectly well in Europe, the US and most of the rest of the world, "region locking" is clearly not the answer.
Apple did not have "lightning everywhere" either - Macs and iPads got it much earlier.
An important take-away from this is the harness is really important. The graphs show that the same model with a better harness performs many percentage points better. OpenAI are getting into the "selling the harness" game in a big way. Why?
What's interesting about that is while not everybody can train or run a model, anybody can build a harness. You and I can build harnesses.
It seems strange that OpenAI would move into a field where any developer can compete with them. I think that tells us a lot about the economics of training and selling inference.
I don't think it says what you imply. I think they're just trying to vertically integrate.
Harnesses are going to be controlled at companies eventually, just like you might not have a choice of OS. They want to make sure they are the complete package.
"We have dangerous AGI that can destroy humanity."
"Also, all of your sensitive legal documents will be totally safe with us."
"Also, for some reason even though we have AGI and selling tokens is a fine business, we need to sell a new product specifically targeted at a very high margin and lucrative industry."
At the end of the day they're a VC-backed company that has to cash in on their brand reputation. Legal is even higher margin than coding with less discerning buyers.
reply