I feel like many of Anthropic's issues are due to Dario being too earnest and open. It seems both refreshing (that a CEO has thought deeply about and is willing to talk publicly about the dangers of their product) and depressing (that so many people cynically think this is some sort of marketing ploy).
I couldn't help but notice how each successive headline reporting our glorious victories seemed to draw closer to Tokyo.
Something like that.
Well. I can't help but notice how each successive headline reporting how this scam/stochastic parrot/"scare quotes intelligence" seems to be solving more and more things that were but a few years ago widely regarded as being indicators of high intelligence.
Year 3 of being told my job will be replaced by AI, and the only thing that's happened so far is that AI vendors keep showing up to my office, begging me to pay them to use it is a tool.
If you're only seeing the charts go up - you're not looking in the right places.
I'm looking at the millennium puzzles, and independently of those puzzles I had asked it for a fluid dynamics simulation engine that runs in my browser, and it put one together for me so I could play with aerospikes and watch the formation of Mach diamonds in rocket engines. The isochrone map generator has also been stuck on my to-do list for years, and yet now thanks to Claude, I have it, and it's real-time and multimodal.
Even before then, the free trials of Claude Code and Codex at the end of last year and start of this year… despite their flaws, they could write all the code I've ever been paid to write. Sure, limited speedup, Amdahl's law and coding is not the only part of the job, but anyone who was fine at PM and QA but not code no longer needs a coder.
Even before then, I looked at the maths puzzles they were doing well at, and I found I did not understand the questions let alone the answers.
I remember when the ability to generate music and art was "uniquely human", and sure there's a lot of cringe there with those models, but they're also winning awards and causing controversy by doing so, and artists are losing clients; I remember when the board game Go was considered to require "human intuition we could never make a computer solve, totally different to chess" (and I remember when chess was so, too).
When I was a kid, cheques and letters on addresses often got read by a human; the OCR which automated this is also AI, though these days image-to-numbers is the "hello world" of the field.
>Even before then, I looked at the maths puzzles they were doing well at, and I found I did not understand the questions let alone the answers.
okay, and?
>Even before then, the free trials of Claude Code and Codex at the end of last year and start of this year… despite their flaws, they could write all the code I've ever been paid to write.
Boy, programmers sure do think programming is like the only thing in the world
I'll repeat it for you again:
If you're only seeing the charts go up - you're not looking in the right places.
They're not only useful for programming, it's just that programming is by itself a trillion dollar a year industry. This, by itself, is sufficient to not be the scam you assert it to be.
They may not work for your industry, but then again I don't know what your industry is.
Bluntly, over my lifetime, I've heard "AI will never/not in my lifetime do X" repeatedly within a year of it doing X, for many different X. It's good at getting good. The most recent one being "make useful contributions to millennium prize maths problems". A year or two before that, it was even "do well on degree-level economics essays".
Artists are unhappy, not only because it rips off their work, but because businesses that were previously hiring them now use it instead. This is much smaller (strictly in terms of money) than with software, but is also measurable.
Now, if you said Tesla's self-driving cars (another AI) are a trillion dollar scam, that I would even agree with. At least, for the market cap part, for actual sales it's more like a billion dollar (ish) scam.
>They're not only useful for programming, it's just that programming is by itself a trillion dollar a year industry. This, by itself, is sufficient to not be the scam you assert it to be.
Yeah, if you think I'm being overly literal. Regardless, people here were positing, yourself included, that they are widely useful.
> Bluntly, over my lifetime, I've heard "AI will never/not in my lifetime do X" repeatedly within a year of it doing X, for many different X. It's
Again, over my lifetime, I've seen it been told to me that the next iteration of each model will surely be the one that does away with my entire profession. Yet, again, here we are, firms at my door, begging me to use their tools.
>Artists are unhappy, not only because it rips off their work, but because businesses that were previously hiring them now use it instead.
No, they are unhappy because it rips off their work. You're totally wrong about that.
It's exactly the opposite. Anthropic has earned their terrible reputation through years of lying, deceit, misdirection, gaslighting, unethical marketing strategies, etc.
I remember it vividly, when Anthropic first came on the scene, people (myself included) were incredibly optimistic about them and their leadership. Everyone hated SamA and OpenAI because they felt they couldn't be trusted.
Then slowly but surely, they showed their true colours. Now their reputation is in shambles due to their own behavior, and people are rooting for OAI to beat them. OAI's reputation gains have purely been a result of NOT following in the footsteps of Anthropic.
Easier said than done, but if you water-cool the GPU just upstream of your domestic water heater, it wouldn't be a bad thing. Perhaps using coolant and a counterflow heat exchanger, rather than the potable water, but the point stands. Would just need a secondary tank as a buffer (able to soak up heat from the GPU at all times, even when there's no demand for hot water) which then flushes in when demand exists.
Let's say your GPU uses a constant 300W. It takes approximately 1.16 Wh to heat a liter of water 1 degree celsius under 100% efficiency. This setup won't be 100% efficient, so let's round up to 1.5 Wh per liter per degree (roughly 80% efficient). A typical domestic water heater in the US 40-50 gallons, so roughly 160 liters.
Let's say your incoming water temperature is 18C and you want it preheated to 50C, which is 32C degree differential, which means you'll need 32 * 1.5 * 160 = 7680 Wh, or 25 hours straight to heat the buffer tank from scratch.
You'll need to purchase a small water-to-water heat exchanger ($50), two pumps ($100 each), a power supply for said pumps, hose and/or copper pipe and fittings, and various other sundries, plus the cost of a buffer tank ($600ish), so figure all in roughly $1000, plus the cost of electricity to run the pumps.
At $0.22/kWh you're saving roughly $450 a year with this setup in foregone water heating, but because it's not 100% efficient you're spending $500 in electricity to run your GPU 24/7/365, and that's the maximum you can possibly save with the above assumptions. Scale up for more GPUs and down accordingly for less usage as you see fit.
Alternatively, use air as the heat conductor by placing the GPU laden machine in the same space as a hybrid heat pump water heater.
Just recycle an old water heater as the tank? I think your cost estimate is way overblown but it would certainly eat up a lot of time and effort to DIY. Regardless there's a much bigger flaw with this plan. Water tanks need to be kept above or below certain temperatures. The "preheated" range being described here is distinctly unsafe due to various microorganisms IIUC.
I'm not sure this is the come-uppance you've been waiting for. Oracle still has a larger market cap than at any point in history except the last ~2 years.
It would require government permission. The top labs would be agreeing not to compete on model advancements for a period of time, which seems like a pretty clear violation of antitrust law.
How is this a clear violation of antitrust law? And have any of the leaders of these companies said that they are asking for government permission for this reason?
It's the classic case of a cartel conspiring to limit competition. Here's Matt Levine's explanation [1]:
> 1. Anthropic, OpenAI, and perhaps a couple of other frontier labs are the dominant providers of frontier AI models.
> 2. They can charge customers a lot of money for using their frontier models, and rather less money for older, no-longer-cutting-edge models.
> 3. Training a new frontier model requires ever-increasing billions of dollars of computing power.
> 4. The labs need to more or less continuously race to build new frontier models, because their competitors are all doing it, and if they don’t they will fall behind and no longer be able to charge a lot of money for their best models. (Also because they intrinsically want to build artificial superintelligence, for cancer-curing and/or killing-everyone reasons.)
> 5. If they collectively slowed down, then (1) they’d spend less on compute and (2) they’d be able to charge frontier-model prices for a longer time.
> 6. But if one of them slowed down, the others would eat its lunch.
> 7. If they got together in a room and agreed to slow down, that would look like an antitrust conspiracy: It is generally illegal for competitors to get together and agree to limit the output of their industry.
> 8. But if they publish papers about how important it is to slow down, that might have a similar coordinating function, at least among the US frontier labs if not necessarily among their Chinese competitors.
> 9. And if the government believes those papers, it might help them coordinate. Maybe the government will impose pacing by regulation that the labs could not impose by agreement. Or at least the government will let them get together and agree to slow down. Amodei’s post calls for “frontier AI companies within democratic countries [to] coordinate to establish common safety standards as well as limits on the rate of unchecked AI progress”; a footnote adds: “With government mediation or waivers of antitrust restrictions.” Just meeting in a room to establish common safety standards is legally risky; the labs can’t do it on their own unless governments affirmatively allow it.
The entire essay is about stopping development until it can be made more safe, with specific ideas on how to do that.
The fact that he would suggest this despite preparing for an IPO is an even stronger signal that it's in good faith -- it will almost certainly delay or reduce the valuation of the IPO.
> The fact that he would suggest this despite preparing for an IPO is an even stronger signal that it's in good faith -- it will almost certainly delay or reduce the valuation of the IPO.
No it won't.
This is a call for a "safety cartel"[1] in which the dominant firms become more entrenched against competition by colluding to limit the progress would-be competitors can make in the market.
> The entire essay is about stopping development until it can be made more safe, with specific ideas on how to do that.
No, the essay does not talk about "stopping development". That's a very clever sleight of hand made in the essay to get readers to draw this false conclusion.
It just talks about building AI at a "balanced rate". Untangling the corporate speak, this amounts to essentially "go full steam ahead, but have more eval oversight before release".
More likely, this is being done to comply with SB 1119 (Adam's Law) [1,2] and similar legislation. These create extra steps that a service has to go through before releasing a new model that minors can interact with and exposes them to prosecution if a minor is harmed or harms someone else due to their interaction with a chatbot. It's safer for Anthropic to just exclude minors -- and they're unlikely to be paying customers anyway.
Thank you, I'm glad someone brought up SB 1119. I hate it.
California already has BPC §§ 22601-22606 to protect people from chatbots encouraging self-harm and I don't see a legitimate reason why they had to ratchet up the restrictions.
> Waymo says it has 170 million miles and I found 2 fatal accidents it was involved in (fault doesn't matter as fault is not included in the other stat either)
I suppose you can claim fault doesn't matter. But if you read the narratives of both fatal incidents, they are so clearly not the Waymo's fault that I don't see how you can hold it against them.
In the first [1], the Waymo was stopped at a red light when the car behind it was rear-ended by an SUV traveling at high speed. Both that car and the SUV then struck the Waymo. The SUV hit a total of 6 cars and someone in one of the other cars died. According to the police, the SUV may also have been involved in "multiple hit-and-run accidents reported just minutes before" [2].
In the second [3], the Waymo was yielding to a pedestrian when a motorcyclist made contact with the left rear corner of the Waymo. The motorcyclist fell to the ground to the left side of the Waymo, then another car tried to pass the Waymo and struck the motorcyclist, killing them. That other driver fled the scene.
If you drive on urban streets, no matter how good of a driver you are, there's a chance you're going to be involved in some sort of incident that's completely out of your control -- that seems to be what both of these are.
I agree that these cases are ridiculous to blame on the waymo entirely, but to give logic to their reasoning here you're unsure if on average the 1.3 fatalities per 100 million miles (FPMM) may also be on average ridiculously not the drivers fault as well. Perhaps if you were to throw out all ridiculous cases you'd have an average of CDL's having a 0.3 FPMM vs Waymo's having 0 FPMM.
The nuance you describe at the end is the better interpretation: fatalities per mile driven is a useless metric since it's such a rare occurance and incredibly circumstantial that it cannot be used as a meaningful comparison metric for safety.
It also gets better depending on the medical services nearby and there competence. So having more competent medical services in cities, lowers murder statistics and car-fatalities. Thus the actual accident rate would only be visible in the countryside.
If we want to get closer to the "real" number, we would also need to account for mechanic failures of the car, wildlife, and suicide. We don't usually attribute people jumping in front of trains as the fault of the person driving the train. Fatalities per mile driven may very well be biased in favor of waymo that has nothing to do with safety of drivers, which in turn create uncertainty when trying to estimate the statistical result of making all cars Autonomous. For example, car owners can have a higher variance in maintenance than waymo, they can use the car in areas with more wildlife than city traffic, and they can commit suicide through other means than personal being behind the wheel (suicide statistics is itself quite complicated).
Um, in those two examples where it was ridiculously not waymo's fault it was still absolutely the fault of a human driver. I'm struggling to imagine a significant amount of cases where it's ridiculously not the fault of any human driver.
Thats why he specifically called out CDL drivers, or in another part of this thread they talked about rideshare drivers. Certain segments of humans likely approach the same rates as the driverless cars.
The commonly cited fatality statistics do not discuss fault, so, absent more precise information, a apples to apples comparison should not conditionalize on fault.
Why? Well let us try to imagine a world where Waymo has the statistically average fatal crash rate, yet is never at fault. In this world, every fatal crash occurs at the Waymos. You just have a bunch of crazy drivers roving the streets causing fatal crashes against (or involving) unsuspecting, not-at-fault victim Waymos.
If that is the world, then are these crazy drivers only crashing into Waymos? Probably not, these crazy drivers would almost certainly be crashing into a statistically average distribution of unsuspecting, not-at-fault victims.
So this imagined world is consistent with the overwhelming majority of human drivers engaging in 0 fatal accidents at fault with a small, crazy roving subset being at fault for all fatal accidents. So, in that world, a Waymo that is only ask good as the statically average driver would actually be infinitely worse than the overwhelming majority of human drivers. If you only replaced the 10th percentile driver (90% of drivers are better), you would actually be increasing the fatality rate. Only by displacing that crazy subset would you actually be decreasing the fatality rate.
Are we in that world? The Waymo numbers on fatal accident involvement with no fault actually kind of point in that direction (though the mileage numbers are still too low to make a strong conclusion). Seemingly massive crash and injury reduction due to massive reduction of at-fault crashes, but the fatality reduction does not seem proportional. Another possibility is that the Waymo driving distribution is biased toward being in situation that result in more no-fault fatal crashes.
But, that is all just conjecture and thought experiments. Unless we have better data, we unfortunately should be restricting ourselves to apples to apples comparisons which, in this case, do not conditionalize on fault.
Though, to nitpick on the original post, I think that the 2 fatal crashes in 170 million miles (~85 million miles per fatal crash which is similar to the USA human average of ~83 million miles per fatality), is probably not the correct analysis. The average fatal crash is probably 2 vehicles and 1 fatality. So, the 85 million miles per fatal crash versus 83 million miles per fatality is probably fine in the crash vs fatality direction. But I am pretty certain you would actually need to double the 170 million to account for the "statistically average mileage of the other cars".
Otherwise, if we have exactly two cars in the world each with 100 million miles and they engage in one fatal crash with one fatality, then we would conclude that car 1 has 1 fatality per 100 million miles and car 2 also has 1 fatality per 100 million miles. Naively averaging them, we could conclude that the overall fatality rate is 1 per 100 million miles. Therefore, over the 200 million miles the two of them drove, there were 2 fatalitys. Obviously incorrect, so we almost certainly need to weight them by their relative proportion when aggregating them.
It wasn't the coverup of the crash. It's the fact that they massively exaggerated their capabilities.
For example, the reason they started with night trips was that their car could not differentiate between kids and adults so they preferred to start operation when kids are less likely to be presetn.
> The motorcyclist fell to the ground to the left side of the Waymo, then another car tried to pass the Waymo and struck the motorcyclist, killing them. That other driver fled the scene.
That second accident looks like it could be one on those typical cases of SDC causing accidents by behaving unexpectedly (like braking for a pedestrian where human drivers would have denied right of way)
I havent read the doc and I am not arguing either way, just saying that you can cause an accident in other ways than by ploughing into someone.
They really can't stop tying themselves into knots to dunk on self driving cars. They don't even know it's the case, they just suspect it's the case. Even if it _were_ the case, the drivers behind them most likely don't have this information available, they probably can't distinguish between "SDV being cautions and giving right of way" and "SDV stopping an immediate impact", they can just see a car in front of them suddnely stopping. So they should stop.
If the pedestrian hasn't gotten to the driveway quite yet, and you're stopping to let them get there and cross out of an abundance of caution, then it may not have been legally required for you to yield. And the motorcyclist behind you could very reasonably have seen the pedestrian too (and reasoned that the pedestrian was still several seconds away from getting to the driveway) and not expected you to stop. In such a situation, such an abundance of caution could have easily "caused" an accident, in that not yielding and proceeding through would have been perfectly safe.
(I'm of course being steel-manning GP's argument here, it could have very easily been that the pedestrian was already crossing the driveway at that point, I didn't see the video. Point is, "more caution" doesn't always mean "less chance of an accident".)
> Point is, "more caution" doesn't always mean "less chance of an accident"
that's a fair point. For example, if you're on a highway and traffic is moving at 80mph however the speed limit is 55mph and so you're driving 55 you're doing way more harm than good. Becoming an obstacle to traffic makes an accident more likely regardless of whether you're right or wrong.
Having said that, I do not see the waymo at fault here at all. The other car driver turned a minor accident into a fatality without involving the waymo, pedestrian, or even the motorcyclist. They were just not paying attention and ran over the poor person.
This is exactly the type of nuance that the comparison fails on. Plus we have limited info here.
Yes ofc we must yield to pedestrians. And we also make nuanced decisions. "There's a motorcycle moving too fast behind me. There's a pedestrian in front of me."
If the pedestrian would be killed or struck by the vehicle, then yes you need to stop and/or steer away. But if it's about merely inconveniencing the pedestrian to prevent an accident, that's the ethical duty.
That's the nuance the rule "always yield to pedestrians" needs to consider.
Hi. This position is utterly idiotic, even if you were just trying to steel-man it. If a self-driving car can "cause" accidents by yielding to crosswalks then the only sensible course of action is to immediately ban human drivers in cities, starting this morning, zero exceptions, no tolerance.
> No modern LLM can "reason" - otherwise you could give it a task like "make me money", it would ask you all the questions it needs about information that it doesn't know about and needs to know to make you money, then it would set whatever it needs to set up to make you money.
If that's your bar for reasoning, then most people can't reason either.
If the model includes unique data from a person then that person can identify the data - the allegedly plagiarised material - and so re-identify it. There doesn't need to be a privacy breach to close that loop as it requires the person to identify the information is associated with them first.
reply