Right? This is going to be an interesting story to follow.
On the one hand, they don’t want to IPO right now, probably because they don’t think investors are keen to buy at the price they’re asking.
On the other hand, I don’t really see things getting better for them as long as cheap open-weight models that are almost as good as Anthropic’s exist. As more people figure this out, I think it’s going to get worse not better. So I don’t think they can afford to wait long.
I think it’s important to call out that this isn’t an issue with AI being brilliant at hacking. It’s that these systems are basically totally unsecured, and the main protection is that no one with the technical skill is bothering to try to get access.
This isn’t to say this isn’t a serious problem. We’re moving from a world where you need a trained security expert to spend non-trivial time to get access to these systems to one where anyone with access to an un-restricted agentic LLM can do this trivially and quickly.
As a civilization, we need to start taking security seriously.
GPL (and copyleft generally) gets way more attention than it should. In the wild, MIT and similar permissive licenses are by far the most common. Linux is on GPL, but that may be a historical artifact as much as anything. I don’t know if Linus himself would choose it if he were making the choice today.
The wider FOSS community, in my experience, leans permissive rather than copyleft. The copyleft people are just loud.
EDIT: To be clear, when I say I don’t know what choice Linus would make today, I don’t mean that I’m skeptical he’d choose GPL. I literally just mean that I don’t know. He’s been quite clear that he’s not a principled believer in copyleft, but it does have real advantages for a project like the Linux kernel.
“The outputs of this opt-in vulnerability scanner will be fully model-generated, without human review or triage,” Anthropic explained. “This will enable faster and more frequent scanning, but means that it is possible reports will be incorrect or invalid.”
When I saw the headline, I was wondering about this specifically. This may make this service not super useful.
My experience with AI security reviews is that they’re fantastic at finding faults, but they always find a list of things to complain about. If there are no real/serious faults they’ll start finding things that kind of have the same shape as a security issue, but really aren’t if you dig into them. I’ve regularly had an LLM generate a list of 10-15 issues ranging in severity from “nits” to “critical” where none of them were actual issues.
Periodic reviews seem like they could get annoying really quickly, becoming more of a maintenance burden than a help.
In his “defense,” the plan to sanction the ICC came before the Nobel. Some of the reporting even claims that the Nobel committee chose to give the prize as a response to the threatened sanctions.
He’s sanctioning the ICC because they’ve stood up to him in the past on issues like Netanyahu.
I don’t?
No one does anything well when they don’t want to do it.
The real solution here isn’t to try to figure out how to force yourself to do something you don’t want to, it’s to figure out how to want to study. The people who can study a broad range of things effectively are the ones who think of themselves as good at learning, and want to understand as much possible. For most of them, part of their self story is that they’re good at learning, and understand things. That can be part of your self story too if you choose it.
I definitely don’t want to pretend that’s an easy change to make. Motivation is one of the harder things. I would like to be able to tell you how to be motivated, but I can’t. But if you’re looking for the difference between you and the ones who seem to study effortlessly, that’s it. They’re not smarter or more capable on any fundamental level. At least not at a level that matters for most things. They just actually want to do it.
Every time I think we’ve hit maximum “cartoon villain” with Trump, I find out I was wrong.
Retractions do happen regularly. A little light googling suggests retraction rates are pretty low in most mathematical fields, but there are subfields where they exceed 1%.
I don’t know if the <0.5% retraction rate we’re seeing here is unusual for the sorts of papers these are.
And that’s kind of my point. I read the article and I still have no idea. I doubt the author of the article knows either. Which makes me think that they had an agenda going into the whole affair.
Domain pricing doesn’t really vary that much over most of the sites. The price is usually based on the specific domain. .com domains are usually pretty pricey. Some TLDs are very cheap.
There are definitely issues with what OpenAI is doing. I’m the last to defend them.
That said, a small correction rate after uploading a bunch of preprints just doesn’t seem like news. An article that headlines with that, but doesn’t even bother to tell us what the normal rate of corrections and withdrawals is for humans really feels like propaganda to me.
I would really like to live in a world where we don’t reward this sort of “journalism” with attention.
I really love the simplicity and security focus of Porkbun. It feels targetted at engineers, not wannabe startup CEOs.
Ohh. Yeah. It’s a concern.
People said the same things about TV. I don’t know that they were wrong.
But it is here to stay, and I think it will do more good than harm on balance. The sheer access to information LLMs can provide to someone who knows how to sort fact from hallucination is staggering.
But, yes. I do share your concerns. My view is that, at this point, people who don’t make an effort to exercise their minds are already a lost cause. I’m not sure how much worse LLMs can make it.
I think LLMs are going to cause massive change, not all of it good.
Ultimately, I’m an optimist. I think in the long run, LLMs are going to enable us to do amazing things that we wouldn’t have been able to do without them. In the short term… I dunno. It’s going to be a rough transition.
I do think the people who think LLMs are going to keep getting better, leading to either some sort of techno-utopia, or kill us all off maybe don’t really understand how they work and what the implications are.
I also think the people who are categorically against AI are making a mistake out of fear, but that’s pretty understandable.
I think of Mistral as primarily making models for API use that you can use in all sorts of places. I didn’t even know they had an app. I think the app is probably mostly intended as an easy way to try out the models, not their main business.
I do see what you’re saying. You can account for all of these factors and it still turns out that, largely, individual LLM use just doesn’t use that much power compared to most things people do day to day. Inference is so cheap that even dozens of requests don’t amount to much. I could look up and give you a bunch of numbers, but I don’t think that’s likely to convince anyone who doesn’t do the research themselves. It’s so easy to come up with sources that say what you want. I’d encourage you to actually look into this yourself.
Training costs are higher, but you train once and use repeatedly. Right now, total training costs are stupidly high, but that’s because we’ve got an arms race between the frontier labs to spend as much money and compute as they can for truly marginal gains in quality. The solution to that problem isn’t for individuals to stop using AI, it’s to stop those assholes from wasting so much power.
Individual LLM use is so cheap, that it really isn’t worth wasting people’s energies thinking about limiting that. Instead of being distracted by attempts to make this an issue of personal responsibility, we should be focusing on what will actually make a difference. We should be focused on supporting policies that lead to systemic change. A carbon tax would change corporate behavior right quick, and not just for AI companies.
I’m going to simplify a little here, so don’t take this completely at face value.
Models are, quite literally, long series of numbers (called weights). A model might store the weights in 16 bit numbers (that is, 16 binary digits). The size of the model (and how much memory it needs) is determined by how many weights there are, and how many bits each weight takes. You can take a 16 bit model and rework it to use 8 bit, or even 4 bit numbers. The result intuitively behaves a lot like the same model, but with less precision to the weights. That makes the model take way less space in ram, but also makes it more likely for concepts (encoded in the weights) to overlap, which impacts model quality. Often the effect is that fine distinctions get lost.
In recent years, I’ve started feeling comfortable recommending it over Windows.
Sure, I’ll still tell friends and family to just get a Mac. But if you’re not going to do that, Windows is just a worse experience in most the important ways at this point, even for a non-techie user.
It is absolutely possible to run 100% locally, but in practice, at the high end, only quantized models. The full size top end models require hundreds of GB of video ram, and while you can buy that, it’s stupidly expensive. Quantized models can often perform nearly as well with a small fraction of the ram, but they do sacrifice a little in precision.
These models (well the good ones) ultimately all trace their origins to what you’d likely consider “stolen” data. Whether that’s ethical is debatable. If you’re in the “information should be free” camp, there may not be an issue here.
As for the power/environmental impact, for what they do LLMs are actually very low impact per-request. If you’re concerned about your personal AI power use, then I hope you never fly in an airplane, and minimize your driving because those are much bigger issues.
It’s the scale of use that makes AI an environmental problem, and that’s a question about corporate use of AI, not personal use of AI.
Honestly, this is nothing new.
Listening to ChatGPT is a solid improvement over listening to Tucker Carlson.
Don’t get me wrong, it’s not good! I don’t exactly trust the Sam Altmans of the world. But this is the same circus, different clowns.
I think most the serious investors know. But, in a bubble, stock value can keep going up long past the point that makes sense. If you think that’s going to happen, it can make sense to invest even if you think it’s a losing play in the long run.
This should really make the timing more urgent for Anthropic. I was legitimately surprised when they decided to delay the IPO.
Also there’s still a real chance that Anthropic has some real value proposition that I’m not understanding, though I’ve yet to find someone who can make a coherent argument for what that looks like.