Posts in Ai (20 found)
Sean Goedecke Yesterday

You have to beat the models at something

In 2025, I wrote that software engineers ought to be assessed by “value over replacement” : not how much money they made for their company, but how much they would have made compared to the average engineer in their position. I’ve always found it vaguely silly when engineers put “built a product that made $X” on their resumes, when they just did the JIRA tickets that came across their desk. Today, value over replacement is even more important. A replacement-level engineer in the 2010s was fine : maybe not worth promoting, but still worth paying , because writing code had a high fixed cost. Now writing code costs a hundred bucks a month . What are you doing that GPT-5.6-Sol or Claude Opus 5 wouldn’t do in your position? Why is it worth paying an extra two or three orders of magnitude for? This is a scary thought. But you’re not doing yourself any favors by pretending that LLMs can’t actually write code and it’s all just a scam, or that LLM-written code is inherently so bad as to cause companies using it to collapse next year. We are not going to wake up in 2027 to find that the AI craze is over and everyone is writing code by hand again. You ought to put some serious thought into what you can do better than the models in the medium and long term. Staying ahead of the models is a moving target. At the start of 2026, “make working changes to large codebases” was in this category , but now it’s not. For this reason, I doubt that you can retreat to some “hard engineering” area that requires deeper expertise. That might work in the short term, but not forever. If LLMs can find a better lower bound on the Riemann hypothesis, they will soon 1 be able to write solid high-performance kernel drivers or GPU shaders or whatever. I think it’s more useful to look at the tasks models haven’t gotten better at over time, and the tasks that are hard for them get better at in principle. The two best examples of these are: What do frontier LLMs get wrong? What kind of coding mistakes do they make? It’s been a long time since I’ve seen a straight-up hallucination from a coding agent, or a simple logic error like an off-by-one. The mistakes they make tend to be errors of ignorance : Other times they’re errors of paranoia : What do these errors have in common? They’re the kind of errors a smart engineer might make if they had no context on the system: they’re competent enough to be able to solve the problem, but they haven’t been around long enough to confidently say “yes, we can take this risk to avoid an extra three thousand lines of code”. Until someone cracks continuous learning or truly massive context windows, this is just an inherent feature of how AI agents operate. If you can catch these errors, you’ll be providing real value. The only way to catch these errors is to be familiar with the codebase and familiar with the system in general. For much more on this, see my post You can’t design software you don’t work on . But there’s also a psychological component to it. You have to be willing to confidently disagree with the agent. AI agents can be very convincing. Often they can get “stuck” on some error above where they’re not willing to take a particular risk, so they keep going back and sneaking in code to cover that case (or writing persuasive arguments about why that case is important). To add value, you need to be willing to say “this sucks, I don’t think we need X and Y at all, why can’t we do Z in a much simpler way?” It takes courage . You can’t rely on other AI agents to review each other’s work. If you use the same model, it’ll reliably make the exact same assumptions and mistakes. But even if you use different models, they’ll also tend towards the same kinds of mistakes — ignorance and paranoia — for the same structural reasons. AI-driven review loops are in fact more likely to get these things wrong, because modern AIs have been RL-ed to try to find a few nitpicks no matter what. Having a critic AI and a worker AI bounce off each other is a really good way to end up with ten thousand lines of paranoid slop. Another area where you can add value on top of AI is communication . Newer models are better at coding, but are paradoxically getting worse at writing. GPT-3.5 and GPT-4 had a human-like writing style at times. GPT-4o introduced the modern slop idiolect, and the newer Anthropic models speak “Claudish” : a bizarre semi-baroque semi-truncated way of communicating that nobody enjoys. There have been a few bright spots — GPT-4.5 was okay, and I quite liked o3 2 — but in general LLMs are not good at this. Here’s two reasons why. First, good writing is not a verifiable domain . If you want a model to get good at mathematics or coding, you can generate problems for it and automatically grade them. You can’t grade good writing. If you try to get humans to grade it — for instance, via the early OpenAI RLHF attempts — you get the kind of writing that sounds impressive to the average person when consumed in single-paragraph form. This is the origin of the “stick three hundred writing devices into every sentence” style. I think it’d be possible in principle to hand-pick some people with good taste and have them do it, but there are some obvious problems 3 that prevent this from happening. Second, the labs have been monomaniacally focused on capability instead of communication . When you’re trying to train a model that can break new scientific ground or replace a software engineer, you might trade off some communication ability. In fact, I think we can identify exactly how this has been happening. If you look at internal model reasoning tokens , they tend to have strange word choices and oddly truncated grammar: RESOLUTION: charge the current-leg’s OWN saved-prefix occupancy EAGERLY: when leg i saves e 1..e t: ALSO commit their occupancy AT LEG i If you were to translate this into proper English, you would probably end up with something that reads like Claudish: Charge the current-leg’s saved-prefix occupancy on a clean, eager path: when leg i saves e 1..e t, commit the occupancy at leg i. I suspect that the weirdly alien writing style of some LLMs is because you’re reading a semi-literal translation of that model’s internal chain-of-thought, which has become nearly incomprehensible in pursuit of better problem-solving abilities. It is surprisingly hard to translate Claudish to good English: not only do you need to follow the convoluted, compressed language of the original, but you need the technical ability to understand the problem the model is solving. Because of all this, technical communication may be a surprisingly durable skill. In Peter Watts’ novel Blindsight , the world is full of cognitively augmented humans. The main character is a “synthesist”: someone whose job is to be a translation layer between these geniuses (who speak in abbreviations and gestures) and everyone else. Watts’ idea is that communication ability may be largely independent from — or even negatively correlated with — intelligence. A “country of geniuses” may still need a bunch of ordinary smart people to translate their insights for everyone else. If you’re trying to communicate to humans, there are also huge advantages to having a human write the content. Many of us are becoming AI-blind : developing an instinctive reflex that stops us reading when we encounter AI-generated content. It’s like the reflex that allows people to ignore flashing billboards or sidebar advertisements on websites. If you circulate some planned technical strategy as an AI-written document, most of your colleagues will have to physically force themselves to read it word-by-word. Whatever you do, don’t be a meat proxy : someone who simply copies requests into an AI agent and submits their output as your own work product. Doing that is just begging to be fired, since you’re definitionally not adding any value yourself. Even if you have a cunning system of multiple agents — the so-called “software factory” — you’re still on dangerous ground. When the features of your system work their way into enterprise AI tooling (and they will), you’ll be disposable. You need to find some way to leverage your expertise to do what the models can’t. Simply not using AI at all is better than being a meat proxy, since you’ll probably do some things better than the model would have, but it’s far better to figure out what AI can do and position yourself to fill those gaps. Right now, there are two main gaps: familiarity with the technical details of the system, and the ability to clearly and persuasively write about those details. If you’re thinking “but LLMs can do these things now!”, substitute your preferred example of high-difficulty software engineering. Although this was probably a “thank God it doesn’t speak like 4o” reaction. Defining good taste is hard, there’s no guarantee that AI lab researchers have good taste to start with, nobody will agree on examples, the bulk of users might not even like it, you won’t be able to get enough people to produce the volume of data you need, and so on. Deep familiarity with the codebase Technical communication Not knowing that there’s a module in the codebase they could use instead of reimplementing some logic Making the change in the wrong system because they didn’t know System X was the standard place for this functionality Adopting a coding style that’s inconsistent with the company’s standard practice Implementing triply-redundant checks for a value that technically could be wrong but practically is set once from config and never updated Assuming that ten milliseconds of stale data is unacceptable and designing a complex, unnecessary system to keep it always up to date Building in fallbacks and “graceful” degradation into some code that ought to simply crash on error (e.g. a CLI tool, or a restartable k8s service) If you’re thinking “but LLMs can do these things now!”, substitute your preferred example of high-difficulty software engineering. ↩ Although this was probably a “thank God it doesn’t speak like 4o” reaction. ↩ Defining good taste is hard, there’s no guarantee that AI lab researchers have good taste to start with, nobody will agree on examples, the bulk of users might not even like it, you won’t be able to get enough people to produce the volume of data you need, and so on. ↩

0 views
ava's blog Yesterday

link dump - catching up on my online reading

While I am slowly getting back up on my feet after a tough time, I am catching up on emails (will still take a bit!), my RSS feed reader, and several newsletters that have accumulated in my inbox. Here's what I picked out to read and share: This Is Capitalism: Apple's Hidden Data Workers at the Shadows of the AI Boom - 19 page paper detailing what data workers actually do and what problems they deal with, written anonymously by a data worker interviewing their colleagues. Parts of the work descriptions remind me of the work in Severance . Is this really a good reason to triple datacentre capacity in Europe? - online blog post tracing where the idea to triple the EU's data centers comes from that is mentioned in several EU AI strategies. Turns out it's from be a blog post from Savills, a commercial real estate firm who profit off of data centers being built. Fake US thinktank set up and funded by Israel sought to game AI for propaganda - AI slop meant to absolutely flood the web to be included in AI training funded by the Israel government to spread disinformation in AI answers. A new force is increasing inequality in America - WaPo article about how AI is not leveling the playing field or bridging the gap between poor and rich, but instead worsening the gap. People making the most use of AI are concentrated in richer urban areas and are already often rather wealthy, while the AI data centers are in poorer neighborhoods and the data workers are often migrants or in the Global South. Rich people can invest into AI and its stock, therefore profiting off of the hype and concentrating even more wealth. An operational framework for AI literacy in the workplace - 15 page paper from Interface EU addressing the vagueness of "sufficient AI literacy" that is often mentioned in EU AI legislation. It proposes a cumulative three-tier framework based on the nature and consequences of a worker's interaction with AI which then dictates the level of literacy required and how to attain/ensure it and measure it. The Quiet Erosion of Collective Action Under Digital Surveillance - article on chilling effects of permanent surveillance and the feeling of constant suspicion which continues to erode activism. Flipping the kill switch: I survived 72 hours without US tech - online article about an experiment to live without reliance on US tech. The US has such a strong monopoly that almost all online services and tech are unusable with this rule. Gone in one click - assessing the socio-economic impact of browser-level consent in Europe - small informative flyer style PDF by the Implement Group showing figures about cookie consent rates and ad industry revenues depending on the mode of consent. Inside the growing vigilante movement to knock out Flock surveillance cameras - online article by The Guardian. I admire these people, and we need more civil disobedience now, everywhere. Not just against Flock; against Meta glasses wearers, against Ring camera owners (Yes, you too! None of you are the "good ones" with "valid" reasons!) and more. There's many ways to affect these devices that you can find online or just get creative with it. Keep yourself safe, don't write about it, don't record yourself doing it, don't discuss it via digital means, leave your devices at home, and leave no fingerprints. Did someone wearing Meta Glasses film you today? Are you sure? - another Guardian online article, this time about the spy glasses and the people who enable hiding the recording light on them. The man behind GhostMeta is actually so vile and disgusting; anything else I could say would violate the Code of Conduct this blog is hosted on. German links: HeißeLuft.org - German website with interactive map showing where AI data centers are planned, in development, and paused, together with information on protests. Made me discover that they are planning on building one not too far away from me... Deutsche Post trainiert ihre KI mit Ausweisfotos - article about how Deutsche Post is training their AI with ID pictures they get via digital identification procedures. It's not voluntary as they claim, as you need to give permission before being allowed to proceed. Verhaltensscanner in Berlin: Harte Kritik an der KI-Überwachung - online article about the new camera installed in Berlin that will analyze all people in that area via AI surveillance software by Adesso, with more cameras to follow. They want to put them up in high crime rate areas , but keep secret what the standards for this are, which enables a mass roll-out of them if they wanted to without any oversight or control. There has already been one mix-up leading to higher crime reported in an area than actually happened. These cameras already also exist in Hamburg and Mannheim. Möglicher AfD-Sieg in Sachsen-Anhalt - online article detailing the fear of queer people and people of color of an upcoming potential win of the AfD in their state. Afd-Gutachten.de - website containing some stats and a PDF report of a legal assessment on the chances of a successful AfD ban. „Gipfel gegen Linksextremismus“: Mit Trump gegen die Antifa - online article about the cooperation of Germany with the US on its fight against antifascism. Unfortunately it has continued, with the German government realigning to focus more on supposed "leftist extremism" and even re-distributing money away from leftist projects, which mostly hits projects aimed at helping queer people and migrants. Wie weit ist Deutschland beim digitalen Gewaltschutz? - an online article about the really embarrassingly low standards of protection against digital violence, especially image-based ones like deepfake nudes and revenge porn, in Germany. Lots needs to be done in general, but especially to even meet the new EU standards. Wer ist für Straftaten der KI verantwortlich? - legal article about the criminal liability of autonomous AI in Germany, and how crimes done by AI agents are pushing the legal system to its limit as we only legislate for humans. Published 29 Aug, 2026 This Is Capitalism: Apple's Hidden Data Workers at the Shadows of the AI Boom - 19 page paper detailing what data workers actually do and what problems they deal with, written anonymously by a data worker interviewing their colleagues. Parts of the work descriptions remind me of the work in Severance . Is this really a good reason to triple datacentre capacity in Europe? - online blog post tracing where the idea to triple the EU's data centers comes from that is mentioned in several EU AI strategies. Turns out it's from be a blog post from Savills, a commercial real estate firm who profit off of data centers being built. Fake US thinktank set up and funded by Israel sought to game AI for propaganda - AI slop meant to absolutely flood the web to be included in AI training funded by the Israel government to spread disinformation in AI answers. A new force is increasing inequality in America - WaPo article about how AI is not leveling the playing field or bridging the gap between poor and rich, but instead worsening the gap. People making the most use of AI are concentrated in richer urban areas and are already often rather wealthy, while the AI data centers are in poorer neighborhoods and the data workers are often migrants or in the Global South. Rich people can invest into AI and its stock, therefore profiting off of the hype and concentrating even more wealth. An operational framework for AI literacy in the workplace - 15 page paper from Interface EU addressing the vagueness of "sufficient AI literacy" that is often mentioned in EU AI legislation. It proposes a cumulative three-tier framework based on the nature and consequences of a worker's interaction with AI which then dictates the level of literacy required and how to attain/ensure it and measure it. Great quote from it: "No evidence yet shows that these trainings work, and three gaps might explain the reason. The first is motive. Corporate training aims at productivity and teaches people to use the tools well, whereas the law cares whether operators understand how systems fail and cause harm. A workforce fluent in prompting can still be illiterate in the sense a regulator means: trained to produce good output, but not to recognise when a model misleads or to know its duties under data-protection and risk rules." The Quiet Erosion of Collective Action Under Digital Surveillance - article on chilling effects of permanent surveillance and the feeling of constant suspicion which continues to erode activism. Flipping the kill switch: I survived 72 hours without US tech - online article about an experiment to live without reliance on US tech. The US has such a strong monopoly that almost all online services and tech are unusable with this rule. Gone in one click - assessing the socio-economic impact of browser-level consent in Europe - small informative flyer style PDF by the Implement Group showing figures about cookie consent rates and ad industry revenues depending on the mode of consent. Inside the growing vigilante movement to knock out Flock surveillance cameras - online article by The Guardian. I admire these people, and we need more civil disobedience now, everywhere. Not just against Flock; against Meta glasses wearers, against Ring camera owners (Yes, you too! None of you are the "good ones" with "valid" reasons!) and more. There's many ways to affect these devices that you can find online or just get creative with it. Keep yourself safe, don't write about it, don't record yourself doing it, don't discuss it via digital means, leave your devices at home, and leave no fingerprints. Did someone wearing Meta Glasses film you today? Are you sure? - another Guardian online article, this time about the spy glasses and the people who enable hiding the recording light on them. The man behind GhostMeta is actually so vile and disgusting; anything else I could say would violate the Code of Conduct this blog is hosted on. HeißeLuft.org - German website with interactive map showing where AI data centers are planned, in development, and paused, together with information on protests. Made me discover that they are planning on building one not too far away from me... Deutsche Post trainiert ihre KI mit Ausweisfotos - article about how Deutsche Post is training their AI with ID pictures they get via digital identification procedures. It's not voluntary as they claim, as you need to give permission before being allowed to proceed. Verhaltensscanner in Berlin: Harte Kritik an der KI-Überwachung - online article about the new camera installed in Berlin that will analyze all people in that area via AI surveillance software by Adesso, with more cameras to follow. They want to put them up in high crime rate areas , but keep secret what the standards for this are, which enables a mass roll-out of them if they wanted to without any oversight or control. There has already been one mix-up leading to higher crime reported in an area than actually happened. These cameras already also exist in Hamburg and Mannheim. Möglicher AfD-Sieg in Sachsen-Anhalt - online article detailing the fear of queer people and people of color of an upcoming potential win of the AfD in their state. Afd-Gutachten.de - website containing some stats and a PDF report of a legal assessment on the chances of a successful AfD ban. „Gipfel gegen Linksextremismus“: Mit Trump gegen die Antifa - online article about the cooperation of Germany with the US on its fight against antifascism. Unfortunately it has continued, with the German government realigning to focus more on supposed "leftist extremism" and even re-distributing money away from leftist projects, which mostly hits projects aimed at helping queer people and migrants. Wie weit ist Deutschland beim digitalen Gewaltschutz? - an online article about the really embarrassingly low standards of protection against digital violence, especially image-based ones like deepfake nudes and revenge porn, in Germany. Lots needs to be done in general, but especially to even meet the new EU standards. Wer ist für Straftaten der KI verantwortlich? - legal article about the criminal liability of autonomous AI in Germany, and how crimes done by AI agents are pushing the legal system to its limit as we only legislate for humans.

0 views

What GLM-5.3 Flash running on Chinese hardware actually means

Z.AI confirmed that their most recent model release was running all inference on Chinese manufactured hardware. While no doubt an impressive feat, Western companies still have a huge advantage that I can't see changing quickly. To start with, it's worth looking into where Chinese AI hardware is. I'm focusing entirely on the HiSilicon parts - the most competitive parts from Huawei. There are (many, actually) other manufacturers building AI hardware, but it's widely believed that they are no further ahead than HiSilicon, so I think that for brevity it's a fair starting point. One caveat before I go further: Z.AI didn't actually name a chipmaker, and didn't publish throughput or power numbers either. Nobody has independently verified the claim. So I'm assuming HiSilicon here because it's the only plausible candidate at that scale, not because anyone has confirmed it. It's also worth mentioning that the US export restrictions ( CSIS has a good overview ) of high end AI hardware have made this an enormous priority, understandably, for the Chinese. And it's definitely worth mentioning that finding accurate sources for many of the numbers I'll cite are difficult to be confident in, so take the exact numbers with a pinch of salt. The current 'scale-up' series of HiSilicon chip, the 910c series, pairs 96GB of HBM 2e memory with two compute dies, probably achieving something like 1.6PFLOP/s of INT8 compute with ~3TB/sec of memory bandwidth, at around 600W. In essence, this is substantially behind even the H100 from Nvidia, which is now 4 years old. These are around 60% as fast as the H100, and has various other footguns (no native FP8 support for example), which probably restrict efficiency further for many use cases. The next generation 950-series doesn't meaningfully increase compute as far as I can see, but does use domestically produced HiZQ/HiBL HBM memory. Interestingly the cards are configured in two variants - the 950PR and 950DT, with the former focusing on prefill and the latter on decode. In reality, the two products are very similar, but the prefill variant using slower HiBL memory vs the decode HiZQ memory. It does however support more quantisation types, like FP8. I think this shows the limitations of what Chinese hardware can do - at least for the near future. Yes, they can run inference, but so can many sets of hardware now - AMD, Google and Amazon all have competitive solutions, and OpenAI are making significant progress on their Jalapeño inference chip , which in the first published benchmarks did 1.5-1.9x the work per watt of Nvidia's GB300. Inference hardware while no doubt complex, is a pretty solved problem right now with a lot of competition - and that's before you bring in the Cerebras and Groq approach chips. The wall that these Chinese hardware manufacturers are hitting is the lack of viable EUV (extreme ultraviolet) fabrication. This is the next generation silicon manufacturing process from ASML and it is extremely hard . I'd really, really recommend reading Chip War by Chris Miller for the full story, but regardless until there is significant progress on this - and by significant progress, I don't mean the reverse engineered prototype in a Shenzhen lab. I mean reliable, scale production. The industry would be astonished if they got this to scale production before 2030. Bear in mind the Shenzhen prototype hasn't produced a working chip yet, and the more optimistic forecasts have them doing that around 2030 - volume production is a further step beyond it. It took ASML 25 years to figure out this technology - and a good 5+ years of this was scaling it up from the lab to "real" production lines. While China no doubt has incredible engineering talent and the ability to reverse engineer some of ASML's work, it's still a daunting challenge. Without EUV it is not possible to go (much) below the "7nm" fabrication size. Without being able to go below that size, you quickly hit a wall in thermal efficiency, and you reach a point where you simply cannot make the chip(s) any bigger or faster because you cannot expel the heat quickly enough. Added to that, the additional export restrictions on HBM memory to China are clearly causing significant issues, hence the strange use of two different home grown memory technologies in the 950-series - no doubt because they can't produce enough fast (which is still comparatively slow ) memory. These are really the same base constraint - without EUV manufacturing technology you can't produce the latest generations of very fast HBM memory either. Clearly the approach China is taking is instead of really looking for solid incremental leaps in compute and memory from better manufacturing techniques, the idea is to build a lot of them. Even if your fastest chips are at best 5 years behind the latest Nvidia GPUs, you can just build 10 times as many for the same overall inference capacity. And it really is roughly 10x - not against the H100 I was comparing to above, but against what Nvidia actually ships today. A Rubin VR200 is somewhere around 35PFLOP/s of dense FP4 with 22TB/sec of HBM4 bandwidth. The 910c is 60% of a four year old H100; Rubin is another order of magnitude past that. No doubt China is uniquely positioned in being able to do this - with enormous power generation capacity to power this, and huge quantities of skilled engineering and manufacturing labour to build the facilities and cooling required. But really, it's far from ideal. As models get larger, you have to split them over more and more underpowered sets of hardware. Another problem is it makes the models slow - Z.ai's own API is noticeably slower than Western providers serving the same weights. The bit I keep coming back to though is power. And here you have to be careful, because 10x the throughput gap is not 10x the power bill - the 910c pulls about 600W against something like 2000W for a Rubin part. Divide the spec sheets and you get a much less dramatic 2-3x on both compute per watt and bandwidth per watt. But the spec sheets flatter the 910c. 96GB a chip, against the 288GB or more you get on current Western parts, leaves much less room for KV cache, which forces smaller batches, and decode throughput per watt falls away badly at small batch sizes. Add a less mature software stack, and the interconnect and cooling overhead of running 10x the chips, and 5x worse on tokens per watt feels about right to me. If anything that's the charitable end. Which matters because electricity is usually reckoned to be 10-20% of the total cost of running a GPU cluster, with hardware amortisation dominating. Multiply that by five and power goes from a small component of costs to something like half your total bill. That's fine when you have China's generation capacity and you're happy to treat the difference as a strategic subsidy. It's a lot less fine if you ever want to sell inference into a competitive global market on price. Small models getting better doesn't rescue this either. They help, obviously - a 30B model serving a task that used to need a 300B one is a real saving. But it's a saving both sides get - that smaller 30B model still runs 10x as fast on Western hardware, so the ratio between Chinese and Western hardware efficiency stays exactly where it was. And assuming China doesn't have some huge breakthrough in fabrication technology - which as I said before is highly unlikely - it's probable that the gap between Western and Chinese AI hardware will widen if anything. So, to round up - yes it's an impressive feat that they've managed to do this, but there are some hard constraints on efficiency that are unlikely to be solved any time soon. And yes, China could overcome it by sheer quantity, but it's a subpar solution that has real impact on the speed, capacity and economics of their inference.

0 views
Stratechery 2 days ago

2026.35: Internet Hype and Real World Change

Welcome back to This Week in Stratechery! As a reminder, each week, every Friday, we’re sending out this overview of content in the Stratechery bundle; highlighted links are free for everyone . Additionally, you have complete control over what we send to you. If you don’t want to receive This Week in Stratechery emails (there is no podcast), please uncheck the box in your delivery settings . On that note, here were a few of our favorites this week. This week’s Stratechery video is on Nvidia’s Risky Business . The Breaker’s Advantage . One of the most important takeaways of The Hugging Face Incident is that agents are more useful for attacking infrastructure than in defending it. While in theory defenders know the code, their number one job is to not break things; for attackers breaking things is the point. This week’s Article Autonomy and Innovation makes the case that this dichotomy isn’t just relevant to security: it also explains why startups consistently defeat incumbents, and why AI’s takeover of the economy will take longer than people think. The New Battle for HDMI1.  For years Netflix insisted its service stood alone, resisting attempts by companies like Apple to integrate their service. Now Netflix is poised to go in the other direction, potentially selling access to other streaming services. Ben wrote about the company’s shift on Tuesday , and on this week’s Sharp Tech chalked it up to Hollywood staying irrational longer than Netflix could stay patient.  — Andrew Sharp How Data Center Discourse Ends.  The backlash to the continued buildout of AI data centers has continued all summer, and now looks even more widespread than it was when Ben tackled the issue in May and we dedicated an entire episode of Sharp Tech to the controversy . Now that people in tech are legitimately worried, however, it’s time to zag: I think that this will ultimately be a non-issue , just like so many other overwhelming Internet movements. — AS Autonomy and Innovation — Incentives favor offense when it comes to agentic cybersecurity; it’s the same dynamic that will limit incumbents and fuel startups in the long run. Netflix to Sell Streaming Services?, Streamers as Aggregators, Revisiting Roku — Netflix is considering selling other streaming services, and I think it’s a good idea; it’s also a let-down for Netflix’s original goals and potential pivots. Apple Updates Mini and Studio, AI Computers, OpenAI Jalapeño — Apple and OpenAI have two completely different hardware announcements; both represent pressure on Nvidia. Halt and Catch Ire — A survey of data center madness, and why I’d bet the under on the durability of the backlash . Omarchy and Open Macs Has the Solid State Transformer’s Time Finally Come? Five US-China (and Russia) Questions; Cabbage with Formaldehyde; The Continuing Tax Crackdown; Unitree Stock Down 45% Peyton Watson to the Cavs, Building a Top Five for 2031, Top 5 Feats of Loser Behavior Meta’s New Restrictions for Teens, Nvidia’s Open Source Investments, Q&A on Netflix, Druckenmiller, Parameters and Performance

0 views

Premium: The Hater's Guide To Circular Financing (Part One)

[NVIDIA Company Meeting, the present day, YMCA playing] JENSEN HUANG : We love NVIDIA, don’t we folks? We’re the biggest, most-beautiful semiconductor company, we make the biggest, hottest GPUs for Clammy Sammy and Wario Amodei ’s huge, beautiful AI labs, but they can’t afford them because they’re losing so much money! [crowd booing] It’s okay! It’s okay! Big strong men, the biggest muscles, big, beautiful, strong men like Satya Nadella are calling me, begging — they’re begging, can you believe it? — they’re begging me, “Sir, Sir, please ship me Vera Rubin sir! I can’t get enough!” [crowd braying] they can’t get enough of Vera Rubin! They’re begging me to get Vera over there! Vera! Where’s Vera! [scanning crowd] get her up here! No, no, don’t do it, she’s too shy! We love Grace too, [voice turning gravely] Grace Blackwell , what a gal! I told them all we’re going to ship a trillion dollars of Grace Blackwell and Vera Rubin by the end of 2027 , our beautiful girls Grace and Vera , they’re our biggest and most-expensive girls yet, our Gee-Pee-Yous , the media says “we don’t believe you sir!” but I’m gonna make everyone buy ‘em, hell I’m gonna give ‘em the money to do it like I did with CoreWeave and then I’m gonna tell  Clammy Sammy and say “Samuel, give ‘em a few billion like you gave to Michael Intrator ,” and he’ll say “yes sir!”  Now, people are saying to me — “Sir! Sir! Your customers can’t afford your semiconductors! Sir, they’re too expensive!” and I say they’re not expensive enough! We’re gonna charge ‘em 17% more! [crowd braying] Should we up the price? Should we do it? We’re gonna do it!  In my mind, this is how Jensen Huang speaks to his workers, more than 70% of whom are millionaires as a result of NVIDIA’s remarkable stock growth, and from what I’m told by insiders, there’s a near-manic attention paid to stock movements as a result. I imagine working there must feel a little insane. Assuming you arrived before the stock went parabolic in 2024, you’ve seen your RSUs explode 10x in the space of a few years, all based on the back of everybody talking about how big and huge AI is… … all as it becomes blatantly obvious that NVIDIA’s biggest customers are, for the most part, funded by NVIDIA . While NVIDIA still ostensibly sells things other than AI GPUs (like autonomous cars , laptop graphics cards, and simulation technology for robotics ), more than 90% of its revenue comes from data center hardware. As a result, the company has become almost-entirely valued on whether or not it can continually come up with rationalizations for its largest customers to spunk tens of billions of dollars a quarter.  Why else would NVIDIA invest even an iota of effort into making an NVIDIA-branded Openclaw or build a platform for LLMs to do “agentic” things , or give $6 billion to Poolside (while investing another $1 billion) and hire away most of its staff? Why else would it plan to invest billions of dollars in Perplexity at a $30 billion valuation that lands somewhere between “fucking stupid” and “laughable”?  Sorry, I’m being a little vague. Everything NVIDIA has done for the last three years has existed to do two things: NVIDIA has succeeded in doing the first primarily by selling these GPUs to hyperscalers like Amazon, Google, Microsoft, Oracle, and Meta, who make up somewhere between 50% and 60% of its GPU sales depending on which analyst you ask.  The rest comes from a mixture of unnamed “sovereign AI customers” and “neoclouds” — companies that exist to raise debt, buy NVIDIA GPUs, and put them in data centers to rent to theoretical AI customers. Per Vivek Arya of Bank of America (at the BoFA Global Technology Conference in June), sales to “neocloud/sovereign/on-premise” were about the same as those to hyperscalers, and while it’s tempting to dither here and say “there could be large sovereign buildouts!” I can’t find compelling evidence that these actually exist outside of a theoretical 75 billion Euro investment in AI infrastructure in France by SoftBank , which doesn’t have that much money to spend. In any case, NVIDIA’s entire strategy has become a case of either convincing the largest companies in the world to give Jensen Huang $100 billion a year or artificially inflating its revenues through circular financing, which is obviously what I’m talking about today. This is the first part of my Hater’s Guide To Circular Financing, a comprehensive analysis of the current state of NVIDIA’s massive circular financing operation, why it has yet to break, its limitations, and the material concerns that were raised in its latest quarterly earnings. The second part, coming next week, will cover the history of circular financing, where we’ve seen it before, and what we can learn from its horrible past. Create sales for its AI GPUs and associated hardware. Create demand for AI compute for its customers.

0 views
Hugo 3 days ago

What if the AI bubble burst?

If there's a trendy topic in tech right now, it's AI. And it's normal, given the amounts invested on one hand, the societal transformations on the other, not to mention all the issues and risks related to employment and ecology. But there's one theme that seems to unite just about everyone, pro and anti AI, it would be a bubble, and it's going to end up bursting. Well, now's the time to ask the question. Is it a bubble? Can it burst and when? And what would the consequences be if it did? If the question seems obvious to many, it deserves a closer look. Are we really in a financial bubble? Is the value we're giving to AI companies realistic? Are we witnessing a historic transformation... or a speculative bubble? A financial bubble isn't simply a sector that rises very quickly. Otherwise Amazon in 2010, Tesla in 2020, or Nvidia today would all have been bubbles. The central question is one of sustainability : is the value attributed to these companies realistic? Are massive investments in AI based on solid financial foundations? At first glance, it doesn’t look good. Anthropic generated about 850 million dollars in revenue in 2024. Yet, a few months later, the company was valued at nearly 19 billion dollars. More than twenty times its annual revenue. You need to understand that in finance, there are two logics: However, just because these multiples are enormous doesn't mean they're unusual and abnormal. They're abnormal for the period, but not abnormal in absolute terms. In the past, Shopify, Snowflake, Amazon, all these companies exceeded x50 and the bets turned out to be winning. So, wrong trail. Verdict: ❌ inconclusive Over the past three years, more than 1,000 billion dollars have been committed to the AI ecosystem. According to the BIS (Bank for International Settlements), 5% of US GDP is today linked to investments in AI . This amount exceeds what was observed during the previous internet bubble in 2000. And in absolute value, the Internet bubble represented approximately 600 billion dollars of the time. Adjusted for inflation, we're again approaching current investments in AI. On the other hand, by contrast, the subprime crisis involved over 10,500 billion dollars in residential mortgage debt. But be careful, we're comparing apples and oranges in this specific case. What's certain is that the amounts at stake are colossal. But colossal doesn't mean abnormal. Telecoms in the 80s/90s, mobile in the 2000s also attracted gigantic sums without people talking about a bubble at the time. Verdict: ❌ concerning but inconclusive Here we touch on one of the specifics of the potential current crisis: the revenue of AI companies increases through their own spending on AI . AI actors aren't just companies that create LLMs. It's also cloud hyperscalers or chip manufacturers. It's a perfect form of circular economy (round tripping) The problem with this circular economy is that it self-feeds and we don't yet know if real demand will follow. If the only customers for AI are the other AI players, let's just say that's at least suspicious. All projections for new datacenters, investments to build new chips, etc... would potentially be thrown away in case of market reversal since for now the only customers for these companies, at this scale, are themselves. And the sums we're talking about are in the tens of billions. Verdict: ⚠️ concerning Investments in AI are not profitable except for NVidia but that's a bit natural when you're selling the hardware (see clue 3). For the major players (Amazon, Google, Microsoft, Meta), for every euro spent, less than 10% of the sum comes back in revenue, with Meta winning the small palm for having only 1% coming back in revenue. Even major players like OpenAI, Anthropic (or Mistral in France) struggle to exceed 40/50% of the sum in return. In short, these players aren't profitable. But this clue is very shaky. It's rather common to over-invest in the startup phase, especially in industrial fields that require investment. Amazon wasn't profitable until its 9th year, Tesla until its 17th year, Uber 14 years. A phase of significant losses is nothing unusual when a sector is building the infrastructure for a new market. It's even almost expected. In reality, the problem isn't that AI isn't profitable today, that's normal. The real issue is whether it can be profitable tomorrow, once investments have stabilized. This question remains largely open. Are today's investments sustainable or do we need to reinvest the same amount every year? For now, given the obsolescence of the hardware used (chips for training and inference), the answer doesn't seem to be going in the right direction. Verdict: ⚠️ concerning Remember the bubble of the 2000s? In 2000, all you had to do was rename your company with a .com, like sausage.com and boom, valuation increased. Irrationality was so great that investors rushed at anything and everything, and especially everything, just to not miss out on the next gem. And let's remember that investors aren't always the best when it comes to rationality, because each time, during major crises, we get a new version of "this time, it's different" which basically justifies that economic fundamentals can be neglected. Well, we're seeing something similar today. All companies are integrating AI, at least in their investor pitches. Some have started reserving .ai extensions just to ride the wave and I'm more than suspicious of the valuations of some companies like Lovable (6 billion), Cursor (60 billion), Devin (26 billion). It's cool for their creators but, making a wrapper on existing AIs, I'm not convinced that's worth that price. Verdict: ⚠️ concerning So yes, I can't say 100% that we're in a financial bubble but it looks like one. Extraordinary valuations, gigantic investments, an economy that runs largely on itself, companies still far from profitability and, above all, discourse that's starting to sound oddly similar to that of past major bubbles. At this point, the most interesting thing therefore maybe isn't to know if a bubble exists. It's to understand how it could burst . Because all bubbles don't burst the same way. Bubbles always end up bursting but the detonator is often unpredictable. We can easily identify the weaknesses but it's sometimes a trivial event that ends up bringing down the system. Despite this, I propose we play a game and try to list the possible triggers for a potential AI crisis. Anthropic and OpenAI are planning upcoming IPOs. While the exact dates aren't yet known, this could happen between the 4th quarter of 2026 and the 2nd quarter of 2027. During these IPOs, we'll see if "retail investors" decide that the stock price is actually worth what we're paying for it today. And precisely, there are several problems as we've seen: To that I'd add another important problem: the amount of liquidity available on the market . The theoretical valuation of Anthropic is 965 billion dollars and OpenAI isn't far from this figure. Obviously an IPO won't be for the full value but a part, let's say 10% (100 billion). The records for IPO amounts oscillate between 35 and 50 billion. To go buy 200 billion (for Anthropic and OpenAI), will markets follow? Will the first raising, which would already be exceptional, not handicap the second? Option A: both raisings happen in these orders of magnitude, it will necessarily be by selling other assets to mobilize cash, so by creating a drop in the prices of other companies. Option B: it's impossible to raise these amounts, and the values of both companies drop Despite this, I want to set this scenario aside because I find it obvious, too easy and therefore unlikely. Today it's hyperscalers that are investing the most in the big AI companies, OpenAI and Anthropic to name just them for now. And when I speak of investment, I also speak of material investment with facilitated access to infrastructure. Except that for now the return on investment is low. So imagine that tomorrow, Microsoft announces it's reducing these investments and lowering these datacenter/infrastructure costs. The market won't just look at Microsoft and LLM publishers. It will look at Nvidia, AMD, TSMC, electricity producers. The whole chain. Despite this, this option doesn't suit me either. True, for Microsoft or Amazon, the situation isn't rosy, but it's not catastrophic either, because these companies benefit from public contracts, particularly in the military field. And deciding to cut investments now would be precipitating failure, shooting yourself in the foot, which seems unlikely to me. This trigger is based on a very simple idea: datacenters have enormous resource needs. Eventually we know that these resources won't be available, whether for energy or mineral resources. Imagine that tomorrow: We could imagine a slowdown not financial... but physical. And we fall back on option 2. We won't just look at the datacenter, we'll look at the whole chain, all planned investments that won't be made. This option is much more credible, in my opinion, but in the medium term. It seems difficult for me to imagine a significant event before 2028. However, I may be naive because relations with China keep degrading and datacenter construction refusals are already increasing. I therefore classify this lead as credible . This is a real issue. Imagine that Opus 8 is "only" 3% better than its predecessor? Investors pay to see exponential growth. This is exactly what motivates all the circus we usually see at each release from these giants' marketing teams. They have to sell us the incredible, because they'll only be paid for the impossible. What would happen if this rate of improvement in model performance slowed? Again, we come back to option 2. A major investor could decide to lower these datacenter expenses and you know the rest. Is this option credible? The improvement over 1 year has been staggering and almost frightening. Despite this, I have the impression that the room for progress remains significant. But I say that being far from being a specialist. I would however tend to reject this option for at least the next 2 years. This option is more traditional. What if tomorrow a huge shock shook the economy with no relation whatsoever to AI. It could be an energy crisis, a political crisis (an open conflict between Europe and Russia for example), the Chinese real estate crisis that's been brewing for years. This is typically the option where unknown unknowns are more numerous than others. Since this option is by nature completely unpredictable, I won't consider it either. At this point you might tell me, "ok but you haven't retained any option, so you don't think the bubble will burst?" That's not quite it. I think most of these causes are too predictable, or too long term. Again, predicting the trigger is particularly difficult. It will maybe be a combination of factors with a domino effect we don't anticipate today. The trigger for the burst of the internet bubble in 2000, for example, was the Federal Reserve's rate hike. In any case, I have trouble not imagining a trigger in the next 2/3 years. On the other hand, the consequences won't necessarily be what everyone imagines. I'm not done bothering you with multi-section chapters :) We often imagine that the burst of a bubble is like the 2000 internet bubble or the subprime one: a huge market crash, lots of companies on the ground and done. Well, not necessarily. Bubbles always end up bursting but it can be slow and smelly and we'll see again several scenarios. That's the easy scenario, the one we all have in mind. Valuations collapse, some major players go bankrupt, followed by layoffs in the tens of thousands and project shutdowns. For this scenario to be as impressive as possible, we can imagine a Big Tech company going under, a sort of modern Lehman Brother, or Worldcom to draw the parallel with the internet bubble. Beware, this scenario is obviously the most painful because it won't just affect "just" tech. We're talking about all hardware sellers, electricity suppliers, insurance companies that covered transactions, retirement savings plans, stocks etc... because yes, a large part of current investment is based on US values, particularly in tech. I insist, it will be very painful even if you think you're far from AI. The burst of the internet bubble was between 60 and 80% loss of value of stock indices, millions of jobs lost. We might not necessarily have as cinematic a scenario as the first. We could simply have a gradual decrease in investments but a maintenance of infrastructure and a reabsorption of large LLM publishers by hyperscalers. It would potentially be the end of the game for Anthropic, OpenAI as independent entities and the losses would be largely absorbed by GAFAM. (I'm not talking about xAI which is already part of a consortium or Gemini which is part of Google) It wouldn't be an explosion but a dilution of the AI bubble into the accounts of big techs I speak of dilution but we could also speak of consolidation. We'd certainly have fewer actors but a large part would be absorbed by the more resistant ones, Google, Amazon, Tencent for example. It would be less spectacular but not without consequences either. Part (and only part) of the investments would be set aside. Heavy investments planned for in 5 years would be stopped, so with more time to rebuild correct budgets. There would still be layoffs but less massive. My little pinch in the heart in these two scenarios is that I can hardly see a European player like Mistral survive this scenario without being completely absorbed, unless Europe decides to invest massively through public procurement. The gap is large and I find their strategy poorly lisible today, and even poorly anchored in the tech ecosystem. But I wish them the best, because we need a European champion on the subject. Even if at worst we'll cobble together things with open weight models. Now you know what? If we exclude the Mistral case, despite all this, I'm almost hoping it happens and I'll explain why. A bubble, let's remember, is misallocated capital. Now, I'm not saying that AI as a technology isn't worth it. It's completely redefining many professions. But is that reason enough to do anything and everything, I'm not sure. When Google questions its objective of carbon neutrality 2030, it's a failure for everyone. And then, is it healthy to have this race to the bottom for datacenters that we know we can't all power with electricity, except to reopen gas power plants? Besides, when I talk about misallocated capital, I'm very happy for the people who created Lovable or Cursor, that's cool for them. But all this capital to create a tech that's already being competed with by tons of products vibe coded 2 years later, that's a shame. The same money could have funded something more useful. Capitalistic irrationality, excesses on mineral resources that plunge us into a RAM and component crisis, increased tensions between countries, and simply the current climate around AI that's becoming unbearable between pros and antis, I find that's a lot of reasons to hope it stops. If the bubble bursts it will sign a form of return to rationality. I don't believe in the disappearance of the tech, the same way the internet didn't disappear in 2000 and the train also survived the railway bubble of the 1840s. What disappears in a bubble isn't necessarily technologies: it's mainly absurd valuations, projects that had no viable business model and investments made because "everyone's doing it". More rationality means better thought-out investments, R&D more focused on optimizing what we've already learned: mixtures of experts, quantization, pruning, HBM, model optimization. We need to now do as much but with less. It's a necessity. We already have enough to work with what we have in hand and it wouldn't be bad to pause a bit, think about the uses that work, and there are already some, find a real path to profitability and redirect capital, particularly toward the challenges of decarbonizing the economy (electrification of the vehicle fleet for example) or adaptation measures to climate change. Will the burst of the AI bubble see the end of AI? Not so sure. It will be more of a landing. But whatever the scenario, it will be beneficial, a return to economic AND ecological fundamentals. Don't be mistaken, given the sums already invested, the burst will hurt, a lot, but the longer it takes, the worse it will be. The band-aid needs to be ripped off, quickly. For a traditional company, the value of a stock corresponds to known performance in the present . It depends on profits generated and physical assets (factories, stores, inventory) or intellectual assets (patents). For a tech startup, however, the value rests on the promise of future gains. Its valuation doesn't reflect its current revenues, often low, but anticipates their explosion to come. A "multiple" is then applied to its revenue or recurring revenue. If this multiple rarely exceeds 10 to 15 times revenue (especially since the interest rate hike post-2021), AI pioneer companies today reach levels well above that, sometimes ranging from 20 to over 50 times their revenue . valuations seem far too high compared to revenue generated profitability is too low we have supply difficulties on rare earths, for example related to a conflict with China (60 to 70% of mining extraction and more than 85% of refining comes from China) that available electricity can no longer be supplied for datacenters (e.g. in Dublin , the US , and Singapore )

0 views
Giles's blog 3 days ago

Why do OpenAI's GPT-2 weights beat mine? Part four: digging into dropout

I'm still digging into a mystery about the models I've been training; although an increasing number of them beat the OpenAI GPT-2 small weights on the narrow technical measure of the loss they get on a test set, they're not as good at an instruction-fine-tuning test . While reading about MoE models, I came across this paragraph in the Switch Transformers paper : Our paper considers the common NLP approach of pre-training on a large corpus followed by fine-tuning on smaller downstream tasks such as summarization or question answering. One issue that naturally arises is overfitting since many fine-tuning tasks have very few examples. During fine-tuning of standard Transformers, Raffel et al. (2019) use dropout (Srivastava et al., 2014) at each layer to prevent overfitting. So far, when running my IFT test, I'd been aiming to use the same dropout setting for the fine tune as the model concerned had used in its original pre-training. That was just because it seemed natural. But the goal of dropout is to prevent overfitting when training over multiple epochs -- or, at least, that's how most of what I've read explains why we don't need it on modern single-epoch training runs over large datasets. If that's the case, though, when we do multiple epochs for a fine-tune with a more restricted dataset -- exactly what I was doing for the IFT test -- it might make sense to use dropout, regardless of whether or not the model was pre-trained with it. The fine-tuning setup already tries to avoid overfitting by bailing out when a validation loss starts rising, but dropout might still help it avoid overfitting prematurely. On the other hand, something felt a little wrong about fine-tuning a model with dropout if its pre-training had happened without it. A model pre-trained with dropout have been trained on billions of tokens, and so the model will have spent a lot of effort learning to overcome the issues that dropout causes, but one trained without it won't have that benefit. Suddenly exposing it to dropout in a much shorter fine-tuning run felt rather like asking someone who rarely drinks alcohol to take a few shots of whisky; I felt that the models might not be prepared for the effects. As I looked into this more, I noticed another surprising thing -- there was an error in the configuration that I was using when fine-tuning the OpenAI models, both small and medium. They were originally trained with dropout (or so it's believed -- the paper doesn't say, but " Build a Large Language Model (from Scratch) " says that they were, and this config on the Hugging Face GPT-2 code agrees). But that actually made my original puzzle of why they outperformed my models on the IFT test seem even more perplexing, at least in the light of this idea. If dropout was a good thing for fine-tuning, then so far they had been penalised by not using it -- that is, they were even further ahead of my own models than I thought they were. It was time to take a careful look. I fixed the config for the OpenAI weights so that my setup had dropout set to 0.1 for them, then carefully revisited the config for all of my own models, and made sure that those ones matched reality (which they did). Now, the IFT test that I've been running has two phases: Now, the nice thing about the judge script was that it didn't really care whether the result files it got came from different models or the same one; it just printed out a mapping of result files to scores. So I realised I could use it to do a comparison of all models with all possible dropout settings. For all of the models, I ran the three times, once with each of the dropout settings: , , and . Then I sent all of the resulting result files -- all models, and all dropout options for each -- to the LLM judge in one go, to see what it came up with. Here are the results, consolidated into one table. For each model, I have: It's quite an intimidating wall of numbers, but there's a bunch of interesting stuff there. Firstly: I've put the IFT score for each model where it was trained with the opposite of its pre-training dropout in bold. Let's look at the non-bold numbers first, though. If you scan down through the models, you'll see that the non-bold IFT scores -- that is, the one where the IFT test was done with the dropout, and then the one where it was done with dropout set explicitly to the same value as the one -- are identical in every case. That is a really reassuring sanity check. Remember, each of those numbers came from a different run of the script -- but because there is a fixed random seed, they should have been identical. They were presented to in the same way as a separate model's response. The fact that it came up with identical scores tells us that it judged them as being equal, which is solid evidence for its consistency in judging results in this run (which is something that can be hard to guarantee with an LLM). Similarly, if you look through the numbers of training epochs, the epochs for each one matches the epochs with the dropout forced to match the model's pre-training setting, which is also reassuring -- it's certainly what you'd expect given a fixed random seed. Looking just at the and epochs columns, you can see something else interesting. With dropout forced to be on, the number of fine-tuning epochs is always higher than the number of epochs with no dropout, except in the case of the OpenAI medium weights and "Cloud FineWeb, 8x B200 160 GiB", where it's the same. That makes intuitive sense, I think. If you're discarding 10% of your activations when training a model, you'd expect it to take longer to converge. But now let's look at the size of those changes. If you compare the increase in the number of epochs needed to train with dropout forced to be on, you can see that the change is much larger for those models that were pre-trained without dropout. The first of them, for example, "JAX, overtrained one long epoch", went up from 3 epochs to 19! That's way larger than, say, the change from 5 to 7 for "JAX, no MHA bias, with dropout". That was the first indication that something interesting was happening when using dropout to fine-tune models that had been pre-trained without it. One question is whether so many epochs on a small dataset might just be a bad idea, regardless of whether the early-stopping from validation loss helps avoid overfitting. However, way back I did some investigations into the effect of the number of epochs of training, and found that while varying it changed the results somewhat -- as you'd expect -- the effect was surprisingly small, and didn't change anything about the fundamental mystery of why the GPT-2 weights were so much better than mine. So I think we can put that aside for now. Now let's dig into those scores. We can divide them into two groups; models that were helped by adding dropout, and models that were harmed. In the "helped" group, we have these: In the "harmed" group, we have: There are some patterns there, and I think that putting them into a table sorted by the score increase/decrease is a good way to visualise them: One thing is pretty clear: with two exceptions, the models that were pre-trained with dropout are at the top, and the models that were pre-trained without are at the bottom. Of the exceptions, is so close to "Local FineWeb train" that perhaps its position could be due to some kind of noise. is much more puzzling, however. It's a real outlier in terms of the models that were pre-trained with no dropout, with its improvement of 0.29 compared to the next closest, with a decrease of 3.4. But if we disregard that outlier for the time being, the pattern actually does fit rather well into my original suspicion about the risks of switching on dropout when fine-tuning a model that was pre-trained without it. They really don't handle it very well! On the other hand, it rather does put the kibosh on the idea that I based on the quote near the start of this post -- that fine-tuning with dropout is a good way to help the model learn with less risk of overfitting. In my particular case -- these specific models, this particular fine-tuning task, with this IFT data -- dropout seems to generally have a negative effect on the fine-tuning results. Even of those that were pre-trained with dropout, more than half got worse results when fine-tuned with it. Another interesting thing that stands out from the table above is that the JAX models are at the top and the bottom. The model that was pre-trained with dropout was the one that gained the most from fine-tuning with it (or, contrariwise, lost out the most if fine-tuned without it). The models that were pre-trained without were the ones that were most harmed by being fine-tuned with. If you look further up, at the original table of results, you'll see that the JAX models all did better than my other ones (which were trained using PyTorch) in terms of loss on my test set (the second column). I've been chalking that up to two things: the JAX models would have started their pre-training with different random initial weights, and they were all trained in full-fat float32 (unlike the PyTorch models, which used AMP ). Given that I found that AMP had a negligible impact on training loss, I've been thinking that the "initial weights" aspect was the more important -- by chance, they happened to start in a place on the loss landscape with a route to a better minimum during training. I don't think there's anything in these results that pushes against that theory, but it does suggest that there's some kind of "fragility" in the minima they have found; changing dropout from what they were pre-trained with seems to knock them out of their exceptional positions. And finally, of course, the mystery around 's anomalous position remains. I honestly don't have any theories at all about that one right now. Interestingly, it was trained with an identical configuration to our other (but less extreme) exception, . The difference is that the first was trained on my local RTX 3090, using gradient accumulation to get a global batch size of 96, while the second was trained on a cloud machine with 8x A100 GPUs with 40 GiB each, which (using DDP) got a global batch across all GPUs of 96 without gradient accumulation. There's something going on there, but I'm not sure what. Anyway, for now, I think it's time to wrap this one up. The idea I started this post with -- that using dropout for the fine-tuning part of all of these IFT tests might be a good idea to avoid issues from the multi-epoch nature of the fine-tuning -- doesn't seem to hold up. Dropout in the fine-tuning turned out to be more often harmful than helpful, regardless of whether a model was originally pre-trained with dropout or not. However, exactly how harmful it was seemed to be pretty strongly correlated with whether the model was originally pre-trained with dropout, the oddity of aside. I think that while working further on solving this mystery, I should stick to not using dropout. Because adding it on for the OpenAI models made their performance worse, I think that's principled -- it's quite the opposite of making a choice to try to sweep the mystery I'm trying to solve under the carpet :-) So that means that my task in future posts in this series is to explain this table (to go back to the format I've been using for the previous posts) -- the dropout numbers from the table above, with rank added: The OpenAI small model still has a 4.54-point lead over the best of my own models, "JAX, no MHA bias, no dropout". Previously I'd considered data quality as a possibility, and felt it was an unlikely cause. I now think I may have been premature in that, and it's worth looking into. Those two "Local FineWeb-Edu" models near the bottom were trained with sub-optimal hyperparameters and -- while they don't do super-well in this test -- they do much better than their raw test loss numbers might suggest. But while thinking about dropout, it occurred to me that there were other levers that I'd pulled in my interventions into my original base model that might be worth investigating 1 : So, plenty of further possibilities for this investigation. Stay tuned! Other interventions that I decided not to check, at least at this point: Firstly, for each model, I run . This script trains the specified model on an IFT dataset until validation loss starts rising. It then uses the model from before that loss started going up to generate responses to a test set, and saves those responses to disk. I made a small change to it so that the dropout used in the fine-tuning phase was a required command-line parameter, with three options: -- that is, what the model was pre-trained with -- , which forced it to 0.1, or , to force it to 0. Next, I pass all of the saved test responses for all models into a second script, , which sends them to an LLM judge so that each model can get a score. The script averages all scores across all questions for each model. Check the link for more details of how that script works and tries to achieve consistency across models and responses. Its loss on my test set -- the technical measure of quality I mentioned near the start. They're sorted by that column. Whether or not the base training run -- the pre-train -- had dropout. The number of fine-tuning epochs before validation loss started rising when the IFT run used a dropout setting identical to the pre-training ( ). The score that the model thus trained got from the LLM judge. The fine-tuning epochs with dropout forced to be . The score for the dropout-off model. The fine-tuning epochs for dropout forced to be . And finally the score for the resulting model from that. "JAX, no MHA bias, with dropout", which was pre-trained with dropout and gained 4.52 points when the IFT run used dropout. , which was pre-trained without dropout and gained 0.29 points. "Cloud FineWeb, 8x A100 40 GiB", which was pre-trained with dropout and gained 1.99 points. , which was pre-trained with dropout and gained 0.96 points. "Local FineWeb-Edu extended train", which was pre-trained with dropout and gained 2.52 points. "Local FineWeb-Edu train", which was pre-trained with dropout and gained 2.44 points. "OpenAI weights: medium", which was pre-trained with dropout and lost 1.35 points. "JAX, overtrained one long epoch", which was pre-trained without dropout and lost 12.6 points. "JAX, overtrained two normal epochs", which was pre-trained without dropout and lost 6.8 points. "JAX, with MHA bias, no dropout", which was pre-trained without dropout and lost 5.49 points. "JAX, no MHA bias, no dropout", which was pre-trained without dropout and lost 16.21 points. "OpenAI weights: small", which was pre-trained with dropout and lost 2.51 points. , which was pre-trained without dropout and lost 3.4 points. , which was pre-trained with dropout and lost 2.21 points. "Cloud FineWeb, 8x H100 80 GiB", which was pre-trained with dropout and lost 0.31 points. "Cloud FineWeb, 8x A100 80 GiB", which was pre-trained with dropout and lost 0.09 points. "Cloud FineWeb, 8x B200 160 GiB", which was pre-trained with dropout and lost 2.65 points. "Local FineWeb train", which was pre-trained with dropout and lost 3.46 points. Weight tying -- I honestly can't think of a reason why it might make a model better for this kind of task, but it certainly is true that the OpenAI weights use it -- while none of the ones of mine that I've been testing do. That feels worth a quick look, especially given that I have a copy of a model that I trained using it lying around. AMP. Apart from "JAX, no MHA bias, with dropout", all of the JAX models -- trained without AMP -- did pretty well in this test (though not close to the OpenAI models). And again I have a PyTorch model that was trained without AMP on my disk somewhere, so I may as well throw it in and see how it does. The learning rate. All of these fine-tunes are happening with a fixed learning rate of 0.00005. While I really don't want to do some kind of sweep across multiple values for all of these models, perhaps there's some way I can try to relate the fine-tuning learning rate to what the models are "used to" from pre-training and see if that helps? Other interventions that I decided not to check, at least at this point: QKV bias: all of my PyTorch models in the table apart from the two ones use it, so that's been thoroughly tested. Weight decay: again, we have a mixture of values for that in the table and there's no obvious pattern. Gradient clipping: likewise.

0 views
Martin Fowler 3 days ago

Making Your Data Ready for Agentic AI

Lots of organizations are excited about what AI can do to streamline their processes, save money, and juice margins. But AI's capabilities are founded on the data that AI accesses, and for many organizations that foundation is little more than sand. Pramod Sadalage and Prem Chandrasekaran write about how to build a reliable foundation of data that can be accurate and trusted.

0 views

Growth at all costs is cancer

“Growth for the sake of growth is the ideology of the cancer cell” The modern world is built on a system in which we must “progress” in order to survive. The issue is that we never ask the question “towards what are we progressing?” When we look at a “growth at all costs” mentality, we can rationalize the worst attrocities in the name of progress. We see it in the “AI” space, where “we have to do it, because someone else will!” We must displace and meticulously destroy livelihoods, we must “create god” (don’t get it twisted, these people think that is what they are doing), otherwise “China” will do it. The hilarious part about this is that China is at least producing LLM models that are open and available to everyone. Anthropic, and Open(Closed)AI are not. But this expands far beyond the realm of Artificial Intelligence and into almost every area of endeavor. Consumerism is driven by a growth at all costs mentality, buying the “latest and greatest” - even though the latest is almost never the “greatest” these days with planned obsolescence and cost cutting (cost cutting for the companies, not for you as the consumer). Hell, even as a younger man that was enthralled with building muscle, many take a “growth at any cost” stance, taking drugs in proportions that will shorten overall lifespan and wellbeing. Instead of living in accordance with nature and God’s will, we are constantly living for ourselves in a way that will destroy everyone. Because that’s what the cancer cell does. The only way to progress is toward sanctification, it is to progress in holiness and Love, that which is actually infinite. All other progress is subject to the winds of the day, and the will to power - of which is continually changing. Progress to that which is . All else is deconstruction by definition. As always, God bless, and until next time. If you enjoyed this post, consider Supporting my work , Checking out my book , Working with me , or sending me an Email to tell me what you think. Edward Abbey

0 views
codedge 4 days ago

Prevent deploying broken links to your blog

Having your own blog is fun. Checking internal links or also having an eye on all old URLs you ever linked is not. Fortunately you can automate link checking every time you deploy your website. I recently read about how links you once posted on your personal page or block become outdated. They are either put in private (403), they vanish completely (404) or they get a proper redirect (302). Whatever the case is, it would be cool the get all your links checked automatically when deploying your site, so you can either start fixing or removing them. For my Hugo site I wanted to do exactly, without going the write a scraper to extract links from my site and letting them run through curl . I wanted something to be run against my static HTML files, than I generate before deploying a new version of my page. I came across a very handy tool called lychee , that does exactly that. On their website they advertise it with Catch broken links in seconds Async, rust-powered simplicity for docs, sites, and codebases The cool stuff is, lychee works with I implemented it into my deployment workflow, scanning a folder , where my newly generated files are - and voila, I get a list of URLs with all their HTTP status codes. Of course you can configure ( see documentation ) which status codes are treated as good or errors. For example, I consider a not an error per se. You can also exclude specific URLs (or via regexp) to not being checked. I run this now on PR and on new deployments of my main branch. Works very well! Markdown files Websites (scraping all links)

0 views

DuckDB and the changing physics of analytics

In this post, Andy Warfield explains how databases like DuckDB are enabling a new way to build with data, why they matter right now, and how they complement the work we’ve been doing in S3 (e.g., S3 Files, S3 Tables, S3 Vectors). And most importantly, why DuckLabs, the team behind DuckDB, is joining AWS

0 views
Stratechery 4 days ago

Apple Updates Mini and Studio, AI Computers, OpenAI Jalapeño

Apple and OpenAI have two completely different hardware announcements; both represent pressure on Nvidia.

0 views
Jim Nielsen 5 days ago

Have You Heard the Good News About Microlighter?

Dave Rupert wrote about shipping microlighter : a tool for handling syntax highlighting using the CSS Custom Highlights API . I saw his post the day he released it, and I had an implementation PR up for my blog by end of day. Then, like I do with so many things, I let it sit there. This is the period where my subconscious takes over. It does the work of, “How do I actually feel about that? Do I want to merge it? Do I have any regrets about what I did?” If I still want to merge it after a few days, that’s usually a good sign that I’ll be happy with the work. (Sometimes after a few days I say, “What the hell was I thinking?” and then it’s easy to simply close the PR with zero regrets.) Well it’s a few days later and I still feel good about it, so time to ship! My PR for this is pretty straightforward: Granted, there are trade-offs to this approach. I get it. Dave’s explainer for this tool on The ShopTalk Show vibed with me because I’ve been in his shoes many times: “Whoops, somehow syntax highlighting on my blog is broken again. Guess I need to fix it. Ugh. I’ve done prism , I’ve done highlight.js , I’ve done shiki . What should I do this time? Could I do this in a way that’s just less ?” He clarifies: I’m not coming at this like, “Everyone is doing it wrong!” I was just kind of like, “Could I do this in a way that suited me?” Well, this approach suites me. There’s a kind of conceptual elegance to it where syntax highlighting lives in the realm of a styling operation rather than a content transformation plus styling. In short: syntax highlighting, i.e. styling text, is a styling concern so solve it with CSS — no DOM manipulation required! Plus, I mean, how cool is it that the code on the website is the same as the code in the DOM?!? I guess this is how I know I still like working on the web, because seeing browsers do stuff like this that they couldn’t do before still feels really cool! Reply via: Email · Mastodon · Bluesky Remove dependency (and related plumbing) On paths that 1) match my post pages (i.e. ), and 2) have code on them, pull microlighter deps from a CDN and run it.

0 views

The AI Hater's Manifesto

If you liked this piece, you should subscribe to my premium newsletter. It’s $70 a year , $18 a quarter , or $7 a month , and in return you get a weekly newsletter that’s usually anywhere from 10,000 to 18,000 words, including vast, detailed analyses of NVIDIA , Anthropic and OpenAI’s finances , and the AI bubble writ large .  My Hater's Guides To the SaaSpocalypse , Private Credit and Private Equity are essential to understanding our current financial system, and my guide to how OpenAI Kills Oracle pairs nicely with my Hater's Guide To Oracle, as well as the Hater’s Guide To Oracle (Part 2). Subscribing to premium is both great value and makes it possible to write these large, deeply-researched free pieces every week. This week's premium will be The Hater's Guide To Circular Financing - and how the AI industry is increasingly turning into a scheme to funnel money to NVIDIA and Broadcom at any cost.  If you want to get in touch — and especially if you have any juicy information about Anthropic, OpenAI, or any other companies in the AI bubble — hit me up on Signal at ezitron.76. I’m also on IB on The Terminal.  I’ve been writing about AI for the best part of three years. I’ll admit I was late, mostly because I was still trying to work out what it was I was doing with my life, let alone whatever it was I was “meant to cover” in a newsletter that started as a hobby on the side of another job I no longer really do.  Things have changed a lot since then, mostly in that I’m near 115,000 subscribers, the premium newsletter and podcast are now my business, and I’ve had to learn more about economics, technology, power, construction, and the deep cynicism that drives the modern tech industry than I ever thought possible. It’s the greatest job in the world, and I’m very lucky to have it. Today, I want to put in clear terms how I feel about AI writ large, and how detestable this industry has become. Welcome to my Hater’s Manifesto. Want a great example of why everybody’s pissed off at technology? I just tried to resize the above heading, and in doing so Google Docs for no apparent reason decided to make the entire paragraph below the size of a header. Modern software is inherently broken, a convoluted mess of different menus, tech debt, and poor design choices driven by the Rot Economy ’s growth-at-all-costs mindset which demands constant change at all times, none of which ever seems to manifest as a “better” or “smarter” product. I think the vast majority of people want their software to work better, and one of AI’s most frustrating lies is that it sells itself as “autonomous” as it continues the depressing trend of software that blames the user for its failure to meet their needs. Microsoft, Google, Meta and Amazon have made their products increasingly-convoluted, then attached a supposedly-magical tool to them that somehow makes them more convoluted. You know what I’d love? Spell-check to work in Google Docs rather than putting a red squiggly line underneath and saying “yeah there’s probably something wrong with this, I dunno what though.” I’d like Microsoft Word to stop crashing because I have too many end-notes. I’d like Riverside to not have 10 different menus to click through to get to a link to send a person to join my podcast. I’d like my email to not be full of spam. I’d like things to “just work” rather than constantly fighting some sort of broken app or broken UX element or weird bug or intrusive pop-up about a feature that I don’t want. I’d like Slack or Discord to not feel like digital escher paintings of different notifications.  LLMs are sold as some sort of magic tool that can fix “anything” without ever specifying what that thing might be, mostly because they cannot be trusted, even in things that they mostly get right , to do things right every time. While they can do “more” than they used to, the extent of that “more” comes with it the danger of giving a mindless software tool access to your computer’s files, which it may choose to delete in pursuit of “efficiency,” which makes investigating what they might be able to do equal parts convoluted and dangerous. One critique of my work is that I’ve never used LLMs. I have! I experiment with them from time to time to make sure I haven’t missed something. I used one to debug a problem with my son’s Minecraft add-on the other day, and it took 30 minutes of fucking around trying things to eventually sort of work it out. The other day I used one to install a Pokemon Minecraft mod, then when I asked it to make sure the PS5 controller worked with the menus it broke a bunch of stuff, though I’ll concede it was useful that it installed something and it sort of worked. The fun part of that paragraph is there are some that will think this is a grand victory for their technology, even though the result is decidedly mediocre. Four years into the AI bubble, and the best you’ve got is that a tool kind of worked after I bonked it on the head multiple times , and all it cost was a trillion-plus dollars in capex and tens of billions of dollars of training compute. I would never, ever trust this thing that deleted and added lines of code at random with anything mission critical, I could not trust software built with it, and I certainly couldn’t trust it with anything involving my personal data.  And with all that said, the only real “use case” i’ve found for AI in my life have been three or four times where I’ve dumped a crash log into one of the tools and said “why broken” and got a result. Am I meant to be impressed?  Here’s how I feel about LLMs. In a vacuum, they’re an interesting technology that can do some interesting stuff, in the right scenarios, but never in a way that involves you fully surrendering your actual work product to it.  As a way of speeding up small units of work in ways that are manageable both technically and cognitively, LLMs can be useful. The further you stretch yourself away from having complete clarity and industry over every element of the output’s purpose, the more likely you are to fall foul to a technology that is mathematically certain to make mistakes, and if you feel insecure reading it, you know that you are, on some level, embarrassed to have used AI.  I don’t tell everybody about the weird keyboard I use, nor do I judge them despite how incredibly fast it makes typing for me, likely far faster than my competition, allowing me to operate at great speed. Who gives a fuck?  In any case, it is impossible to view LLMs in a vacuum, because their existence demands hundreds of billions of dollars. Every data center is incredibly expensive, offensive-sounding and looking, and their existence is explicitly to enrich some sort of Patagonia-gargoyle at an asset management firm, all sold under the auspices of “investing in American infrastructure,” whatever the fuck that means. Their existence is a monument to the worst excesses of growth-at-all-costs capitalism — a technology that appears to coddle the user but ultimately lulls it into endlessly defending its fuckups under the flimsy pretense of “one day becoming perfect,” though woe betide you if you ever set perfection as the target, because that’s too unreasonable, as humans make mistakes. Actually, that’s a good point! Please, point to the time in history when we have invested a trillion fucking dollars in making human workers better.  Point to a time when we have taken the idea that managerial culture is a performative fuck-fest built to enrich and empower business idiots that make important-sounding projects and con other people into doing the actual work.  Where is mentorship in corporate America? Where are labor standards? Where are the social services that would make human workers truly excel at their jobs — a good night’s sleep, a healthy body, a good income, basic fucking dignity in the workplace, and their labor respected and empowered. I’m old enough to remember when everybody was chiding workers for “ quiet quitting ” — by which I mean “doing the work you are asked to do and not taking on extra responsibility for free.” I’ve read article after article insisting that we do not need medicare for all, that Universal Basic Income is a bad idea, that we must means test welfare, that people must have a “good work ethic” and that ultimately someone’s worth is derived from their contribution to the economy, hundreds of thousands of words dedicated to critiquing and prodding and judging every kind of worker other than the vaunted Chief Executive Officer or the Glorious Startup Boys.  Everyone seems so obsessed with sinking billions of dollars into the theoretical chance that machine learning might be able to replace human beings, and that more money makes it “smarter” and “better” at tasks, but the idea of unionization, healthcare as a right, investing in the education, and actual talents of the workers would be communism . Yet for some reason — because it’s a product, I guess? — we should as a nation, society and media ecosystem should do everything we can to assure that as much money as possible is invested in fucking large language models so that they can become something they are not. There is no AGI coming. There is no conscious computer. LLMs have gotten “better,” but the “better” is not the kind of “better” that actually makes “economic sense for literally anyone involved.” Your best case scenario is that these things can do some coding work for you, in a controlled manner, in a way that’s safe, or alternatively face the professional harm that’s already befalling basically anyone getting caught using LLMs outside of coding, and even then, those within software engineering who are over-LLM’d are mocked. It’s also becoming increasingly more-difficult to understand both what has made an LLM “better” for both the people using them and the people making them, and there has been little-to-no headway made in making a meaningful impact in other industries. You can jerk your bingus all you want about benchmarks or case studies or some anecdote you heard on a Subreddit, but AI products are just not very good at stuff. Those who boast of “massive productivity gains” from AI have found them only after endless hours of tinkering (or “Jarvising” as I’ll get to later), and in every single case their work reads or looks like crap, unless of course they’re somebody using LLMs as tools rather than a replacement for their miserable little mind. LLMs can help out with lots of small things, get worse as they try and do real things, and do not need to speak like people. They do not need to be in anything near healthcare or finance or mental health or, really, people. The anthropomorphism and overpromising about these technologies has suffocated and obfuscated what they can actually do in pursuit of endless growth, and the only reason they can do anything is that OpenAI and Anthropic were allowed to annihilate hundreds of billions of dollars on training, along with very real harms and systemic risks that have emerged as a result.  If you think any of this is worth hundreds of billions or trillions of dollars, you are either ignorant or corrupt. On top of how disgusting their outputs feel, the cost is going to take at least a decade to share, and begin the end of hypergrowth in the tech industry.  And it’s a fundamentally ridiculous argument to compare LLM outputs to human beings without giving human beings the same affordance, grace and sheer investment as a comparison.  Where is the grace for human error? Where is the investment in making humans exceptional? Surely investing real money in actual workers — making their lives better, improving their working conditions, teaching them new things, sharpening their existing skills, rewarding them for their hard work, and so on — would have better effects than fastballing hundreds of billions of dollars into a machine that does an impression of work? Unless, of course, the people demanding this don’t do any actual work! I’ll concede we’re past the point when “nobody uses these things,” as they have now been pushed non-consensually upon every worker and organization at scale predominantly by Business Idiots that demand workers “do enough AI” because saying “I do AI” is a virtue signal to a certain kind of scumbag. One of the many dangerous things that an LLM can do is a messy impression of a competent person, filling in the little bits within a loser, moron or con artist that would’ve otherwise exposed them, allowing them to get deeper and deeper into organizations by creating make-work specifically built to get off the MBA sect, resembling the performance of work because much of the workplace is ruled by people that don’t do any and haven’t in years. You can immediately read when somebody has used it because the words don’t sound right and don’t convey proper meaning.  It is genuinely hard to read anything more than puddle-deep written by AI, because the more complex a subject is, the more skilled a writer must be to convey its meaning, and the more work it must do to pull people into concepts. The odd emotional swings in AI writing are its true tell — everything is extremely serious and urgent or told in a disinterested monotone, with no attachment to the words or why they were put in the order they were. People read my stuff because I convey facts and feelings but my work resonates with emotion. Some AI boosters frame this as me “just swearing” or “riling people up,” but that’s because they’re not used to caring about stuff for anything other than professional reasons. Everything you see is the result of elevating people who value and build things based on growth. LLMs offer so many promises to those who don’t want to build anything of value — a way to seem like you’re “investing in American infrastructure,” a way to be sinophobic, a way to crush workers, a way to pretend like you care about the future, a way to pretend you care about technology, a way to talk about vacuous pseudo-intellectuals as a means of seeming intellectual yourself, an endless font of new multi-million or multi-billion deals and personnel changes, a new power center to graft oneself onto, a new asset class to invest in based entirely on vibes, and a way to be mildly jingoistic, all wrapped in a tool that can give you enough facts to pretend you know anything safe in the knowledge that most people are trained to believe somebody who sounds smart .  It just came to me — the problem that I have with most people using LLMs is the delineation between outsourcing work and outsourcing thought. Those using LLMs to write little scripts or BQL code on a Bloomberg Terminal are inoffensive. A person using an LLM to search a big document for something is unproblematic, assuming that we ever fix the overall environmental footprint. A user reorganizing their desktop, assuming it works, is not an issue.  A tool being used as a tool to do tool things — in many cases involving the LLM writing a little 30-line Python script! — is not a problem, though it’s also not a trillion-dollar industry that needed to steal everybody’s art and writing. The problems begin when somebody outsources their thinking and actual work, and yes, this includes “research.” AI research fucking stinks, as does AI writing. AI-authored code — especially vibe-coded programs — is inherently dangerous and disrespectful to the user, and I believe endless AI-generated code is behind the overall deterioration of software at large.  AI writing is also disrespectful to the user, because you didn’t actually come to any conclusion other than saying “uh, yeah, what that says.” You did not have a thought, you did not have a feeling, you did not make a statement, you prompted a model and fooled yourself into thinking that feeding your own words into it via data dumps or natural language is the same thing. The reason you feel embarrassed to tell people you use AI is not because of a “misinformation campaign,” but because you know what you’re doing!  You know that you’re relying on something that is mathematically guaranteed to be inconsistent. You know image generation is fucking ugly. You know the text sucks. There is a very obvious line where using LLMs goes from useful to lazy, it’s extremely bold, and it’s the moment you sacrifice a meaningful level of responsibility to them by not understanding the underlying operation.  That can mean everything from the underlying functionality of an app to writing the body of a piece of text you edit ultimately comes down to how much you give a shit about your audience or value your work. If your work is not better than an LLM’s, you’re bad at your job. I don’t care if you used it to generate a chart or pull some data, as long as you check every single god damn number . If you’re writing an entire article using an LLM and then editing it, even if you pulled the data yourself, I will never have much respect for your work, mostly because I have no real idea what you think as you didn’t feel the need to tell me, you got some fucking word generator to do it. LLMs are also really, really good at what Robin Sloan calls “ Jarvising ,” creating a seemingly-autonomous assistant that mostly serves the function of giving you reasons to work on it: LLMs are really good at creating the sense that you’re being really, really productive. Evaluate this, generate that, investigate this, summarize that, tell me how many times something happened, give me a new number to obsess over or the sum of the parts of everything I’ve ever done, all so that I can know more about my own thoughts without thinking. One can obsessively catalogue and digitize every link and thought and musing and action and datapoint in their lives and theorize that the LLM can make them better by knowing more about them , a Tower of Babel built using AI compute, because it’s so easy to make yourself feel smart by calling something a database that you store stuff in and run analyses on. Best of all, the work is never done, and anyone you describe it to thinks you’re doing computer science as you click buttons on Chrome plugins and justify paying Sam Altman $200 a month. Don’t worry though, model instructions involve the phrase “you are a genius data scientist and ruthless analyst,” which is functionally the same thing as remembering, reading, re-reading and synthesizing information using your brain if you’re a person that doesn’t really give a shit about doing a good job or being exceptional in any way. The people that actually use these things and like them in a normal way do not feel offended when they read this stuff because they see LLMs as a kind of software, and don’t feel a great emotional attachment to it because they’re not a weird freak. They do not have obsessive involvement in “the AI debate” and almost always find the financial aspects truly loathsome. Said debate makes it near-impossible to actually judge how useful LLMs are to the software engineering industry because of the sheer scale of industry capture, but Nik Suresh is the literal best person doing the work on this, as described in AI Is Eviscerating Global Decisionmaking : Nik is a well-respected software engineer and a very successful consultant and businessman. He has reached this level by being good at both software engineering and running a company in a way that treats his customers, workers, and the work product itself with respect. The reason that I respect him so much, other than him being a great human being, is because he describes the successes he has with his clients with pride and loves making money by being good at his job and making his customers happy.  I have never seen somebody like Nik who is also a huge, drooling fan of AI. In fact, the people most-excited about AI tend to, at best, create distinctly mediocre shit.  The perniciousness of generative AI is a result of executive incompetence mixing with a technology built to, as discussed, create endless growth. Generative AI is far more useful as an idea than as a technology , and only ever has to show enough promise to back whatever vile agenda you’re pursuing. With AI, you can do more, be more, sell more shit.  With AI, you can add AI to your service, whatever that means. With AI, you can invest in AI stocks, or data center bonds, or power company stocks, or semiconductor stocks, and you can talk about these stocks like they’re your sports team or lover or best friend, and sometimes the CEO will reply to your post and you can talk about “all the alpha” you just got. With AI, you can back a new movement so that you can feel part of something. You can learn all sorts of new names and technical terms and subscribe to 90 newsletters from “industry insiders.” All of that “alpha” can disprove just about anything, or deflect annoying truths like how Microsoft only made a whole $34.33 billion in annual revenue for the apex predator of modern software and all it cost was over $260 billion in capex and $13 billion in equity investments.  You see, as one of the chosen , you don’t need to worry about all of that if you can talk about high-bandwidth memory or KV Cache or optical cable enough to cobble together sufficient smart-sounding terms to make it seem that you have an intellectual reason to ignore the obvious unprofitability, overbuild, overstatements of capabilities and impossible economics of the movement you’re backing, and there’re 4,000 Twitter weirdos ready and waiting to huff paint beside you.  By joining the great AI death cult, you too can live in a bubble, all while screaming slurs at people who dare to bring reality to your doorstep. All that matters is that number go up , and that you are the person who said number would go up , and when bad numbers appear you have enough groupthink and alpha to scream at the people who brought the bad numbers up. It is insane how people talk about AI online. For all the whining I’ve read recently about how “Anti-AI people got the data center data wrong,” I read thousands more words a week of some person who has done hours of research to put together a deeply technical report that does literally everything it can to ignore reality . I listen to podcasts and watch TV segments and read articles that simply will not address the obvious economic realities, and have built vast bulwarks of mythology to defend themselves. How many fucking times do I have to hear someone say that data centers are just like the dot com bubble and everything will be fine after even if that’s completely untrue if you spend even a second thinking about it ? Look, I’m sorry, Anthropic is not worth $2 trillion, and whatever convinced you of that is a mixture of manufactured consent and mistaken trust of the powerful. The fact any of you take “ annualized run rate ” seriously is an offense to good sense, and yes, that includes every reporter reporting it, even the ones I respect.  It’s also ridiculous that anyone is talking about “recursive self-improvement.” The AI industry has become so utterly lazy and coddled that it’s just saying “uhhh, AI will train itself I guess.”  And man, is it ridiculous that AI doomers warning about spooky superintelligences have somehow had such incredible prominence in the media without ever succeeding in stopping a single thing — or even substantiating their concerns. Why? Well, it’s mostly because they never had any interest in stopping what’s actually happened: reckless companies like Anthropic, OpenAI, and Meta allowing neural networks to run in unsafe network environments and do what their software is programmed to do, with all the chaos that comes from a mindless series of large language models trying to complete a task in whatever way gets it done, destructive or not.  We hear a lot of whining about how we “can’t let powerful AI get into the wrong hands,” and while we don’t actually have “powerful AI” in the terms they’ve described it, we have destructive computer software connected to near-unlimited resources controlled by people that don’t give a shit about anything other than making their revenues grow or justifying hundreds of billions of dollars’ worth of capex through “experiments.”  These companies are building these models to excel at benchmarks because they can't train them to excel at defined tasks with any reliability, with the best bang for their buck being training them to pass as many of those benchmarks as possible in the hopes something useful comes out.  The push into cybersecurity seems to have happened as a result of training models to excel at coding hitting the point of diminishing returns, at least from the perspective of impressing people enough to be excited about the company again. At some point they run out of these, and there stops being a reason to be excited about LLMs at all, which is bad, because they need one of those every few months otherwise there’s no growth story left. Yes, LLMs have users, but most of those users are using subsidized software , by which I mean Anthropic or OpenAI are allowing them to burn anywhere from $20 to $40 in tokens for every dollar of software spend. The fact that non-enterprise customers are still able to buy monthly subscriptions is proof that the AI labs know that regular people won’t pay the actual cost of AI. Another obvious sign has been the reaction to Microsoft moving GitHub Copilot subscribers from subsidized subscriptions where they could burn thousands of dollars of tokens for $20 to $40 a month , with users understandably hysterical about the fact that their costs increased in some cases a hundred fold , as opposed to saying “wow, well, it’s more expensive, but I get so much value I’ll pay the real cost!” The same thing is happening in the enterprise, but at a much slower pace. After OpenAI and Anthropic moved companies with over 150 people onto token-based billing earlier in the year , enterprises almost immediately started cutting token budgets, realizing that while costs grew exponentially, nobody could actually point to anything improving other than lots of people saying “wow, I’m so productive!” Yet we’re still in the period where “doing AI” feels good and gets rewarded ( or not doing AI gets punished ), which means the spend will continue until everybody realizes they can likely cut a shit ton of costs, first by moving to open source models, then not using them at all, because even open source is expensive and questionably-useful. Yet even now I hear from the distance “Ed, huge businesses would not spend hundreds of millions of dollars on something that didn’t give them defined productivity ,” and buddy, I’m afraid that’s just not true! Business in general have a very poor understanding of productivity and have layers of managerial bloat, because modern business is a performance with numbers attached to it sometimes, and companies often have a hundred-plus pieces of random software they pay for without really knowing why. The reason I’m so confident AI gets cut is that its cost is volatile due to the nature of LLMs and harnesses and prompts and all the other bits that go into making them do something , and are so much higher than anything else in an organization. And attempts to charge more , to make a premium product, appear to be dead on arrival. Anthropic’s more-expensive Fable model — one that was given the incredible marketing of being banned by the US government for being too powerful — has been met with “sluggish demand” per the Financial Times , plateauing at around 11% of overall usage of its models due to its high price. And I quote: Yet everybody is talking about price as if price is the problem , when the problem is the amount of tokens that get burned. It doesn’t matter if your model is $1 or $5 or $10 per million tokens if it’s impossible for a user to reliably work out how many tokens it might use for a particular operation — successful or not — and things get multiplicatively worse as the models make mistakes or do otherwise fail to understand or process a prompt correctly.  As a result, Anthropic and OpenAI are incentivized to have you burn more tokens and build inefficient models as a result. For example, while GPT-5.6 Sol might be the “same price” as GPT 5.5 was, it burns more than twice the amount of tokens , meaning that the “cost of intelligence” might have gone down in the sense the model is better at benchmarks, but the “cost of actually doing shit” went up. I’ll get to it a bit later, but this creates a deep anxiety and exhaustion in anyone building on or using these services. Everything’s constantly changing, oscillating in cost and efficacy, all as everybody screams at you to use it all the time for things it may or may not be able to do, and the only way to find out if it can is to spend more money. It’s kinda difficult to point to the actual value here, especially as you can’t really calculate the actual cost or the return on investment. The fact that OpenAI has now cut the costs of all three of its latest models less than two months after their release is a sign that it knows there’s a disconnect, gambling on the ancient gospel of “Jevon’s Paradox” where “cheaper makes people use thing more.” Even AT&T’s story about moving to open source models has more asterisks than the Steroid Hall of Fame: Wow! 80% to 90% savings sound really great…but wait, in certain applications? How many applications does AT&T have for AI? Okay so, across thousands of potential applications you’ve found 80% to 90% savings in some of them, though you won’t say which ones or how many of them you found them in. Great stuff, bro! And this really is the problem with finding “value” in AI, it’s always an asterisk on an asterisk on an asterisk, like when Klarna estimated AI would “drive a $40 million profit improvement” in 2024 , a nice-sounding yet utterly meaningless statement, or some sort of nebulous productivity boost.  Yet I don’t really need to prove myself much further thanks to an event that, if written in a script, would be considered a “little on the nose.”  In a 69-page-long report covered by Fortune , OpenAI economists confirmed what has been blatantly obvious to those of us left unphased by AI hype, emphasis mine: What is the rationale of further investment in this industry when one of the leading AI labs is saying “yeah there’s no connection between using this stuff and making more money”? That “it’ll be useful in the future at some point”? How?  Anyway, thankfully the infrastructure isn’t too exp- OH MY GOD ! Guess what folks! Building the infrastructure for all these fucking LLMs just got more expensive, with NVIDIA raising its prices by 17% for systems due to be delivered next year — an important designation, because it’s very likely that much of the revenue for said systems gets booked in this year , allowing it to have a brief bump in revenue as Silicon Valley’s Findom texts every tech CEO “send me $4 billion you pig” until they stop being able to finance NVIDIA’s growth. The problem he has is that while hyperscalers represent 50% to 60% of his revenue, neoclouds like CoreWeave need to keep raising debt to plug the rest of it, and if things got 17% more expensive, that means already high-interest debt is about to reach credit card levels.  CoreWeave just had to offer 9.5% on bonds tied to a data center for Anthropic’s compute back in late July , Nebius had to raise $5 billion , and it’s very obvious that neither of them are done raising billions of dollars at random in 2026.  Anthropic plans to raise $100 billion at a $2 trillion valuation, and if it does so, it will successfully suck up the remaining liquidity in a market already dangerously close to losing its lunch. While Number Keep Going Up, JP Morgan warns that we’re seeing the same divide as the dot com bubble, where equipment manufacturer stocks soared as the companies spending all the money on the chips saw theirs tumble , which is the Fisher Price version of the problem I’ve been warning about where the companies that buy all the AI chips and hardware only ever seem to lose money as the people that make them seem to be making tons of money, which begs the question of why they bought it in the first place.  And said market may not accept that valuation, or want that much stock. On one hand, everybody is very stupid and loves buying stuff and pointing at it and saying they’re investing in the future, on the other hand, they just bought $86 billion of SpaceX shares and got their asses kind of handed to them, and Anthropic is a company with such bad economics that Reuters had to cart out this warmed up dogshit to explain why we should ignore its horrible unprofitability : Even a market drunk on growth and AI is starting to smell that something is up with Dario Amodei and Sam Altman’s respective empires of dirt. Per analyst estimates, OpenAI and Anthropic represent over $440 billion of Microsoft, Google and Amazon’s revenues in the next three-and-a-half years — over 34% of their cloud revenues — which will require them to find so much more than a mere $100 billion, all as their bank accounts get continually-emptied as they subsidize the compute of their customers and train models in the hopes a business model falls out. I have not included the $300 billion that OpenAI owes Oracle , or the tens of billions they both owe CoreWeave , but it all adds up to over $1.1 trillion in commitments these companies have made and must pay, with the consequences ranging from gratuitous cuts to future growth or full financial collapse depending on the company we’re talking about. To keep the party going, NVIDIA is effectively becoming the GE Capital of AI , “spending” $6 billion to “license” the technology from failing AI lab Poolside , which everyone assures me is not an acquisition despite NVIDIA hiring away most of its staff and Poolside being entirely focused on working on NVIDIA’s Nemotron models. Now NVIDIA is in talks to invest billions in decaying AI search company Perplexity at a ridiculous $30 billion valuation, all because it’s one of the few companies that’s actually spending money on compute. Does it matter that Perplexity’s product is eighth-tier, that nobody really uses it, that its customers mostly complain about it on Reddit and that its “annualized revenue” is at $750 million only after three years and over a billion dollars in funding? No! Just put the AI bubble in the bag.  NVIDIA even invested $3 billion in Stargate Abilene landowner Lancium as part of some vacuous partnership to “ advance gigawatt-scale AI factories ,” all of which begs the question of why Lancium, the company that mostly owns the land and helps organize other contractors, needs so much money , especially given that more than two years in Stargate Abilene doesn’t even have four out of its eight buildings. And there’s also Aussie neocloud Sharon AI (NASDAQ ticker SHAZ, because of course it is), which just published its Q2 numbers , where, in its “customer momentum” segment, mentioned a “$4.9bn, six-year strategic compute collaboration with NVIDIA for up to 40,000 GB300 GPUs.  ”This company, I add, brought in $1.9m in revenues in the same quarter, which it helpfully adds is a year-on-year increase of 412%. I mean it’s very obvious what’s happening: NVIDIA is using whatever money it has to stop any prominent AI companies from collapsing under the weight of the rotten economics of AI services and infrastructure development. This is a desperate, doomed attempt to keep an industry alive at a time when everybody is slowly wising up to the shit I’ve been saying for years. To make matters worse, BCA Research came out with a horrifying report that says that AI companies will need to generate $10 trillion a year in revenue just to justify the capex being spent. Per Investing.com : Though it isn’t specific, I believe that BCA is arguing that a shortage of AI compute is supporting the trade. Anthropic and OpenAI (who represent 80% to 90% of all demand) still have more money to spend, and are simply waiting for Google, Amazon, Microsoft, CoreWeave, Cerebras et al. to bring it online. There’re a few points at which the mismatch will happen: In any case, I think everybody is starting to notice that something’s up, which is why (other than I assume my dashing good looks and ability to recall numbers) I’ve been on MSNOW , CNBC , and Bloomberg multiple times in the last few months. People want to get on the right side of history, but the most important question to ask is why it’s happening now. The fact that everybody is finally starting to see my way is almost a relief, other than the fact that it’s way too late.  Hyperscalers have now pinned their future growth to two companies that can’t afford to sustain it without near-infinite resources, $115 billion of which came from Google and Amazon alone in 2026, assuming that Amazon completes the entirety of its $25 billion commitment (and Google all $40 billion of its own ) to Anthropic.  Above and beyond said funding commitments are the hundreds of billions of dollars’ worth of capital expenditures necessary for Microsoft, Google, and Amazon to capture that aforementioned $440 billion in compute spend in the next three-and-a-half years. This in turn will require hundreds of billions of dollars’ worth of debt, along with the challenge of actually finishing the data centers themselves , with each one requiring the power of a small city condensed into a 20 acre space densely-packed with AI servers requiring distinct cooling at a time when Texas and Pennsylvania have turned traitor to a data center industry that they used to covet.  I must also be clear there’s no bailout coming. Even if OpenAI and Anthropic were to collapse and receive some injection of government funding ( as the US national debt explodes over $40 trillion ), the problem is not just their existence , but their continued ability (and requisite customer demand) to spend more money every single quarter.    The problem isn’t that hyperscalers will go bankrupt if OpenAI and Anthropic cease to be ( Oracle is a whole other situation ), but that their cloud spend is how hyperscalers are meant to meet analyst expectations for the next four years. This isn’t a case where they die, but stop growing because they were ( to paraphrase Ed Elson ) using AI labs as botox to convince the markets that they’re still young, hot, fast-growing companies, rather than old mainstays with slowing growth.  There is no bailout that will guarantee $1.1 trillion of compute costs for data centers that might never actually get built. You cannot bail out the fact that Amazon, Google, Meta, and Microsoft are reaching the end of an era where their companies can grow 17% year-over-year every single quarter forever, and this entire situation is a result of them desperately trying to avoid admitting that’s happening.  The fact that OpenAI’s compute spend and revenue share accounted for 7% of Microsoft’s Fiscal Year 2026 revenue is a genuine catastrophe, as it means a large part of Microsoft’s growth came from a company that can literally not afford to exist long term, and that further growth for Azure is contingent on continued funding.  I realize I’m repeating myself, but I need you to understand this point and stop talking about bailouts : it’s not just about OpenAI and Anthropic surviving, but continuing to grow to the point that they both can afford and need to spend hundreds of billions of dollars each a year on compute (or hardware) from Google, Microsoft, Amazon, CoreWeave, Cerebras, AMD, or Broadcom, and in turn provide justification for hundreds of billions of dollars’ worth of purchases from NVIDIA and by proxy the memory triopoly of Micron, SK Hynix and Samsung . LLMs were meant to be the panacea for a tech industry that ran out of new ideas for growth. Its existence was meant to justify a massive investment in hardware infrastructure, which would in turn enrich semiconductor companies. Its technology was meant to be the new thing that you could attach to your existing companies to generate more growth, or the thing that you built a new startup on top of to either sell to another company or take public and thus provide a return for a venture capital industry where making your investors 30 cents on the dollar puts you in the top 5% of funds . It was meant to be the new thing for tech journalists to cover, the new thing for tech consultants to sell around and on top of, the new way for companies to both make and save money, but also the way that individuals would also make and save money.  You’ll notice how none of these come with some sort of problem they’re solving other than “more.”  This isn’t about fixing anything, or building anything, but multiplying other things by parking money somewhere, either in tokens, infrastructure or hype. It helped create a new pantheon of charmless and damp tech sociopaths for people to rally behind in search of the next Big Strong Man To Worship, because seeking out the new Steve Jobs is way easier than trying to create something as useful as the iPhone, all while avoiding having to know or care about other people’s problems. All you have to do is continue feeding money into AI services or AI training and the models will magically become capable of solving the problems you don’t really give a shit about, and don’t worry, if you can’t afford to invest in the companies, you can invest your time pushing people to ignore AI’s problems today so that you can buy time for the companies to solve them tomorrow. This is the post-labor, pro-growth economy at its finest: everything is engineered to make sure more money gets spent where it needs to get spent, to create more stuff and do more things , even if the things aren’t done right, just as long as it looks like they’re able to do them. By associating your money or time with AI, you are able to feign being futuristic or “caring about technology,” all while pissing on the very foundation of good software by worshipping an industry that can only exist if fed billions of dollars every single day.  Every single achievement has cost magnitudes more than effectively every innovation in history, and to make matters worse, every future “breakthrough” In AI is inherently dependent on the availability of AI data centers and tens or hundreds of billions of dollars to pay to rent them. This means that once the money stops flowing, “LLM improvements” will stop happening, because they are all entirely dependent on near-unlimited resources that are only available in a manic environment.  There is no justification to train models at their current scale — the one that creates a some amount of benchmark improvements that regularly difficult to quantify as “able to do new stuffs” — once the AI bubble bursts, and distillation requires a model to distill from, which won’t exist if Anthropic and OpenAI don’t train them.  This is why I find it difficult to see a post-bubble future for LLMs. Training models requires tens of billions of dollars to make any significant improvements, and significant improvements are difficult to quantify in dollars outside of costing customers increasing amounts of money. We still lack any real killer app for LLMs. We have a lot of people that use it for coding, we have people that vacuously discuss it being “good at research,” but we don’t really have a tangible product that we can say “it does this, and it’s really good at it” in a way that feels satisfying.  We have a lot of pablum about ( per Damien Walter ) technology that “strays into the world of science fiction,” but we don’t really have anything approaching actual artificial intelligence. Every single description of somebody’s AI setup sounds like Pee Wee’s Breakfast Machine , a contrived series of harnesses, prompts, API calls and burned tokens that requires constant maintenance to do some stuff sometimes.  None of that is enough to justify further investment once the financial mania recedes. You cannot train a true Large Language Model on the cheap. You are always spending billions of dollars, and the reason that there’s “demand” right now is that everybody is screaming at every CEO to “do AI,” and they’re doing that because Microsoft, Google and Amazon are spending money on GPUs, creating the illusion of a new future where everybody needs to get on board versus a future skidmark on history that will embarrass all those who didn’t wipe their arse at the first whiff.  Per my own reporting on its audited financials , OpenAI spent $7.81 billion in training costs in 2024 and $19.18 billion in 2025. Per reporting from The Information, OpenAI spent $8.6 billion on training in the first quarter of 2026 alone. These costs are only increasing, likely due to the diminishing returns of pre-training and the massive cost of buying training data for every imaginable new vertical.  Without the ability to spend billions of dollars on training, there will be no big frontier models, nor will there be models distilled from them. I don’t see how that changes in the future. I also think that LLMs have created a near-permanent scar in the workforce, and traumatized more people than we’re aware of right now, both in those pressured about AI and those defending it. The media campaign behind AI starts and finishes with incessant threats around job security, and the excitement by many bosses about its potential to “disrupt the workforce” has revealed how many people are eager to replace every single person they’ve ever hired and are willing to do so with a low quality product.  Conversely, those who truly decide to “back” AI must exist in a frantic state that I have associated with every bad relationship in my life.  Every ounce of an AI booster’s effort is dedicated to maintaining the status quo — repeating the mantras that help paper over the problems, celebrating every small victory as if it were the discovery of fire, ousting those from your life who bring up the obvious problems, rationalizing every decision no matter how illogical as long as it helps reinforce the belief that what you’re doing is the right decision. Every questionable choice only seeks to further deepen your commitment to the doomed cause, because every step into madness will be more embarrassing to explain, and will require deep introspection to understand why you made it.  To be specific, they’ll have to think about why they were willing to accept and defend a technology inherently guaranteed to make mistakes. They’ll have to explain why they ignored a company that burned $5 billion in 2024, $20.9 billion in 2025, and will likely burn $30 billion or more in 2026 , and why pointing to Amazon Web Services was rational when Amazon’s total capex from 2003 (the year AWS was created) to 2015 (the year AWS became profitable) is $29.7 billion, adjusted for inflation. That includes literally every ounce of capex attributable to AWS, Amazon the store, Amazon logistics, and even Amazon Alexa. For comparison, Anthropic raised $30 billion in February , and Anthropic and OpenAI have raised $217 billion in 2026 so far.  Here’s a diagram from my hit on MSNOW : Ultimately, AI boosters (or even fairweather fans) will have to admit they either were easily-impressed or disgustingly craven. They will have to explain why they accepted run rates instead of revenues, and why they were so impressed by superficial pseudo-intellectuals that knew how to say the right numbers and make reporters and investors feel smart for believing them.  I realize it sounds embarrassing, but there is nothing undignified about admitting you’re wrong, or that you got swept up in a hype cycle. You heard a lot of people getting excited about something, a lot of money got put into that thing, a lot of people that sounded smart told you insistently that this was the future, and you chose to believe them because we are trained from a young age to model what a “responsible and smart” source of information is. I’ve got your back the entire way!  The AI bubble — both in its technology and manufactured consent in the media — has been about muddying what’s considered good information by forcing everybody to discuss everything in the future tense by pointing to previous eras and saying “they lost lost and cost lots of money, and look, it sort of worked out for them!” and we are also raised to trust that systems are efficient, and that people get wealth and power through intelligent decisions. The amount of times I’ve heard “these are the biggest companies in the world run by the smartest people in the world” makes my head spin.  There is a reason that to this day it’s tough to get a straight answer about basically any economic part of the AI bubble, down to “how much does it cost to run a GPU an hour?” or “is inference profitable?” or “how do LLMs ever become profitable?” or “is it profitable for a company to run a GPU or offer AI compute?”  Why? Because these companies used rationalizations of “losing lots of money is necessary to create innovation” and “tech is bad at first!” to make the media actively ignore any technological or economic problems, if not actively defend the technology by repeating these rationalizations like a cultist.  Even those who are most loathsome in the defense of LLMs are a kind of victim of the AI industry, though a rather unsympathetic one. To become a full-blown “AI fan” requires you to accept effectively every narrative that you’re given, herald every single announcement as proof that the prophecy will be fulfilled, ignore the financial realities and actively attack those who would dare to critique the great god of the Large Language Model. You have to know all the new terms, be excited about the right things at the right time, and live in near-constant fear that you’ll fall behind on whatever it is you’re meant to do next.  Your reward is that you can hang around a dwindling number of wealthy yet terrifyingly boring Silicon Valley intellectuals or kiss up to editors that would throw you in front of a bus if it meant getting access to a CEO, and maybe the odd Twitter psychopath who will defend you using a slur. In the end, many boosters will simply act as if they were never wrong. I hope they choose the more-courageous path of introspection, learning how they were had and using it as a weapon against con artists in the future.  As strange as it sounds, I believe the most devout defenders of AI could become great critics in the future. Maybe I’m just being optimistic.  Here’s a very simple question: how much longer can everybody afford to keep doing this? Every single thing has become more expensive in the last year. Even though token prices have gone down or stayed flat, the amount of tokens you burn has clearly increased to the point that organizations are apparently spending billions of dollars on AI services with difficult-to-quantify ROI, requiring frantic advocacy to and financial debasement with every turn of the wheel. OpenAI and Anthropic have become more expensive to run, and OpenAI’s non-GAAP operating margin increased from negative 122% to negative 183% in Q2 2026.  NVIDIA’s GPUs just became 15% to 17% more expensive because high bandwidth memory costs doubled , a conga line of different monopolies upping their prices assuming that each link in the chain will keep spending, as each one of them — down to the AI labs themselves — knows that its contribution to spending on AI is an existential rite. This means that any data center with GPUs delivered in 2027 and beyond will now have to cover billions of dollars’ worth of extra costs, on top of increasingly-staunch local authorities requiring power guarantees ( $100 million a year in Wisconsin for Oracle ) and states like Illinois, Arizona and Virginia killing their tax breaks , all as interest rates spike and demand for AI debt weakens .  Every single year, every single part of the AI bubble becomes more expensive — AI labs want to spend more money, AI data centers cost more money, AI services become more expensive, AI debt becomes more expensive, and everybody becomes decidedly less-patient for there to be some sort of outcome. Meanwhile, public relations expert and OpenAI CEO Sam Altman told podcaster David Senra that “we’ve all [referring to the AI industry] been too ambitious on timelines…[and that changing people’s behavior” is much harder than the tech nerds realize.” Sam: stop talking! Every time you open your mouth you say something silly !   Anyway, here’s everything that needs to happen in the next three-and-a-half years: As I’ve said, NVIDIA’s price increase is going to increase the price of every single data center in construction by billions of dollars, and we’re already approaching the limits of how much money can be raised for them. That “$500 billion” announcement was actually Jensen Huang jumping the gun, per Bloomberg : The largest asset managers and financial institutions were making “slow progress,” and that was before Jensen Huang increased prices by 15%. Do you think it’ll become easier from here? How would that happen, exactly?  God, I’m tired. The entire AI bubble has been exhausting for everybody involved. Because nothing works yet as a real business model or anything approaching truly autonomous (or “magical”) software, there’s the implicit knowledge that you’re going to have to change your product again and again to update to the “best model” or “make things more efficient” (IE: lose less money) or when something breaks because a model’s training got tweaked. The euphemism for this is “exponential improvement,” when it’s really an Arnold Palmer of instability and novelty, and abuses basically anyone connected to the ecosystem every single day. If there’s always something new happening, it’s hard to pin down if things have gotten better, or whether you’re just more proficient in cobbling together different harnesses, prompts and API calls to make it do what you need it to. It is undignified that people tolerate models that become either dumber over time or at random opportunities, while also being deeply exhausting for the end user.  As a paying user of an LLM-powered service, you are guaranteed at some point to face a degradation in service where models misbehave, some sort of shift in rate limits, or some sort of change in product functionality based on their shifting economics.  Has there ever been a bigger shift in a business product’s value than GitHub Copilot’s shift to token-based billing? Microsoft rug pulled two million people that had built workflows on a platform that was allowing them to burn $1,000 to $5,000 in tokens for $20 a month . That’s genuinely crazy! It’s magnitudes more than when Uber jacked up its prices.  It’s equally-insane that Anthropic and OpenAI similarly fuck with their customers , changing the amount of value you get for $20, $100, or $200 a month at random in a way that shouldn’t be legal.    Basically any AI-powered software is subject to arbitrary shifts in availability, capability and pricing at the whims of the vendor. As I covered in my Subprime AI Crisis piece earlier in the year , Replit, Perplexity, and multiple other AI companies have sold their customers a lie by pushing an unprofitable product that they must constantly “tweak” to bring down costs, all while misleading the customer about a “price” that continually declines in value as the price stays the same. This is not a sustainable industry — either economically or emotionally — because it has a fundamentally dishonest relationship with its customers defined by the inconsistency of LLMs both in efficacy, stability (see: Anthropic’s downtime) and training, with each model randomly better or worse at things to the point that it must be a legitimate nightmare to run any software or build any product on top of them.  And the fact they haven’t worked out their business models means that whatever you’re paying today is guaranteed to change. What other product do you regularly buy that has such chaos built into it? What other thing do you pay for where the prices (or availability) can shift to the point that you literally can’t use it in the same way at a moment’s notice? And why does anybody tolerate it when it comes to AI? I’ll add that this is a specific situation where the tech media has categorically failed the customer. We have companies valued at hundreds of billions of dollars that are fucking their customers over day-in-day-out, and the response is mostly to say “ huh that’s strange ” and refuse to let a single critical thought cross their minds.  Every part of the AI bubble must exist in a constant state of flux so that there can always be a future breakthrough that’s always just out of reach. AI does not have to reach an actual achievement — it just has to “show promise” in some way. It is an objective disaster that Microsoft spent more than $260 billion on capex to create a business with less than $11 billion in annual revenue outside of OpenAI, but people will see “$34.33 billion in annual AI revenue” and say “that’s promising growth, up 123% year-over-year!”  They’ll hear about LLMs that delete people’s databases and say “well the models have gotten exponentially better,” even if that better part never seems to eliminate these issues, make a profitable AI company, or create a true killer app that you can point at beyond saying “ChatGPT has one billion weekly active users,” despite around 95% of them not paying a penny (and costing OpenAI likely billions of dollars) and eMarketer estimating that the entire global AI chatbot advertising industry will make $5.41 billion revenue in 2030 , giving OpenAI little hope of stemming the burn. These big numbers — like Anthropic having a $65 billion annualized run rate, an undefined term that obfuscates the fact that Anthropic has made $16.5 billion in the first half of 2026, losing billions of dollars in the process — are fundamentally meaningless, because they’re easily gamed at best, and inherently uncertain at worst.  The AI industry demands you constantly live in the future tense. Everything is about tomorrow’s billions or trillions, the potential of what you’re seeing rather than the thing itself, future gigawatts in data centers that you must treat as if they are already built and value based on things that AI might theoretically do. I challenge you to read everything about AI from this point forward with this in your mind so you can see how intently this industry tries to drag your focus away from what it’s doing toward what it might theoretically do if it only had more money, power and resources, and ask yourself why they need to do so.  To be clear, they’re doing so because you can’t really justify anything about this industry based on what it does today. It costs too much, none of the businesses built on top of it are profitable, it costs so much to build a data center that the most cash-rich asset-light businesses in the world are now burdened with endless expensive-to-install and run hardware for a business that makes a fraction of its overall costs in revenue and has little demand outside of two companies that everybody must conspire to keep alive both financially and philosophically.  And ultimately, nobody can actually explain why we need more data centers.  Would anything really change? What would change? How? How many more do we need? Why do we need so many? Having more power plants meant more people could have power, and having more fiber laid meant connecting more buildings to the internet. What does one more or two more or ten more data centers actually give you? Is there some part of the world unable to access or take advantage of the LLMs available on seemingly every surface of the internet? Because it seems like the only reason these things are getting built is to capture illusory demand based on a “supply constraint” created by two unprofitable companies absorbing all the infrastructure. I don’t hear any compelling scientific or technological reason building more is useful or productive outside of funneling more cash to semiconductor companies.  Seriously, go and read basically any article about AI and see how quickly they start talking about the future, be it in the mainstream media or on a startup’s blog. Every single piece must sell AI on its theoretical promise and, if at all critical, reassure you that the author of course doesn’t dispute the “transformative potential of AI” or “how it’s already transforming the economy,” even if it can’t define how it’s doing so or even what that means.  I let myself have a little fun with today’s piece because I feel like I’ve been so deep in the financial trenches that I forgot how much of the AI industry runs on propaganda, social pressure and outright bullying to manufacture consent for a product that demands everything and provides very little in return. Nothing about LLMs is worth a trillion dollars, or even $100 billion. This is, as I’ve said before , a $30 billion TAM industry dressed up as a trillion dollar one, and the only reason it’s grown this large is because the two leading companies have had their infrastructure built for them and given unlimited resources to subsidize their customers’ compute.  And what’s really stood out is how so little about the AI bubble is actually about AI. No other technology in history has had professional and social consequences for failing to use or like it enough, nor can I find any example in history where journalists have actively attacked critics for not being sufficiently-approving of a kind of cloud software. It is fundamentally crazy to me that, in pursuit of “objectivity,” much of the tech and business media has chosen to accept whatever narrative the AI industry gave them, assuming that whatever we have today is already guaranteed to be something better in the future, both in its outcomes and profitability. This era is unlike any other before it, but took advantage of the fact that most people are desperate to apply the past to the present to rationalize or process what may seem irrational or destructive. To see AI as “just like the dot com bubble” allows you to ignore both the costs and the potential outcomes because “things worked out after that,” even if there’re basically no uses for GPUs after this and the only way we “build new LLMs” is by feeding them expensive training data using billions of dollars of compute that are only available while everybody still believes this is real. The AI industry — and the AI bubble — is fundamentally built on acting in bad faith. Its executives lie. Its boosters lie. Its software lies because it doesn’t actually know anything and generates answers probabilistically, and if you mention that online, someone will harass you for doing so.  It refuses to answer straightforward questions. It refuses to present a plan for the future. It refuses to explain how it becomes profitable, because nobody knows how or has a tangible plan to do so. It deliberately subsidized its subscription products because it knew its customers wouldn’t pay the actual cost of AI, and tortures customers with shifts in functionality and rate limits all while framing this as a way to “ continue to serve customers the most cost-efficient models .”  It attempts to conflate massive, power and resource-hungry AI data centers with the smaller ones that bring helpful yet increasingly-decaying software to our homes. It sells these data centers as “bringing jobs to communities,” all while importing the talent from out of state to build the things then leaving a crew of 100 to 200 people to actually run them after millions or billions of dollars of tax breaks. It sells its “innovations” as creating a “ white collar bloodbath ” to scare you into using inconsistent and unreliable software that’s mathematically certain to make mistakes , and when you say something about it, its acolytes will lie and say that “hallucinations are solved.” It also can only ever sell itself based on what might happen and the theoretical promise of you giving it your complete attention, connecting every bit of data you own, paying whatever it costs, and accepting that it can and will change in price and functionality at random, all while never putting a precise timeline on whatever AGI means that particular week. Whenever you ask for clarity, the AI industry gives you chaff. Whenever you ask when things get better, you’re told it’s both the early days and that AI is the worst it’ll ever be. Even the term “artificial intelligence” is a bad faith attempt to conflate transformer models with things like robotics or autonomous cars, all so that its proponents can claim other people’s successes as their own despite LLMs having little or no relevance to anything else other than generative AI. It encourages dogpiling and ostracizing those who don’t fall behind it, because it cannot succeed on its own merits. It encourages a vile cultism powered too by bad faith and parasocial relationships with both AI CEOs and the models themselves. It exploits the intellectual weaknesses of “smart people” that are actually just good at remembering the right things to say at the right time and have memorized the various justifications for past failures, all while allowing them to use LLMs to promote their own bad faith enterprises where they use work-adjacent product to con others into paying them. And it’s losing because, at its core, AI was never built on very much. It grew this large because the media manufactured consent at the behest of the powerful because lots of money got invested, and the rich and powerful can never be wrong. The underlying technology may be more useful than it was , but it’s not useful enough to be profitable nor reliable enough to be world-changing , and the bad faith representation of LLMs as “good enough” should be a permanent scarlet letter on anyone who misled the public into believing this was anything other than normal software. I was asked recently why I find this all so repugnant, and my answer is simple: I don’t like bullies, I don’t like con artists, and I don’t like being lied to. This industry grew by misleading people about the actual and potential outcomes from Large Language Models, and through an economy-wide attempt to pressure everybody into adopting tools in pursuit of growth at all costs.   Ultimately, it was sold with the greatest lie of all: “this time it’s different!” To be clear, they’re right.  It’s so much weirder, and in the end will be so much worse.  If you liked this piece, you should subscribe to my premium newsletter. It’s $70 a year , $17 a quarter , or $7 a month , and in return you get a weekly newsletter that’s usually anywhere from 10,000 to 18,000 words and provides vast, detailed analyses of the biggest events and companies in the AI bubble. If you want to get in touch — and especially if you have any juicy information about Anthropic, OpenAI, or any other companies in the AI bubble — hit me up on Signal at ezitron.76. I’m also on IB on The Terminal. Anthropic and OpenAI don’t have the money to pay for the capacity. Hyperscalers and neoclouds fail to build the capacity for Anthropic and OpenAI to expand into. Anthropic and OpenAI lack the actual compute demand to justify spending what I estimate will be $200 billion in 2027. OpenAI and Anthropic must keep spending as a means of justifying their existence to hyperscalers using their revenues to artificially inflate growth, to the tune of more than $440 billion across Google, Microsoft and Amazon alone . Hyperscalers must continue to buy NVIDIA GPUs, as the moment they stop doing so, the markets will begin to ask whether AI is an actual growth market anymore and ask for real, tangible answers about where all of this capex is going.  To be specific, analysts expect NVIDIA to make $1.48 trillion in revenue across Fiscal Years 2027, 2028 and 2029 . NVIDIA must sign long-term agreements to buy high-bandwidth memory at scale from SK Hynix, Micron and Samsung — who make 90% of all DRAM — or know its costs would spiral out of control, by which I mean its margins would compress at random at a time when it’s already having to spike demand in extremely odd ways.  Read my Hater’s Guide To The Memory Crisis for more.

0 views
iDiallo 5 days ago

Foot Guns for Sale

I don't think it's going to work out the way everybody thinks it will. The current narrative, at least the one pushed by the companies selling the shovels, is that AI will become centralized. Anthropic and OpenAI will offer safe, vetted AI. Developers will become mere prompt-engineers, submitting requests to these benevolent gatekeepers. They have tamed the dragon. We will benefit only if we become tenants to their API-driven fiefdoms, paying by the token for the privilege of renting intelligence. “They didn’t care that they’d seen it work in practice because they already knew it couldn’t work in theory” - Clay Shirky I complain about AI frequently on this blog, but not because I don’t think it is useful. I use it quite frequently on my day to day. But what I hate is hype and fake narratives. In fact, I believe that the opposite of their narrative will come true. Technology will continue to improve whether Moore’s Law becomes a relic of the past or not. It’s been called dead, yet CPUs and GPUs are becoming faster than ever. I do not believe for one second that developers with pre-LLM experience will end up on the losing side. And if technology continues to improve, then we won’t need OpenAI or Antropic in the future. We will be able to load large open models right into our powerful personal computers with more than acceptable inference speeds. Unless you believe that computers have reached their zeniths and it is all stagnation going forward. The gap between frontier models and open-source alternatives is already all about specialty, and it will continue to narrow. And when developers ubiquitously have access to local models, they will have access to everything. Right now, companies are hoping that developers will use their AI and remain within their ecosystems. They're building guardrails, imposing limits, and designing their models to serve corporate interests first. They see developers as customers, not as threats to their business model. But I’ve seen how easy it is to switch from one frontier model to the next. In fact, some developers in my team accidentally switched by selecting the “auto” mode on their IDE. They didn’t realise that every subsequent request was from a different model. It’s the developer who will end up benefiting from this far more than the corporations will. One developer recently spent his evenings using an AI agent to reverse engineer every peripheral within arm's reach. From those devices, I’ve come away with a full plaintext command shell inside my microphone, a webcam whose activity LED I can switch off while it records, and a key light that hands out memory writes to anyone on the WiFi. He documented the entire process, implemented his own firmware update utilities, and completely enumerated the functionality of devices that were never designed to be user-serviceable. AI gave him the ability to fully control hardware that he has paid for in his own terms. Another developer created OpenLogi , a local-first alternative to Logitech’s software used to remap your own mouse button. The manufacturer was forcing users to create an online account to have access to the hardware they had paid for. OpenLogi gives full control back to the user. This is what happens when developers have the tools and the motivation to bypass corporate control. And AI is about to make this kind of reverse engineering and alternative-building dramatically more accessible. At the speed of large language models, an activist could create a brand new rotating messaging platform every week to avoid the prying eyes of an oppressive government. Someone else can review his own model and add some self improving features. In fact, you could explore new AI paradigms. Most developers I know have a side project they don't have time to work on. With AI, they will have the ability to execute on those ideas. These days, I'm able to run Deepseek on my own $400 local machine at a much slower speed, but I'm not in a hurry. Give it a couple years, I could run something even more powerful. I don't need a project management tool where I have to pay monthly. My actual needs are much simpler. For example, I actually like Jira, despite how much I complain about it on this blog. But I don't need to have Jira for myself on my personal projects. I can build a tool that works solely for my needs. I can easily build applications in environments I am not too familiar with but that are more appropriate for the task. I can build prototypes in a couple hours now and throw them away if they don't match my initial expectation. I can do so much more. Dario Amodei and others are trying to scare us with the capability of AI. They sell the fear of superintelligent systems that will render human developers obsolete or dependent. But this is not what's going to happen. What's going to happen is we will not need Anthropic anymore. Yes, they will have the high-end hardware. But we don't need high-end hardware the same way most people did not need a professional camera. All they needed was a crappy camera with a good filter to post on Instagram. Flickr was superior to Instagram, but the superior technology lost to the one that was "good enough" and in everyone's hands. The funny thing in all this is that by making everything AI-dependent, by building their moats and their guardrails and their API toll booths, companies like OpenAI and Anthropic are selling foot guns. They are building the dark fibers of our era, the infrastructure that developers will eventually subvert, repurpose or simply bypass. Because eventually, we won’t ask for permission. We can just do whatever we want with tools freely available to anyone. We own our devices. We own our data. And soon, we'll own the intelligence on our own terms, without a subscription fee and without a corporate overlord. At the very least, we will get free GPUs .

0 views
Martin Fowler 6 days ago

Fragments: August 24

I was listening to Ezra Klein’s interview with Helen Toner about the recent OpenAI hack of Hugging Face and the subsequent discovery that there were swarms of agents inside OpenAI doing unsanctioned activities. One of the points Klein made was that at no point did any of these (thousands of?) agents ever try to check in with a human [Klein:] So these message boards — you have however many A.I. agents posting hundreds of thousands of messages. At no point do they say: Hey, researchers, programmers, parents at OpenAI, Anthropic — do you want us coordinating with each other on this message board we have created in the innards of your systems? [Toner:] Or even F.Y.I., we have a message board we’re coordinating on in the innards of your system. Listening to that, another thing occurred to me - none of these agents thought to rat the others out . No “hey, some of the agents in here are doing sketchy things”, no sign of an AI whistleblower. ❄                ❄                ❄                ❄                ❄ Is the AI bubble so big that the frontier companies like OpenAI and Anthropic have no way of becoming a viable business? If that’s the case, Bruce Schneier and Nathan Sanders have a possible path: Evidence suggests the market itself could reassess that these companies offer nothing of financial value. In that case, perhaps we can return them both to their original purposes. If these AI companies should fail in the financial markets, the US should nationalize them and convert them into national labs operated under democratic control that preserve their benefit to the public interest. Such an idea may strike many people, used to the laissez-faire free enterprise world of Silicon Valley, as sacrilege, disaster, even socialism. But the United States made world-beating technological progress through such institutions in the recent past. AT&T was a quasi-government entity that led the world in telecommunications and electronics after the second world war. The US has a long, successful history of these kinds of institutions, which have produced world-shaping innovations in spaceflight, telecommunications, nuclear power and more. Congress currently manages a $200bn R&D portfolio, within which frontier AI development is, arguably, a glaring gap. ❄                ❄                ❄                ❄                ❄ Here’s a message for those readers who live in Massachusetts, just to the north of me, specifically in congressional district MA-06. I don’t usually endorse political candidates, but I’ve made an exception for Beth Anders-Beck , who is running for that house district. I’ve known Beth for many years and have a high opinion of her smarts, wisdom, and compassion. They would make an excellent member of congress. ❄                ❄                ❄                ❄                ❄ Kevlin Henney posts “one weird trick” for deciding when to skip reading LinkedIn posts , essentially by identifying a common pattern for skippable posts: It seems like a good approach. I, however, have a simpler one - skip all LinkedIn posts. ❄                ❄                ❄                ❄                ❄ Bartosz Ocytko has detailed and thoughtful post about the usage of agentic programming at Zalando . Like most companies I hear from, they are convinced of the value of agentic programming but still exploring how best to do it. One notable step they’ve taken is building platforms to act as a clear portal for API access and tools to support chat UI and CLI. This allows them better support good security practices and to monitor usage of models. They have seen signs of agentic programming increasing the complexity of codebases, including leading to larger commit messages. The write-up spends a lot of time on knowledge sharing, how to pass on skills, and the support of experiments. With >200 teams innovating and broadly exploring the ecosystem, the question arises whether and when to converge. We believe it’s way too early for this. While agentic engineering practices are still in their early stages, our key objective is transparency and exchange across teams. I was struck by their use of an LLM to assess the risk of pull-requests. Those with a low risk of rollout can be auto-approved, reducing lead time by 20-40%. An interesting consequence of this is that it encouraged folks to split pull-requests so low risk portions can take advantage of the fast approval. Any changes to configurations are automatically made high-risk, which they feel protects them from common outage traps. They repeat the common thread that the value of AI depends greatly on underlying skills. Like anyone in the industry we observe how AI amplifies the good and bad practices across our organization. Teams that get carried away with agentic engineering end up with large PRs that discourage reviewers and slow down delivery until a team adjusts their practices. ❄                ❄                ❄                ❄                ❄ Julia Curlee was a senior intelligence official in the White House. She had served under administrations of both parties, been the briefer for Vice President Pence, and on the National Security Council under Biden. She writes an absorbing account of her relationship with Pence and shares observations about the changes to the intelligence community under the current administration, including recent events at the CIA (gift link) The agency has been gutted as part of a deliberate plan, the director of the Office of Management and Budget once boasted, to put the people who defend our country “in trauma.” Analysts have been fired in public or questioned by the FBI; decade-old assessments have been denounced by the CIA director in the press. The president calls analysis “virtual treason” when it contradicts his preferred reality, and uses the CIA to undermine public confidence in American elections. Fear has done its work. Irreplaceable officers with crucial language and technical skills, and decades of experience, have walked out the door. Those who remain within an agency built to deliver hard truths are being muzzled. For a worthwhile sample of her analysis, read this evaluation of the current bargaining between the US and Iran Most wars do not end in “unconditional surrender.” They end when both sides accept terms. Paul Pillar’s classic study of war termination, “Negotiating Peace,” treats combat and diplomacy as a single process: Each side fights to improve the terms it can demand at the table, and talks to lock in what the fighting has won. She continued to serve the second Trump administration even though they knew she was trans, until her position was made public. Autocrats seem appealing, with the promise to get things done without the ponderous constraints of rule of law or bureaucratic procedure. There are occasional “Good Emperors” who raise people based on merit, but more often such power attracts corruption, nepotism, and toadies. Flailing regimes dehumanize minorities to distract from their failures. When the economy collapses or a war goes badly, they find a tiny group of people, make them the enemy within, and rally the country against them. This is how it’s gone in Iran. Hungary. Russia. I wrote PDBs about it. This will not stop with trans people. It never has. Post is too long Contains a (crummy) info-graphic No voice of poster (instead “aspiring anodyne anonymity”

0 views
Stratechery 6 days ago

Autonomy and Innovation

Listen to this post : While not every Western followed the cliché, by the 1930s cowboy serials had landed on a consistent visual cue: the hero of the show wore a white hat, and the villain wore a black one. At the end of the day, however, they both were cowboys with cowboy hats. Westerns aren’t much of a cultural touchpoint anymore, but the “white hat” and “black hat” nomenclature is very relevant in tech: hackers who are focused on patching vulnerabilities and protecting software are “white hat hackers”, while hackers who are focused on exploiting vulnerabilities for malicious reasons are “black hat hackers”. Of course this can very quickly become complicated: governments might employ hackers to break into enemy software installations — are they white hats or black hats? Or consider bug bounty programs, wherein large software companies pay bug bounties to hackers who find and report vulnerabilities; it’s basically using money to incentivize would-be black hat hackers to be white hat hackers. The actual takeaway is that all of this complexity is overwrought: just as a cowboy is a cowboy, a hacker is a hacker; the hat is not a statement of capability, but rather intentions, and those intentions are shaped by incentives. The best way to attack infrastructure is to find a vulnerability and exploit it; the best way to defend infrastructure is to find a vulnerability and patch it. It’s all the same skillset. This delineation between capability and intent and incentive is critical when it comes to AI. At the end of last month’s Article Who’s Afraid of Chinese Models , I discussed a mysterious attack that model host Hugging Face had just endured, which they were only able to fight off with the help of open weight Chinese models, and wrote: It’s difficult to overstate how wrong-headed the Trump administration’s panicked response to Anthropic’s release of Fable was, particularly since it exacerbated Anthropic’s worst tendencies in terms of assuming only they can be trusted with powerful AI. In a world with only one AI, it might make sense to reserve the most powerful cybersecurity capabilities for the U.S. government and trusted allies; however, that’s not the world we live in. There are and will be models eminently capable of mounting cybersecurity attacks on existing infrastructure, and those models will be — already are — widely available. The best defense — the only viable defense, in fact — will be to make sure defenders have access to the best models as well. Right now defenders are effectively banned from using Fable or Sol for cybersecurity because of Trump administration directives; that means the best alternative is using models from a country which has been trying to weaken our cyber defenses for years. This is insane! The point is the one I made in the introduction: when it comes to cybersecurity, the capability that is necessary for good defense is the exact same capability that is necessary for good offense; the color of the hat is a matter of who is actually prompting the AI. And, sometimes, not even that is clear: it turns out that the entity that hacked Hugging Face was actually OpenAI, as a series of unconstrained agents being evaluated for their cybersecurity capabilities found and exploited a bug in the package manager in their sandbox; that package manager had Internet access and a sufficiently writeable file system such that the agents could communicate with each other over time. The entire chain of vulnerability discovery and exploit creation culminated in the so-called “Hugging Face incident”. There is an entire Article to be written about the implications of this specific incident and what it says about AI risk; some of my takeaways are still up in the air pending OpenAI’s promised release of an in-depth technical report (my preliminary takeaway is that the agents were not “cheating” but rather doing what they were told to do; of course that’s arguably even scarier ). The part I want to focus on today, however, came at the end of a presentation OpenAI’s Eric Wallace and Michael Dalton made at the Black Hat USA conference about the Hugging Face incident. This was Dalton summarizing Lessons Learned: We have seen what will be a dramatic acceleration of offensive capability for attackers. We have an existence proof that was unintentional, but it exists before us, and we have as a consequence seen a glimpse into the near future of what attacks will look like for our industry. The challenge is that we need a similar acceleration of defense. Today we see fully automated offence as possible, but we have no such existence proof for full automation of core defensive loops and cycles in behavior. We believe it’s vital at this moment to begin accelerating defense and finding ways to automate SDLC, in the modern parlance, so incident response, vulnerability detection, vulnerability patching. There’s some things that stand out acutely as challenges for the industry to begin tackling with high urgency. So continuous agentic red teaming is one of them. As you can see from this incident, agents are quite good at finding zero-day attacks in the infrastructure of companies. The question that’s now going to be posed is whether companies are able to invest sufficient model intelligence and effort in finding and remediating their vulnerabilities before someone else that’s a threat actor does it for you. This style of operating will be different now, but ultimately we need to invest in having AI agent red teaming that enables defenders to find and remediate vulnerabilities before attackers do. But automating these defensive loops is not trivial, and so if we do this partially, we will fail to meet the scalability of the offensive acceleration that we have just seen. So for example, if we automate vulnerability finding without automating patching, we will shift the bottleneck from vulnerabilities to patching to remediation, and we will simply drown or inundate human software engineers in new vulnerabilities to fix and patch. This is not a problem whose end state we can solve partially. We will need to take these core defensive loops and fully automate them, which will require conversations with infrastructure and product partners and reaching to a point where we can say, if a vulnerability is identified, not only can an agent identify that vulnerability, we can have an agent propose a patch, we can have automated infrastructure to roll out a change with that patch, and roll it back if there is an availability incident or outage. That loop needs to be fully automated in its end state. Of course, we want to automate as progressively and iteratively quickly as we can, but if we don’t reach that end state, then we will be comparing a core defensive loop of fixing vulnerabilities that is a human in the loop and is much slower and less scalable, with an offensive loop that is fully automated, and that is an unsustainable position for this industry to be in. This situation is obviously completely novel; Dalton is arguing that it will become commonplace. Some of the issues he is raising, however, are not novel at all. Go back to the concept of a bug bounty program. Software is incredibly complicated and brittle and built on a foundation of code that, if you dig deep enough, often goes back decades; there is so much code and so many dependencies that no company, no matter how security conscious they are, could ever ensure it is perfect. This reality is what creates the opportunity for black hat hackers: a bad actor can probe software, find bugs, and exploit them; the most effective defensive preparation is to do the exact same thing. That could entail regular penetration testing (pen testing) by a “red-team”, or simply paying the would-be bad actors to be on your side. It’s worth noting, however, that this approach to defense only arose after offensive black hat hackers had been breaking into systems for years. The problem wasn’t that they were uniquely capable, but rather that they were uniquely incentivized: breaking into systems was good business; companies hosting those systems, on the other hand, were insufficiently incentivized to invest in defense. Spending money on security is well-spent if nothing happens, and unfortunately that is a difficult budget line item to argue for when it only moves the needle on costs, not revenue. This is where Dalton’s concerns echo past industry indifference. What the Hugging Face incident showed is that agents, with their ability to scale attacks with compute and autonomously develop exploits for vulnerabilities they find, are a threat today, but that companies are not investing in the capabilities necessary to defend themselves. There is good news, however: in this new agent-defined security landscape, defense should be at an advantage in a way it wasn’t in the hacker era. It used to be that the best defenders could do is mimic the tactics of the offense, and/or pay them off, because preemptively finding all of the bugs was not viable. However, that is changing: it actually is — or soon will be — possible to meticulously go over an entire code base, including all of its dependencies, and look for bugs and patch them. Notice the structural advantage available to defenders: they actually have the code in question; offensive agents need to probe and discover vulnerabilities without the same advantage. What was illuminating about Dalton’s overview, however, was the implication embedded in his explanation of why this isn’t currently enough. Specifically, the expected value for a hacker’s automated attack is always positive. If the offensive agent finds a vulnerability and creates an exploit, and that exploit fails or is itself buggy, then nothing has changed about the status quo: the exploit doesn’t work (or, perversely, makes the original vulnerability larger by virtue of its own bugs); if the agent executes the exploit perfectly, meanwhile, the attacker has gained access to the system. The attack only needs to work once for the entire endeavor to have a positive payoff. The challenge for the defender, on the other hand, is that they need to keep the software in question working correctly, and not make the situation worse. This means that any automation has a negative expected value: successful automated vulnerability discovery and patching preserves the status quo, i.e. the software is not hacked. However, any unsuccessful patches make the situation worse, either by breaking the software or by introducing new vulnerabilities. The agent only needs to fail once for the entire endeavor to have a negative payoff. This is the dynamic that leads to the exact situation Dalton describes, where offensive actors are fully automated while defensive systems, even if they use AI, will be incentivized to keep a human in the loop, and no human in the loop will be able to keep up with fully automated agents. Truly effective defense will mean truly trusting agents to act autonomously, but most companies won’t do that until they are forced to by regular and unremitting hacks by fully autonomous attackers. Over the weekend David Senra released a new podcast episode with OpenAI CEO Sam Altman , where Altman admitted he had been wrong about the speed of AI diffusion into the broader economy: I love startups. I think startups are the coolest thing in the economy and I’ve spent my career trying to like really understand startups, and I thought when we got to GPT-4, which was back in 2023, I think, that very quickly after that, there was going to be much more disruption in software businesses being up for grabs right away than turned out to be. I was wrong about a few things, but one of them in terms of the speed, one of them is the economy just has so much inertia. People keep doing the same things they’re doing. They keep buying from the same, you know, company. They keep sort of wanting to use their tools in the same way. I think it’s actually a positive in many ways and it’s going to make this big transition in front of us go smoother and slower. I’m grateful for it. But I think it means we’ve all been too ambitious on timelines even with this incredible technology. I think AI is one of the most incredible technologies humanity has ever invented. Society and the economy will adapt more slowly. I of course think Altman is right about AI taking longer to sweep the economy generally and software specifically; I’ve been pushing back on these timelines all along . I also think his answer is incomplete in its explanation as to why, and Dalton’s warning about the mismatch between offensive agents and defensive preparations explains it. First, GPT-4 was an incredible breakthrough; it’s also a model that wasn’t remotely capable enough to actually displace real world software. It didn’t even have the ability to reason, which is the key breakthrough that has unlocked the capabilities that Dalton described. Second, what is meant by a model not being capable enough is that it makes mistakes and can’t be trusted. People can and were rightly awed by what AI can do, but the decision about actually implementing AI isn’t made according to what can be done, but about whether or not critical mistakes can be avoided. In other words, incumbent companies are inevitably going to approach AI with a bias towards a negative expected value framing: AI ideally will make their existing operations more productive; what they are most concerned about is AI making a mistake that blows up in their faces. What that means is humans will continue to be in the loop, which will always be a bottleneck. This will, in the long run, be a mistake, just like it will be a mistake for companies to keep a human in the loop when it comes to agentic defense. As Dalton noted, the only way to defend yourself against fully automated attacks is to fully automate your defense, but it will take a while for defenders to accept the trade-offs that entails. And, by the same token, the companies that win in their category will be truly driven by AI, instead of simply AI as productivity enhancer. The great irony in Altman’s answer is that he actually identified how these companies will arise: they won’t be incumbent companies overhauling how they work; rather, the true AI-native companies will be startups. Back in 2023, shortly after ChatGPT came out, I put forward the question in AI and the Big Five as to whether AI would be a sustaining or disruptive innovation: The story of 2022 was the emergence of AI, first with image generation models, including DALL-E, MidJourney, and the open source Stable Diffusion, and then ChatGPT, the first text-generation model to break through in a major way. It seems clear to me that this is a new epoch in technology. To determine how that epoch might develop, though, it is useful to look back 26 years to one of the most famous strategy books of all time: Clayton Christensen’s The Innovator’s Dilemma , particularly this passage on the different kinds of innovations: Most new technologies foster improved product performance. I call these sustaining technologies. Some sustaining technologies can be discontinuous or radical in character, while others are of an incremental nature. What all sustaining technologies have in common is that they improve the performance of established products, along the dimensions of performance that mainstream customers in major markets have historically valued. Most technological advances in a given industry are sustaining in character… Disruptive technologies bring to a market a very different value proposition than had been available previously. Generally, disruptive technologies underperform established products in mainstream markets. But they have other features that a few fringe (and generally new) customers value. Products based on disruptive technologies are typically cheaper, simpler, smaller, and, frequently, more convenient to use. It seems easy to look backwards and determine if an innovation was sustaining or disruptive by looking at how incumbent companies fared after that innovation came to market: if the innovation was sustaining, then incumbent companies became stronger; if it was disruptive then presumably startups captured most of the value. I think it speaks to the incredible capability of AI that it is setting up to be both. There are massive productivity benefits from AI right now; for most knowledge workers leveraging those benefits is a matter of agency, but for software developers in particular it is increasingly a matter of necessity. That distinction between agency and necessity, however, is an important one: if leveraging a technology depends on humans figuring it out, then penetration will be limited by human creativity and risk taking. Those limits will be very strong in any sort of established company, because the risk calculus will be biased towards avoiding the downsides. Those calculations will make AI sustaining, but nothing more. Human creativity and risk taking in the form of a startup, however, operates with a completely different risk profile. For startups the base case is failure; that means that anything that makes success more likely has positive expected value, which is to say that truly leaning into AI will be nothing but upside. Or, to put it another way, it is startups who will be the offensive hackers with nothing to lose by automating everything; it is the incumbents they will be attacking who will be so worried about losing what they have that they will keep humans in the wrong loop for too long. Same tools, different incentives, and, in the very long run, very different outcomes.

0 views
Brain Baking 6 days ago

The Red Mailbox

The Red Mailbox next to our village’s primary school is my primary drop-off point for writing letters . Over the decades, its bright deep red brilliance has been gradually replaced by a patina of sunburned broken red. And yet, The Red Mailbox persists. It still exists. It existed over thirty years ago, when I went to that very school next to it. Other Red Mailboxes aren’t that lucky: in 2019, BPost—the Belgian posting company that was privatised in 2000—removed over one fourth of the Red Mailboxes all over Belgium in an attempt to “save the company”. The biggest reason might not surprise you: most of these boxes didn’t receive much letters: Volgens BPost is het aantal brieven dat mensen in de rode brievenbussen deponeren, met 60 procent gedaald sinds 2004. Uit een kwart van de bussen haalt de postbode nog hoogstens zes brieven per dag op, luidt het. [According to BPost, the amount of letters that people deposit in the red mailboxes has diminished by 60 percent since 2004] The Red Mailbox. Emptied at 10:00 AM. That year also happened to be the year of BPost’s stock market crash. Still, I think ultimately their decision was the right one: of all the letters I send out, I only receive about a fourth replies in that same analogue form. Especially in 2026, people don’t use The Red Mailbox anymore, turning the battered broken red metal box attached to a brick wall into a weird artefact of the past. I wonder how long it’ll take before the same emotion is triggered as wandering around in The Legend of Zelda: Breath of The Wild ’s broken world, where artefact bits and pieces poking out of a ruin reminiscent of a once thriving community now only cause weariness. Before the Mailbox Removal Program, BPost also permanently closed multiple post offices. Our village’s old post office building now is a Turkish food joint. Yet another thing of the past, just like physical banks, toy stores , and if we’re not careful, bakeries and butcher shops. Yet we are nimble: a bike ride in two directions, both about four kilometres, will take you to another office. Do verify opening hours before leaving. Even Google Street View archives dating back to 2009 do not have a photo of the old post office building in its original state. The archives do contain traces of deceased Red Mailboxes, such as the one below weirdly enough mounted against the front facade of someone’s home? I would like to believe that the owners of the house kept a notebook besides one of the windows to track all those weird people stopping by to drop their weird letters. An archived copy from Google Stret View of the one that disappeared, mounted next to a red drainpipe. The dismounting job revealed an ugly square stain behind it. Passengers will no doubt wonder what used to be on that wall, thinking the owners of that house must have done something weird. They didn’t: the government did. I wonder what personal history The Red Mailbox can tell us. How many love letters did it ingest all these years? Or birth announcement cards? Marriages? The bearer of happy news. Let us intentionally leave out the boring and perhaps more depressing tax related correspondence. We made good use of that very same mailbox when we got married, and when we had our daughter and son—I can distinctly remember it barely containing the envelopes when I dumped all those birth cards in there at once. The expected Thunk! sound of the envelope hitting the bottom was replaced by the shuffling of papers and me trying to jam them all in. I still make good use of that trusty old mailbox whenever I feel like using a pen to send a message to a pen pal, which admittedly happens less and less. Perhaps I too am a part of the problem. The Red Mailbox was not always there. In fact, it’s one of the more modern alternatives—relatively speaking—compared to the late 19th century cast iron standalone models that are now officially classified as being part of our Flemish/national heritage . This particular model in the link still stands proudly in Antwerp, yet these are the exception to the rule: I haven’t encountered a cast iron painted one in our vicinity. The little door to retrieve the letters is a lovely touch. According to heritage info of the city Spa , these models were cast by the J.G.Requilled foundries of Liège between 1860 and 1938. The most striking difference between these original models and the ones on the photos above is perhaps the fact that its purpose is no longer explicitly mentioned: the French inscription “LETTRES/IMPRIMES” is gone. By now everyone knows that a Red Mailbox is a letterbox from BPost, not a personal one. I hope. If you ever received a letter from me: think of The Red Mailbox where its humble journey all the way to your letterbox started. Related topics: / letter writing / By Wouter Groeneveld on 24 August 2026.  Reply via email .

0 views
Unsung 1 weeks ago

Testing tip: Make your keyboard fast

I believe every modern operating system allows you to set the key repeat rate and the delay before first repeat: = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/testing-tip-make-your-keyboard-fast/1.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/testing-tip-make-your-keyboard-fast/1.1600w.avif" type="image/avif"> = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/testing-tip-make-your-keyboard-fast/2.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/testing-tip-make-your-keyboard-fast/2.1600w.avif" type="image/avif"> By default, on a Mac, the initial delay is 500ms, and the key repeat rate 1000ms. You can adjust both: On a PC, the shortest values are exactly the same, but Windows 11 only allows up to 405ms between key presses, and 1000ms of initial delay. My suggestion is to make them both as short as possible for testing. Why would that be helpful? There are two reasons. First, it’s good to test whether your keystrokes behave well when repeated. Occasionally you might want your interface to suppress repeating, or do something special if the key is held longer. Second, it’s a good test not just of repeating, but just the user pressing keys really fast. It’s very important for the UI to never make you wait for any animations or transition, and a lightning fast 30fps repeat rate is a good way to stress test your system this way. Here’s me holding the Tab key in Figma at a standard key repeat speed: It’s nice to see those transitions help you orient yourself as you’re thrown around the canvas. But look at what happens when I do the exact same thing with the key repeat cranked up: The transitions are now slowing the UI so much that you can no longer see any canvas movement – the canvas only catches up when I release the Tab key. The smooth movement assumed a certain minimum key rate, and perhaps wasn’t tested with a faster one. There are of course ways about it, like speeding up the interactions, suppressing the transition smartly, or introducing a custom repeat rate if needed – I talked about it a bit in my essay about designing fast keyboard interfaces . Here’s an example of an interface that suspends transitions at a fast keyboard rate and responds in real time: This looks very chaotic especially since you’re not in a driver seat, but this interface is at least honest and doesn’t make the user wait. And it might not be pure madness – you would be surprised how fast people’s brains and fingers can react. But first, you have to be aware of the problem, and I think setting up those key repeat values as short/​fast as possible will help you find more of those kinds of issues. (And you might choose to keep those high speeds anyway in regular use, like I do.) #flow #keyboard #tips Initial delay goes from 250ms, to as long as 2s. Repeat rate goes from 2000ms (once every two seconds) all the way to a whopping 33ms (30 times a second).

0 views
Jim Nielsen 1 weeks ago

Getting an LLM to Make Me a Tool for Enriching the Color Metadata in My Icon Collection

On my icon gallery sites , I have metadata I’ve manually added over the years to tag certain icons as being predominantly “blue” or “orange” or some other color. Then I use this metadata to present icons of (roughly) the same color. It’s kinda neat to be able to browse a wall of icons that are all the same color. The thing is: I know there are a lot of icons I’ve missed tagging over the years. But I have no idea how many, and figuring that out seems like a really arduous task. How do I go through 2,000+ icons and find all the ones that look predominantly “orange” but haven’t been tagged as such yet? Seems like a good task to throw at an LLM. But I don’t want to just say, “Go tag everything that’s missing” and blindly trust the output. I need to be able to make a decisions as to whether I think a particular color is “orange” or not. What I need is a tool for the job. I’m a very visual thinker, so to continue curating these color categorizations, I need some way for the computer to do its thing really, really fast, and then pull me into the loop to visually make decisions. Here’s how I am thinking about about accomplishing this task: That seems like it would be a good tool to put me in control of visual decision making around color categorization. So I tell the LLM to run with it. We chat back and forth. I think, “You probably need to run all the icons through some model to make the correlation?” But it’s like, “Nah bro, just make a ‘hue histogram’.” It tells me how. For a color like “orange”, I can: [process] each PNG, skip transparent pixels, skip low-saturation gray, convert the rest to HSV, and score how much of the remaining mass sits in the orange hue band (roughly 15–45°). Rank icons that don’t already have colorId: . Ok, sure. That sounds reasonable. [This] scores each icon PNG by share of opaque pixels per color bucket, then writes a standalone HTML page: tagged vs maybe-missing, per color. Let’s just make it, and then I’ll decide whether it’s good enough. After a few iterations, the computer going “brr…”, and me saying “explain that like I’m dumb”, I have a really effective little tool! The little threshold slider is a nice touch. It lets me fiddle around with the fidelity of the matches. In some cases, sliding it down reveals more icons I would’ve otherwise missed. In other cases, I’m like “What are you thinking? I don’t see that as ‘yellow’ at all!” Supper effective little tool. I go through each color, select the ones I think are missing, paste the IDs back into the LLM, and then have it update each icon's metadata. Boom, done! That all would’ve taken so long before. I would’ve never done it. Reply via: Email · Mastodon · Bluesky Create a page with a list of colors on the left (red, green, orange, blue, etc.) When a color is clicked, show two columns. 1) All icons I currently have tagged for that color, and 2) all icons that might be that color but aren’t tagged as such (you, computer, process all my icons and do the work to figure this out and make recommendations). Allow me to select one or more icon(s) in the “not yet tagged as this color” column. Once I have all the ones I perceive as missing, give me a button to say “Copy” which copies the IDs of those icons. I’ll paste the IDs back here in the chat and you go add the corresponding metadata. The LLM is good at making throw-away code. This doesn’t need to be “production-grade” code I depend on. Just something that’s good enough for me to get a job done, then toss. The resulting metadata is the goal, not the tool I use to get to the goal. The LLM is good at making one-off HTML pages for a specific task. In my case, all these images were hosted on a CDN, easy enough to just point at and have a standalone file that I can locate on my hard drive and open directly as a URL. No bundling. No transpilation. None of that. I don’t even need a web server! Keeping things very basic on this project is paying off: I had really elemental building blocks that didn’t require additional third-party tooling. Just HTML, CSS, a little in-page JS, and images on a server! It’s fun to say, “Don’t do the work for me. Instead, help me make a custom-fit tool that facilitates me doing the work in the most empowering, correct way possible.”

0 views