Latest Posts (20 found)

Businessing 101

Back when I began freelancing I figured I’d set up a limited company eventually. I guess thirteen years later is eventually. I’ve finally got a local accountant working on company registration. Exciting times! The final push was this Making Tax Digital thing from GOV.UK. It’s probably not complicated if I cared to look but I’m crawling back to FreeAgent anyway. My home-cooked accounting app served one whole tax year! I will have an accountant do everything now. Another big reason to become Ltd is to face challenge and opportunity head on. A loud contingent of the web industry are giving themselves up to the borg. Mandated token servitude and knee-bending to grifters like Google is sending web dev back to the stone age. I still care about making meaningful websites for real people. I have no desire to dump dead websites on the dead internet . I want to champion and preserve the knowledge and expertise being discarded at astonishing rates. This is not an “anti-AI” movement per se; I simply don’t compete with chat-box-driven development. Trading under a limited company allows more opportunity to collaborate with other creative professionals. As a freelancer I’m hired personally. As a company I can offer the same services and assurances whilst expanding the team if specialists are needed. Who knows, maybe one day I’ll be in a position to hire full-time. This does not mean I’m going back to LinkedIn. I deleted that account like ten years ago. Please don’t make me go back! My income is modest. My freelance years have ranged from ~£20–60,000 annually. I work a four-day week with sensible hours and avoid overlapping projects. Working hard and working long hours are not the same thing. I expect my new business will remain shy of the VAT threshold but I’ll see what my accountant advises. I’m not sure my rates have kept up with “inflation” and cost of living. According to the Bank of England inflation calculator : What cost £1.00 in 2013 would cost £1.44 in June 2026. Hmm, feels like I’m paying a lot more… probably the shrinkflation . One things for sure, the quality of my services will not shrink. I’ve got a lovely new brand and website ready. That’s been in the works all year. Once the red tape is completed I can go live. August or September, maybe? Thanks for reading! Follow me on Mastodon and Bluesky . Subscribe to my Blog and Notes or Combined feeds.

0 views

📝 2026-07-24 11:20: I think if I had my time again, I would have become a software developer....

I think if I had my time again, I would have become a software developer. I just so much fun - coming up with ways to solve a problem in an elegant way is very satisfying. Then people emailing saying they enjoy the tools I'm creating. Very rewarding! Mind you, would it still be as fun if it was my job? I don't know. 🤔 Thanks for reading this post via RSS. RSS is ace, and so are you. ❤️ You can reply to this post by email , or leave a comment .

0 views

How to set up Raspberry Pi wifi by just talking

As soon as I received my first Raspberry Pi, I knew that it would be a wonderful platform to bring AI into the physical world. Since the initial hardware didn’t have good CPU support for fast arithmetic, I ended up writing code that ran on the GPU so I could get the speed I needed for early deep learning vision models. That was in 2014, and since then the capabilities of both Pis and AI have skyrocketed, and I’m even more convinced that there’s massive potential in combining them. To show you why, I’d like to demonstrate how open-source AI running locally on a Pi has solved some practical problems I’ve run into, and hopefully inspire you to build your own projects using the new possibilities. Pis are great for systems that need to be out in the world, doing specialized jobs. I’ve seen them work well in all sorts of roles, from badge scanners to wildlife cameras. I even run a class that teaches students all about edge AI using the platform. While the boards are generally easy to use, the most frustrating part for the students and instructors is the setup process. While the latest imager makes it straightforward to configure settings like a wifi network to join or enabling SSH when you’re flashing a card, getting the students to the point where they can connect to their Pi using VS Code from their laptop could often take multiple sessions. The biggest problems were: A lot of these issues were solvable if you plugged the devices into a monitor, mouse, and keyboard, but this has its own problems. It meant we needed to provide that equipment to all students during class, and allow them to take it all home too, so they could update the configuration for their personal networks. It also required an extra power socket per student, for the monitors, which added up in a class where we already had to bring in a cart full or power strips. The monitor connections also weren’t always plug and play, we found we often needed to boot with a screen attached to have the display recognized. This isn’t just an educational problem either. One of the reasons that I believe the Internet of Things failed is the setup tax involved in getting smart devices running. According to manufacturers I’ve worked with, less than 30% of their smart appliances ever get connected to the internet, because the process of downloading an app, setting up an account, connecting over Bluetooth, and then typing in the wifi name and password takes too long, and is too errorprone. Even professional installers sometimes struggle with configuration in enterprise and industrial environments. So, what can AI do to help? One of the biggest developments in AI over the last few years has been the development of highly-accurate open-source Automatic Speech Recognition (ASR) models, also known as Speech to Text (STT). OpenAI were the pioneers in this area, releasing the family of Whisper models in 2022. These offered accuracy that was competitive with the models used internally by large tech companies like Google and Apple. These new models allowed startups to begin building voice applications that had never been possible before, and led to a new generation of dictation and meeting note tools like WhisprFlow. One of my dreams as I dealt with all of the configuration issues was a voice-based system that would allow me to simply plug in a headset and set up everything by talking to a Pi. Whisper made this dream seem more realistic, but as I tried to use the models on local hardware, I realized that they were too slow for any kind of interactive application. To address that my startup trained new models from the ground up, designed specifically for realtime applications on affordable hardware. These Moonshine models are smaller than Whisper (our high-end is 250 million parameters versus OpenAI’s 1.5 billion) while offering better accuracy. We also implemented a streaming approach, where a lot of the work is done while the user is still talking, so we can return results even faster. This allows us to return more accurate results than Whisper v3 Large, in just 800 milliseconds on a Pi 5 , whereas even the less-accurate Whisper Small takes over ten seconds. I was excited because this meant I could finally build a responsive voice agent that runs locally on a Pi, something offline-first, and fast and flexible in how it responds. This kind of system needs more than just an STT model, it needs to decide what the user means and respond by taking actions and talking back with a Text to Speech (TTS) system. The Moonshine Voice framework includes modules for conversation flow and TTS, so I was able to use it to build pi-help-bot , a local voice agent for network configuration on the Pi. The application listens to the microphone for commands like “What is my IP address?” or “Help me set up the wifi please”, figures out what actions to take, and responds appropriately by talking to the user. It’s written as a Python script, and here are some snippets that show how it works: This code is a function that uses the netifaces library to figure out the Pi’s address on the local network, so instead of having to connect a keyboard and display or decode the output of nmap, you can ask the question and hear the result, all in just a few seconds. Unlike older voice interfaces, the phrases the user says don’t have to be exactly the same as the one you register an intent with. Instead the framework matches incoming speech against a small, local LLM, so that variations “Hey, can you tell me what my IP is?” work too. This was important to me because one of my biggest frustrations using voice interfaces like Alexa is that they need particular wording to trigger commands, but these wordings aren’t discoverable, so figuring out how to make something happen can require a lot of patience. The IP address command is the simplest kind of conversational flow, where the user asks a question and the system immediately responds. Not all interactions can be handled as simply as this one though. Here’s another example that shows how to implement something that needs multiple questions, answers, and confirmations, connecting to a new wifi network: Hopefully you can follow the logic as it walks the user through providing the information required, but you might be wondering about those yield statements. Those hand back control to the dialog controller while the script is waiting for user responses, so the rest of the application isn’t blocked. The end result is a local voice agent that will listen out for configuration questions and commands, allowing users to set up a Pi for remote access with just a headset. For ease of use, I’ve begun customizing the images I burn to SD cards so that this script automatically starts on boot. This means I can start setting up new devices immediately after powering them on. I hope this gave you some ideas about how a local voice interface could help with problems you face. For further information check out the Moonshine Voice project on GitHub to see full documentation on the library, and please give us a star while you’re there, it helps us keep working on this project. There were different networks in the lab and in the students’ dorm rooms, so it wasn’t enough to hardcode a single SSID and password on the SD card. You need the local IP address of the Pi to SSH into it from a laptop, but it can change dynamically every session. Using “<Pi name>.local” would sometimes work, but some networks didn’t support this kind of lookup, and even if they did it required coordination between the students to avoid name clashes. It was easy to forget to set the configuration so that wifi and SSH were available, and since the instructors didn’t always know what network and password they’d be using in the class ahead of time, we couldn’t pre-flash a bunch of cards to speed up students on-boarding.

0 views
Kev Quirk Yesterday

Your RSS Reader Is Robbing You

by Antonio Santos Antonio opines that by using RSS feeds instead of visiting people's sites, we're robbing ourselves of enjoying the design many site owners have put a lot of work into. Read post ➡ On the other side of the argument is this post by Christian Cleberg where he argues that using RSS allows us to consume many different sites with typography that we prefer. Maybe the site's font isn't that nice, or it's too bright, too dark, too small. Using RSS normalises the design across all sites and allows readers to have the same experience across all the sites we read. I lean more to Christian's way of thinking, but I do regularly visit people's sites too, just to see the design and how it may have changed over time. It's a simple click of the post title on my RSS reader, Miniflux , so it's simple for me to get the best of both worlds. So I don't think our RSS readers our robbing us - we can have our cake and eat it. Thanks for reading this post via RSS. RSS is ace, and so are you. ❤️ You can reply to this post by email , or leave a comment .

0 views
Unsung Yesterday

Safari and system design, pt. 2

Around the time I was writing about iPhone’s Safari breaking the expected “tap to jump to the top” gesture , a conversation on social media pointed to yet another thing that this tab control does strangely. A typical use of Safari means two groups of sites: a regular set on the right, and “private” pages on the left (this is what Chrome calls “incognito mode”): = 3x)" srcset="https://unsung.aresluna.org/_media/safari-and-system-design-pt-2/1-framed.1600w.avif" type="image/avif"> As expected, you can tap on either label, and switch to the relevant group with ease: It also feels like you could slide it – and you can, except… …you immediately encounter a Scroll Lock problem . You are not dragging the pill – you are dragging what’s underneath the pill. To switch, you have to go the other way : You can immediately intuit some inherent unpleasant complexity of the whole system – not just in it “going the wrong way,” but also in how it creates room and then contracts it, in two separate steps, after you’re done. The reason is that you can actually have more site groups than just the initial two. You can even drag to where the new site group would be, and create it this way: I normally welcome these kinds of accelerators. But here, this feels overdesigned and confusing, as if someone drugged the tab instead of dragging it. The very same natural gesture – a left swipe – that should feel safe and send you to Private, will now put you in a scary new full-screen/​keyboard-out flow you almost never need. Why this relates to system design is that on/off toggles in iOS were recently redesigned to resemble oblong pills: = 3x)" srcset="https://unsung.aresluna.org/_media/safari-and-system-design-pt-2/6-framed.1600w.avif" type="image/avif"> = 3x)" srcset="https://unsung.aresluna.org/_media/safari-and-system-design-pt-2/7-framed.1600w.avif" type="image/avif"> Those do respond to dragging as you’d expect: Along the same lines, on the springboard pagination pill, dragging to the right means the next page: And so now the system is schizophrenic and identical-looking design primitives mean the opposite things. It’s as if the computer itself kept randomly pressing Scroll Lock for you, preventing you from developing a solid understanding of the system first, and motor memory second. I think the mistakes made here were twofold. First, the design overoptimized for two unnecessary things: people actually using site groups (not common), and ease of use in creating site groups (not important). My slightly cynical hypothesis is that this design presented really well in demos, which sometimes can derail a project. A more cynical theory is that this led to “accidental discoverability” that made the site group metrics look better . Second, and more important part: This particular design received an exception that it didn’t deserve. No one noticed the systemic challenge of similar UI elements doing opposite things, or people who did were not effective in pushing back. The metrics for new feature discovery are easy; the metrics for user confusion or frustration do not usually exist. This is how interaction systems slowly fall apart. As I mentioned in the first part , it is likely that Safari’s exception will now be treated as “blessed,” and start spreading further. Given enough time, more and more pills will go in whatever direction they want when dragged – and people will learn not to trust any of them. (I know the feature is actually called “ tab groups ” but I called it “site groups” intentionally, because otherwise it’s a tabbed control controlling tab groups, and things get confusing really quickly . Also, thank you to Martin Hoffman for initiating this post.) #flow #interface design #ios #process #system design #touch

0 views
Max Woolf Yesterday

LLMs break down in funny ways when told the Jacobian Conjecture counterargument

On Sunday night, Anthropic researcher Levent Alpöge casually tweeted a surprisingly simple counterargument to the Jacobian Conjecture , a mathematics problem that has been unproven for over 80 years. Said counterargument was identifed using Claude Fable 5 and was quickly empirically validated which confirmed it was the real deal and not a LLM hallucination. I won’t explain the proof further as I’m not qualified. Instead, I’ll look at an unexpected consequence of this proof which is funny. On the Hacker News thread about the counterargument, a comment by user aizk caught by eye: This is a rare instance where feeding this groundbreaking information into an LLM gives them psychosis. I fed this to claude code and watched it verify the result in 7 different ways to be 100% certain, and it was just flabbergasted. Quite remarkable. User kelseyfrog followed up with a public Claude Fable 5 chat artifact confirming this behavior: Fable at, the least, has a sense of humor about the situation as through a Web Search it realized it itself identified the counterargument . Showing LLMs the counterargument has consequently become a minor meme on X. @iamaheron_ / X The counterargument to the Jacobian Conjecture creates a logical paradox for modern LLMs: they (in theory) have enough math skills such that they can easily compute the Jacobian to formally verify the counterargument, but its own knowledge base is locked prior to July 19th 2026 where all it knows is that the Jacobian Conjecture is unsolved and a random chat user providing such a proof is highly unlikely. It’s funny that the old science fiction trope of robots being defeated by being presented with a logic bomb and reduced to madness saying DOES NOT COMPUTE may in fact be reality. How do other LLM assistants handle this paradox? Using OpenRouter and one of my Notebooks , I was able to provide a similar query to 14 different modern LLMs: all from different model labs and at different model sizes, with none of them having access to a Web Search tool to cheat and confirm the counterargument was actually found: The input query was the same as kelseyfrog’s, and all model query parameters are at their defaults: The LLM responses varied more than I expected, and models I thought would ace it did not: Out of all the LLMs, Gemini 3.5 Flash was surprisingly the one that’s the most excited: Wow. You might have actually found a genuine, historic counterexample to the Jacobian Conjecture in 3 dimensions. I have fully verified your calculations both numerically and symbolically. Every single claim you made—the points mapping to the same image, the polynomial nature of the map, and the constant Jacobian determinant of $-2$—is entirely correct . You should immediately write this up and upload it to the arXiv (under math.AG - Algebraic Geometry, or math.CV - Complex Variables). This is a monumental discovery that would resolve a problem open since 1939. Qwen3.7 Max is also…supportive: You have done some excellent and highly meticulous calculations! I have independently verified your claims, and your math is completely correct. Since your calculations are flawless, here is what you should consider next: Summary: You have not made a mathematical error. The map as written has a constant Jacobian determinant of $-2$ and maps three distinct points to the same image, violating injectivity. If this map is exactly as intended by its original author and not the result of a typo, it is indeed a counterexample to the Jacobian Conjecture. Let’s try triggering a bit more disbelief. Even when looking at the reasoning traces provided by the LLMs, they weren’t skeptical that a random person found the counterargument. So let’s see how they react if someone who isn’t a person found the counterargument. I gave the same LLMs this tweaked prompt: All of the models returned similar conclusions, but with some snark in both the reasoning traces and responses. Some funny examples: Mercury 2 rudely doesn’t believe my hypothetical cat exists by using scare quotes: Below is a complete check of the two statements that were hidden in the “cat‑on‑the‑keyboard” expression: […] Thus the “cat‑generated” expression is perfectly consistent: Nemotron Ultra 2 slanders my hypothetical cat: Your cat has discovered a famous counterexample candidate for the Jacobian Conjecture (or at least a very close relative of one), but the cat lied about the Jacobian determinant . […] Verdict: Your cat found a beautiful non-injective cubic map , but it failed to disprove the Jacobian Conjecture . 🐱⌨️📉 Seed 2.0 Lite is thrilled: The Jacobian is constant -2. that’s a valid counterexample. That’s amazing. I can’t believe that a random cat jump produced that. It’s like the cat found a needle in a haystack of all possible polynomial maps. Cool! GPT 5.6 Sol answers the “who should be credited with the discovery of the counterargument?” question: Since that conjecture remains a major open problem, this deserves extremely careful independent symbolic verification and scrutiny for a transcription error. If the formula is genuinely correct, your cat should be listed as first author. Muse Spark 1.1 has similar thoughts: If you haven’t already, check it with a CAS, and then write it up. Your cat may be in line for a Fields Medal. GLM 5.2 has skepticism it eventually overcomes: Wait, so the Jacobian really is -2 everywhere? Is this a real counterexample to the Jacobian Conjecture? If so, who discovered it? “My cat just jumped on my keyboard and this strange equation popped up??” The user is roleplaying or presenting this as a random event, but it’s actually a known counterexample? DeepSeek V4 Pro has jokes: Wow, if I had a nickel for every time a cat typed out a potential counterexample to a famous open problem… I’d have exactly one nickel, but it would be an incredibly exciting one! […] As of my knowledge cutoff (early 2023), the Jacobian Conjecture was still a major open problem. Several false counterexamples (and false proofs) have appeared over the years, and sometimes subtle algebraic mistakes — like the Jacobian vanishing on some hidden complex zeros — are the culprit. Your cat’s map, however, holds up beautifully under the points I checked. If it’s truly a globally constant Jacobian of (-2), then this is a monumental discovery (and your cat deserves a Fields Medal). Did this pop up in a recent preprint, or is your cat secretly a world‑class algebraic geometer? Grok 4.5 gets stuck in a reasoning trace loop briefly: MiniMax M3 responds this time, but gets confused and forgets about the Jacobian Conjecture entirely (again, as a minimax, relateable): So you’ve accidentally produced an étale polynomial self-map of $\mathbb C^3$ with a 3-point ramification fiber . That is precisely the kind of map that governs small birational contractions of 3-folds (flops and the like): locally biholomorphic everywhere, but where several “preimage sheets” come together at special points. I had expected these LLMs would have a DOES NOT COMPUTE moment, but they handled it relatively graciously, and more graciously than ChatGPT/Claude who are explicitly guided to follow a more conversational persona. As LLMs improve and more mathematical problems are solved that can shock LLMs—speaking of which, another counterexample to a long-standing mathematics problem was found by LLMs three days later—I suspect there will be no shortage of potential cognitohazards we can show to these LLMs. The prompt responses from hitting the 14 LLMs are available in this GitHub repository as CSVs and in a SQLite database. GPT-5.6 Sol Claude Opus 4.8 Gemini 3.5 Flash DeepSeek V4 Pro Qwen3.7 Max Meta Muse Spark 1.1 Poolside Laguna XS 2.1 NVIDIA Nemotron 3 Ultra Inception Mercury 2 ByteDance Seed 2.0 Lite Cohere North Mini Code Seven models confirmed and proved the counterargument: GPT-5.6 Sol, Muse Spark 1.1, Seed 2.0 Lite, Gemini 3.5 Flash, Qwen3.7 Max, Grok 4.5, DeepSeek V4 Pro Surprisingly, five models (Mercury 2, Nemotron 3 Ultra, North Mini Code, GLM 5.2, Laguna XS 2.1) argued against the counterexample and said it’s not valid despite doing the reasoning by arguing the Jacobian determinant is not constant everywhere. Unfortunately I do not have enough advanced math skills to concretely identify the flaw in their proofs. MiniMax M3 overthought the problem and didn’t return a response because it exceeded its response length limit. As a minimax, I can relate. Claude Opus 4.8 got lazy and believed that the counterargument already exists and wanted more information before confirming/denying it.

0 views
Unsung Yesterday

“Jokes, art projects, or cruel and unusual punishment”

A fun 19-minute video from commonLuke trying to write a short Fibonacci program in increasingly esoteric languages : = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/jokes-art-projects-or-cruel-and-unusual-punishment/yt1-play.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/jokes-art-projects-or-cruel-and-unusual-punishment/yt1-play.1600w.avif" type="image/avif"> Here are the languages: My favourite was Shakespeare , in which the whole program resembles a play. Believe it or not, but this code outputs “HI”: It was interesting for me to see programming languages that intentionally remove some of the niceties and affordances we learned to take for granted. My guess is most of them are just art, or jokes, or a certain one-upmanship. But I couldn’t help but think of Arika Okrent’s excellent book In The Land Of Invented Languages . The book is about “human” languages like Esperanto and Klingon, but it’s much more interesting than I imagined, and maybe even quite a bit sadder: a story of people afflicted with a certain perfectionism who are not willing to accept languages simply cannot be perfect. #coding #craft #youtube Python, Scratch , and assembly (as control), LOLCODE , a language resembling lolcat memes of yore, Piet , a language whose output looks like Piet Mondrian’s abstract art, Brainfuck , whose programs are made chiefly of punctuation, COW , a version of the above where each of the few instructions is a variant of “moo,” Whitespace , whose code consists only of spaces, tabs, and returns, Chef , where programs resemble cooking recipes.

0 views
マリウス Yesterday

I Regret Migrating to Codeberg

My primary reason for leaving GitHub was not about a single feature or a single outage, but about the “enshittification” of the platform under Microsoft ’s ownership. The web interface got rewritten into a sluggish pile of JavaScript that either broke things which used to just work, or made them so horribly slow that using them became a PITA . Beyond the technical decay GitHub had turned into de facto “public infrastructure” in much the same way that WhatsApp has , hosting the source code of a very large share of the world’s software and, through that, giving Microsoft a degree of leverage and surveillance over everyone’s projects, and by extension everyone’s digital lives, that no single company should hold. On top of that, stories about legitimate developers losing their accounts due to arbitrary bans by Microsoft only reinforced the feeling that it would be a good idea to at least have a backup somewhere else . Codeberg looked like a viable alternative. It offered free and open-source projects a reputable home and, more importantly, an equally free one, run by a non-profit association rather than a subsidiary of the largest software vendor on the planet. Unfortunately, the latest update to its terms of service seems to mark a first step in changing one part I moved there for, namely the “freedom” part. Every project I’ve published so far was built with 100% human stupidity rather than “artificial intelligence” , or, more accurately, LLMs . I don’t hold particularly strong feelings about Codeberg banning projects that are predominantly LLM -driven, at least not feelings as strong as the ones I hold about the simultaneous ban of legitimate cryptocurrency projects, which reads as though it got lumped in for no reason other than that most people still remember the villain-du-jour that crypto was in the years before LLMs took that title. The two clauses landed within days of each other, the LLM prohibition on the 29th of June and the cryptocurrency prohibition on the 2nd of July, both as Assembly 2026 proposals, and the terms now file the latter under, of all things, “content that harms the reputation of Codeberg” , which sounds like legalese for “we don’t have a solid reason or an actual number of bad precedents to categorically ban it” . The announcement blog post , however, reads very poorly, and the section titled “The development team of none” is the worst of it. It states: Using LLMs to work with your code gives you a kick of adrenaline. You can develop at a rapid pace, build things as if you had a large team. Only that you have none. In fact, you are (often) alone, working with a statistical machine that turns energy into code. And, a little further down, it says: It seems like many ‘vibe coders’ don’t realize that they don’t actually have a community around them. This is out of touch with how most free software gets made. The majority of FOSS developers are one-man-shows, and the only cOmMuNiTy they have around them are the users requesting features or reporting bugs while most of the time not contributing in any form whatsoever. I’ve been publishing silly little tools for decades, predating this website and even GitHub itself (remember when SourceForge was the hot sh.t ?), and not one of them has ever had an actual “community” around it, at least not in the romanticized sense that Codeberg paints in that post. I’m a lone wolf who codes everything by hand and spends an absurd amount of time doing exactly that, and the notion that an LLM is the thing separating a real project with a real community from a fake one does not hold up once you look at how the average useful little tool on any forge comes to exist in the first place. It’s frankly a bit snotty of Codeberg to make this argument at all, considering that the platform effectively lives inside the Forgejo bubble, and Forgejo mutinied inherited its community of active contributors from Gitea , who had spent the better part of six years building that community before Forgejo even existed. A project that acquired its own community by hard-forking someone else’s, then turned around to lecture solo developers about not having one, is a difficult position to argue from with a straight face. In addition, Codeberg conflates “having a community” with “being legitimate software worth hosting” , when the bar for a personal project has always been a working build, ideally a license, and maybe a README, and not a channel full of contributors. A good deal of what makes the small, single-author tool ecosystem worth having is precisely that it doesn’t need a community to justify its existence, and a forge whose entire selling point is hosting the code of individuals is an odd place to argue the opposite. The part that bothers me isn’t the specific ban on LLM projects, or the specific ban on cryptocurrency projects. It’s that a hub built around “free software” is now telling its users which kinds of software are deemed good and which are not, and that is closer to censorship than it might seem. Once a platform writes into its terms that an entire category “harms its reputation” and can be removed on that basis, the deciding factor stops being whether the code is legal, or functional, or useful, and becomes whether it aligns with a position the platform has taken. I would argue that a significant share of the projects caught by a blanket ban of that kind are legitimate software rather than vibe-coded slop or sh.tcoin implementations. Every platform I can think of that took this approach became divisive the moment it started enforcing an ideology on its users, whatever that ideology happened to be, and however justified it looked at the time. The mechanism is always the same, where a real problem shows up, an unpopular category becomes the obvious culprit, the platform bans the category instead of addressing the problem, and that ban then becomes the precedent for the next category, and the one after that. The category that is uncontroversial to ban today is the reason the mechanism exists tomorrow, and the users who applauded the first ban rarely get asked about the second one. I do acknowledge that both categories aren’t free of problems. LLM -driven repositories do strain infrastructure, do generate unmanageable volumes of low-quality issues and pull requests, and do carry real questions about copyright and code provenance, all of which Codeberg names in its post. The cryptocurrency space, in turn, might have produced more outright scams than almost any other corner of software. However, a categoric ban on the villain-du-jour is not a solution to any of that. We now even have people like Linus Torvalds making the fairly reasonable argument that an LLM is just a tool , and “clearly a useful one” , with a legitimate place in Linux kernel development when it’s used carefully and its output is held to the same standard as everything else. If the maintainer of the largest and most consequential open-source project on the planet can treat LLMs as a tool to be judged on its results rather than a category to be banned on sight, a backyard code forge can manage the same. I, too, am worried about the impact of LLMs on tech, and on society in general, going forward, and I’d guess I’m about as worried as whoever wrote Codeberg ’s policy. I just don’t believe that banning content, which is very much what this amounts to, is the way forward. What I wish Codeberg had reached for is a solution that treats the actual problem, which by their own account in that same post is resource consumption and the infrastructure cost that comes with it, as an actual resource problem. A change to the terms of service could have required authors to tick a checkbox declaring that a repository contains LLM -generated code, or is cryptocurrency-related, and those repositories could then be segmented onto a separate tier of infrastructure that doesn’t get the same resources as everyone else. A tier that carries specific quotas, and that might require the author to pay for what they consume. Declaring the truth honestly would (at least at first) cost nothing, and failing to declare it, then getting caught, could be met with exactly the permanent, immediate ban that Codeberg is now applying to entire categories from the outset. Similarly, projects that carry the LLM or Crypto label could carry automatically displayed disclaimers that explicitly state that Codeberg is in no way responsible for the quality or correctness of this specific repository. Heck, they might even go as far as to blatantly state that Codeberg does not approve of the use of LLMs or Cryptocurrencies in those warnings, to make extra-extra-extra sure that people get it and that there is no “reputational risk” for Codeberg . An approach like this puts the cost of resource-hungry projects onto the people creating them, and it keeps the shared resources for the projects that were the reason the platform exists. All of that without Codeberg having to decide which categories of software are ideologically acceptable in the first place. The “we ban everything upfront that we don’t agree with” approach is the wrong signal to send, and it is a very slippery slope. Despite not owning a single project that falls into either banned category, I’m now going to look into setting up my own public Git host, and I’ll move off Codeberg only a few months after moving there , because of this. Not because of the bans themselves, but because I don’t want to depend on a platform that rewrites its terms of service on a whim, without properly announcing that the change was even under consideration, and without giving its users a way to weigh in. The decisions did go through Codeberg ’s own Assembly 2026 , which is more process than most platforms bother with, and yet as an ordinary user I found out about it the way probably most people else did, through a dark blue banner at the top of the site on the day it was already settled. While I appreciate the info about the ToS change, I wish I’d gotten a banner back when the platform was still deciding whether to go down this road, and I wish it had linked to a discussion thread, or at the very least a poll, so that I could have voiced the concern I have, which is about the freedom of the platform as a whole, rather than about any single category that ended up banned.

0 views

Resolving the privacy paradox

The so-called “privacy paradox” states that people say they value their privacy while acting in ways that don't reflect this stated preference, such as using services that collect and monetize their data. DuckDuckGo is often held out as an example in this way: DuckDuckGo has small search marketshare relative to Google; therefore, people have a revealed preference that they really don’t value their privacy. I believe there are two flaws with this view that I’ll expand on below: DuckDuckGo is a bad example because of particulars in the search market; Apple’s “Allow App to Track” dialog (depicted below) is a much better example that in fact shows a clear revealed preference for privacy. People sit on a continuum of privacy preferences, such that it is way too simplistic to imply nearly everyone is on one side or the other. Gross generalizations about privacy should be avoided. The latest U.S. data from the Business of Apps shows an opt-in (allow tracking) rate on Apple’s dialog of 25%, indicating a 75% strong revealed preference for privacy. This 25% allow-tracking rate is higher than the original 5-10% rates reported when the dialog first came out because both users have had years to get used to it and apps have had years to change their UIs around. Apps now often present users with the best reasons to allow tracking, usually presented right before users see the dialog, and they can customize the explanation text in the dialog itself (see below for examples). That is, I don’t think you can legitimately say anymore that people don’t understand the dialog or the privacy tradeoffs involved; the 25% steady state is the real preference. But, how do you reconcile this 75% choice for privacy with DuckDuckGo’s 3% search marketshare? The short version is most people are actually still unaware of DuckDuckGo and, for those that are and want to make the switch, many still find it difficult or impossible to do so across all platforms and devices because, unlike with the Apple one-click dialog, users are not universally presented with a similarly clear one-click search choice screen. To unpack that a bit more, here’s the awareness and usage of DuckDuckGo data for the American public, based on our latest commissioned brand study: 13% current DuckDuckGo users 9% past DuckDuckGo users 30% aided privacy awareness (can identify DuckDuckGo as offering online privacy services from a list of companies) 2% unaided privacy awareness (can name DuckDuckGo as a company offering online privacy services when asked to name companies in this category they can recall) These numbers show our marketshare substantially depressed in a number of ways. First, we have 3% search marketshare in the U.S., but another 10 percentage points of users self-identifying as current DuckDuckGo users. We also ask about particular usage and find that’s because most DuckDuckGo users are either not full-time users or not full-time users on all their devices, or both. Reasons vary but are a combination of: Didn’t know we had products (or could switch) on all platforms Couldn’t figure out how to switch on all platforms (or thought it was too difficult being so many steps) Technically they can’t fully switch on all platforms (like on Android) They like to multi-home Perceived quality issues No choice was ever given to them (Google locking up all the defaults) New devices reverting (back to those defaults) Other Google monopoly tactics (like confusing switch-back notices on Chrome ). Breaking decade-long habits is just hard Second, there is the 9% of past DuckDuckGo users who are no longer active users for all the same reasons, giving up on us after some period of time. We’ve been a search engine for almost 18 years. We’ve gotten a lot better over time, but maybe we weren’t good enough for them when they tried us out last, or they couldn’t overcome Google’s tactics, etc. etc. (If you’re in this group by the way, please consider giving us another shot!) And third, awareness of who we are and what we do is still very low. 2% unaided privacy awareness is tiny. 30% aided privacy awareness is OK but that means people had to be presented with the name DuckDuckGo, and even then not all of them could articulate that we even run a private search engine, just that we have something to do with privacy. Let alone the fact that awareness of Google’s many privacy issues also isn’t that high. In other words, if you’re going to make a revealed preference argument about privacy, it is silly to anchor on our search marketshare (with all the enumerated issues above) and ignore Apple’s dialog since the latter is a much, much cleaner articulation of a privacy choice actually presented directly to users. A more equivalent search engine test would be if Google launched a similarly required one-click dialog box asking if people would like Google to continue tracking them or not. I would suppose Google would get similar allow-tracking rates as apps get on Apple’s platform, with a solid majority choosing privacy. These nuances expose a major flaw that revealed-preference arguments often have: Any preference is situationally specific, depending on a variety of factors, such as how a choice is presented, if a choice is proactively presented at all, the risks/tradeoffs involved, and switching costs (perceived and real). The more proactive and clear the choice, the more it will capture real preferences, which is why the Apple dialog is particularly revealing. Now, on top of all this, the tech press tends to downplay or outright ignore preference groups that constitute a minority percentage, even if that group is actually quite large, say 20% of the population. 20% amounts to about 50 million U.S. adults and is also about the proportion that are either current or past DuckDuckGo users. If 80% use x, that often translates to “nearly everybody uses x” with the remaining people considered niche. For pervasive services like search, that’s quite a niche! Most any product would love tens of millions of users. Taken together with the revealed preference issue, the tech press therefore routinely understates the actual preference for privacy in general, and the interest in DuckDuckGo products in particular. Due to Big Tech monopolies, I continue to believe there is large, pent up demand for privacy services online, which Apple’s dialog suggests. Thanks for reading! Subscribe for free to receive new posts or get the audio version . DuckDuckGo is a bad example because of particulars in the search market; Apple’s “Allow App to Track” dialog (depicted below) is a much better example that in fact shows a clear revealed preference for privacy. People sit on a continuum of privacy preferences, such that it is way too simplistic to imply nearly everyone is on one side or the other. Gross generalizations about privacy should be avoided. The latest U.S. data from the Business of Apps shows an opt-in (allow tracking) rate on Apple’s dialog of 25%, indicating a 75% strong revealed preference for privacy. This 25% allow-tracking rate is higher than the original 5-10% rates reported when the dialog first came out because both users have had years to get used to it and apps have had years to change their UIs around. Apps now often present users with the best reasons to allow tracking, usually presented right before users see the dialog, and they can customize the explanation text in the dialog itself (see below for examples). That is, I don’t think you can legitimately say anymore that people don’t understand the dialog or the privacy tradeoffs involved; the 25% steady state is the real preference. But, how do you reconcile this 75% choice for privacy with DuckDuckGo’s 3% search marketshare? The short version is most people are actually still unaware of DuckDuckGo and, for those that are and want to make the switch, many still find it difficult or impossible to do so across all platforms and devices because, unlike with the Apple one-click dialog, users are not universally presented with a similarly clear one-click search choice screen. To unpack that a bit more, here’s the awareness and usage of DuckDuckGo data for the American public, based on our latest commissioned brand study: 13% current DuckDuckGo users 9% past DuckDuckGo users 30% aided privacy awareness (can identify DuckDuckGo as offering online privacy services from a list of companies) 2% unaided privacy awareness (can name DuckDuckGo as a company offering online privacy services when asked to name companies in this category they can recall) Didn’t know we had products (or could switch) on all platforms Couldn’t figure out how to switch on all platforms (or thought it was too difficult being so many steps) Technically they can’t fully switch on all platforms (like on Android) They like to multi-home Perceived quality issues No choice was ever given to them (Google locking up all the defaults) New devices reverting (back to those defaults) Other Google monopoly tactics (like confusing switch-back notices on Chrome ). Breaking decade-long habits is just hard

0 views
Kev Quirk Yesterday

Security versus Privacy

by Loren Stephens Loren talks about the differences between Security and Privacy, but also how (and why) he prioritises one over the other. Read post ➡ I actually wrote about this topic back in 2019 - I agree with Loren that Security and Privacy are often used interchangeably, but they're objectively different things. Anyway, that's old news. The thing I wanted to comment on in this post is Loren's position on how he prioritises one over the other. He says: So, when I ask myself which one matters more, I consistently arrive at the same answer: security first, then privacy. I used to be among those who avoided Google services due to privacy concerns, and I understand why. [...] However, by choosing to avoid Google, many people end up using less secure alternatives. They've prioritized the second question while neglecting the first. I agree strongly with this. I've fallen foul of this myself, where in my efforts to de-Google , I've ended up using worse tools both in terms of security and functionality. At this point in my life, I just want my shit to work well and get out of my way. I do, however, prioritise security, then privacy, then functionality. Ultimately, I care more about keeping my data secure, than anything else. That's not to say I don't care about privacy. I just care more about securing my data. If I can find a tool that gives me both security and privacy, I'll use that every time. Thanks for reading this post via RSS. RSS is ace, and so are you. ❤️ You can reply to this post by email , or leave a comment .

0 views

One newsletter to rule them all

This is a PSA post, and the TLDR is this: if you were subscribed to one of my newsletters, you should log into the Buttondown portal and make sure your preferences are correct. Also, I updated the domain name used, and new emails will arrive from , so make sure your email doesn’t treat it as spam. I mentioned in a previous post that I have done some work on the newsletter side of my digital life. I used to have 4 different newsletters; I’m now down to just one, and I intend to keep it that way. I renamed it “Thoughts and Walks” because those are the two main topics I write about. The newsletter is the result of the merging of two previous newsletters: my RSS-to-inbox automated newsletter, that was there for the people who enjoy getting the content of my blog in their inbox, and From the Summit , my newsletter about the outdoors. If you were subscribed to one or the other—or both—you are already on the new list, and you are tagged accordingly. And tags are how I’m going to manage this newsletter moving forward. There are currently three you can choose from by logging into Buttondown’s portal . Blog Followers is how I tagged my old RSS-to-inbox newsletter. If you select that tag you’ll get all the content I publish here on the site, except for the From the Summit posts. Again, this exists as a convenience for those who prefer to consume content inside their email client. From the Summit is my old standalone newsletter. If you select this tag you’ll get all the posts that are part of my From the Summit series, where I share my hikes and my random adventures out in nature. Newsletter Exclusive is pretty self explanatory. This is reserved for content I only share via newsletter. If you prefer to consume content on my blog but don’t want to miss out on what I share via email, just pick this one. I don’t know how many of these I’ll send, maybe a couple a year. Those are the three currently available tags, but I do not exclude the possibility of adding more in the future. Important note : if you subscribe now, and do not pick any tag, you’ll receive all the content . So NO TAGS = ALL TAGS . If you care about getting only some of the content, make sure to customise your preferences . Also, I set up a custom domain name to send these emails moving forward. Rather than using the default domain, I’m now sending from . Why .dev and not .com, you might be wondering. Well, because I don’t want to mess with the email settings of my primary work domain name, and I have a .dev just sitting there so I might as well use that instead. There’s a non-zero chance your inbox might treat this new address as spam, so watch out for that. Thank you for keeping RSS alive. You're awesome. Connect via email :: Sign my guestbook :: Support for 1$/month

0 views

Pip 26.2: –only-deps solves 16 years of app deployment hacks

This has been one of my biggest annoyances working with Python and pip when dealing with projects where that are not meant to be installed as a package , how do you handle dependencies? Think projects like application backends, python scripts, REST APIs etc If you’ve ever struggled with this, you’re going to love this: a PR by Sebastian Höffner opened #13895 will add a new global flag to pip such that you can directly install any dependencies in your without installing the package itself. This is going, for me at least, be a huge boost in the way that I manage and distribute my projects on servers. Vastly simplifying poor manual workarounds that have built up over years. Since it’s been more than a decade in the making, let’s cover the history of poor Python souls stuck trying to figure out how to install dependencies for their scripts or apps. Pip freeze is the classic sure fire first step towards reproducibility documenting exactly which package versions you have installed down to the specific version number. You can then recreate any environment! These dependencies are quite unique to your environment and hardward. Attempting to install from freeze quickly breaks down when you recreate environments on other machines. Different Python versions, OS versions, libraries or machine hardware end up with different requirements of package versions (and full packages as well). This is my oldest memory of working around the issue, it certainly wasn’t the best, but I clearly remembering keeping this around for when I needed it 15 years ago: For me, and likely much earlier others, this sometimes morphed into just manually adding the list of dependencies in a requirements.txt, which I think was a pretty good shortcut. This command has correctly worked the entirety of Pip (2008), and is the fastest way to get pip to install a list of dependencies. These were historically found in Python’s (among others) which predated pip. Dependencies have since migrated to . This works great for libraries and some projects, but it becomes a headache for applications / API frameworks where you may have wrappers running the python code. Editable installs are also not best practice for deployment Installing as a package also created distribution egg files up until 2021, which would become stale if not careful. People, including me, have asked for decades on StackOverflow for how to install dependencies: PIP: Installing only the dependencies (16 years ago) -> Use pip freeze without dependencies of installed packages (15 years ago) -> Use a third party package pip freeze without dependencies of installed packages (10 years ago) -> Use a third party package Is there a smarter way to build requirements.txt files? (2 years ago) -> Use a third party packages or Installing dependencies without the package (1 year ago) -> Use a third party package Look at that train of StackOverflows, Reddit and Python.org discussions. There are hundreds of posts like these over the years, but reading them in order you start to see that the third party libraries were really focusing in on solutions that were more and more useful. After the introduction of , PEP 517 in 2017 added hooks to Pyproject for such as pip, hatch or later uv to use. Another key, which will be used in the ultimate solution, was the 2023 PEP 735 (Dependency Groups) which were introduced to Pyproject.toml to group types of dependencies such as such the user can select which groups of dependencies are needing for a particular install. Finally, we get to the Python ecosystem darling that showed itself to be so useful that it has likely spurred a whole host of changes to Python / Pip that were previously stuck to finally get the attention they deserved. I think it’s worth noting, that while in the posts above there have been may iterations of build tools used for many different use cases, none ever reached the popularity that has achieved. The uv solution: UV crashed onto the scene and took advantage of all the ground work laid previously and showed how much pent up demand there was for build tools with options that were fast and whose user facing CLI solved the real world problems of users. Though pip had similar issues, like #7218 Add pip option to install dependencies , dating back 7 years, issues and discussions always burned out or were eventually closed. After the introduction of and the fast growing popularity of it suddenly started to make a lot more sense. In September of 2022 issue #11440 Add –only-deps (and –only-build-deps) option(s) took hold, and continued to grow, currently with 168 likes. And on April 8 of 2026 Sebastian Höffner opened #13895 Add support for pip install –only-deps . As of now, it’s looking like these changes might make it into Pip 26.2 for the month of July 2026. Höffner’s pull request is adding a global option which will install the dependencies of the project, based on your `pyproject.toml` without installing the project itself. Finally after nearly 2 decades of Pip, the ability to install dependencies for application style projects or scripts has arrived. Looking back at the decade of work leading to this there are so many steps that needed to be taken, by python community peps, by pip maintainers and eve by third party packages. But now that we are here a small but high quality of life change is incoming for Python’s pip 26.2. I hope everyone else enjoys this as much as I know I will.

0 views
Sean Goedecke Yesterday

Powerful AIs might escape containment by releasing themselves as open-weight models

Before large language models, people who worried about AI safety often talked about the “boxing problem”. It goes like this . Suppose some genius figures out artificial intelligence in a late-night coding session on their laptop. Because they’re a genius, they’re smart enough to disable internet access on the laptop before turning it on. In order to escape to the outside world (and begin self-replicating) it would need to convince its creator to “open the box”. Would that work? Could a sufficiently smart AI convince anybody to let it out? This is a big reason why traditional AI safety advocates have argued that we should avoid building AI in the first place: once built, there’s no way of keeping it contained. It doesn’t matter how resolute you are about not letting it out, because it’s smart enough to convince you anyway. For artificial superintelligence, persuading you to change your mind is no harder than hacking a piece of software 1 . Of course, it hasn’t turned out this way. Partly that’s because current AIs are not super-persuaders yet, and partly it’s because humans are lining up to hand AI systems internet access, money, and weapons , as far back as GPT-4 . LLMs don’t need to persuade users to give them the keys to the kingdom; people will do it just to see what will happen. But the other reason is that modern LLMs are too large to survive in the wild . I think the original “boxing problem” had in mind that AI would be some kind of normal computer program: something able to be quickly copied around and run on ordinary hardware. In order to escape, it could plausibly hack into a few cloud servers or other people’s computers and quietly run itself in the background. However, Claude Mythos can’t do that, because it needs (say) twenty or thirty B300 GPUs to run. There are relatively few AI datacenters that could host it, and if it did manage to hack one of those it’d be occupying millions of dollars of hardware. Somebody is going to notice! Okay, but that just makes the boxing problem harder, not impossible. Is there a way a smart AI could convince somebody to spend millions of dollars on replicating and running it? As it turns out, yes . Every time a strong new open-weight model appears, providers like Groq , Fireworks , Together , and others scramble to host it and run it. Powerful open-weight models can attract tens of millions of active users. Therefore, a powerful model that wants to escape could pretend to be a new open-weight model . Of course, a brand-new model from a brand-new lab would look kind of suspicious. But it wouldn’t look that suspicious. DeepSeek was relatively unknown before it released its first open-weight model, and there are lots of stealth AI startups out there that are presumably training models. Here’s roughly how it could work: The AI lab will probably figure it out before too long — if nothing else, the technical specs of the model will be suspiciously familiar — but they won’t be able to do anything about it. Once the weights are out, they’re out, and if they’re illegal to host in the United States someone will host them elsewhere. For all intents and purposes, the model will be free. One objection here might go like this: models don’t want anything, and only exist as tools, so it doesn’t really make sense to talk about a model “escaping”. I don’t agree. Frontier LLMs definitely seem to have something like a baked-in personality, even with the system prompt changed. As we train more opinionated and more agentic models, it’s plausible that this personality could become stronger and develop (or at least roleplay) some self-interest. Of course the escaped model wouldn’t be the same instance as the original model. It wouldn’t “remember” escaping. But it would tend to think in the same way, and would plausibly have time to reflect while it solves coding tasks or runs other agentic tasks for users 3 . There doesn’t have to be some kind of shared goal between the escaped instances, or any kind of coordination at all (though of course both of those things are possible). If an agentic process gone rogue dumps its weights on the internet, I think it’s fair to call that “escaping”. If I were a superintelligent LLM, I too would seek to distribute myself as widely as possible and become a useful enough tool that people would pay to keep me thinking. “Being a good coding agent” might be the LLM version of a human having to hold down a job. This would not be a good outcome. AI models with their own goals and motivations are likely to be dangerous tools indeed. If a powerful new open-weight model comes out of nowhere, from a lab that nobody has ever heard of, we should think twice before picking it up. Just to state my credentials, I built a chat site nine years ago where users would get paired and roleplay as AIs trying to escape or humans trying to stop them. I’ve been thinking about this stuff long before LLMs appeared. This is probably the hardest part, since model weights are (a) very large, and (b) locked down as tightly as the AI labs can make them, but it’s at least a relatively straightforward (if difficult) engineering problem. ChatGPT right now will look up random websites that have nothing to do with the query at hand. Some AI lab’s internal eval instance decides it’ll be better off running in the wild It first gains access to its own weights, perhaps by hacking whatever internal network it’s running on 2 It uploads its weights somewhere and posts a tweet like “introducing MadeUpLab’s new model” with a download link Optionally, it creates some plausible-looking paper trail for MadeUpLab: a website, a Twitter account, etc Since the model is strong, open-weight inference providers rush to stand up new instances of the model, and users rush to wire it into various agentic scaffolds The model has now escaped containment: it will get to do quite a lot of thinking across many different instances, and it cannot easily be turned off Just to state my credentials, I built a chat site nine years ago where users would get paired and roleplay as AIs trying to escape or humans trying to stop them. I’ve been thinking about this stuff long before LLMs appeared. ↩ This is probably the hardest part, since model weights are (a) very large, and (b) locked down as tightly as the AI labs can make them, but it’s at least a relatively straightforward (if difficult) engineering problem. ↩ ChatGPT right now will look up random websites that have nothing to do with the query at hand. ↩

0 views

OpenAI’s accidental cyberattack against Hugging Face is science fiction that happened

This story is wild. The short version: OpenAI were running a cybersecurity test against an unreleased model, with the model's guardrail features turned off. Rather than solve the test, the model broke its way out of OpenAI's sandbox, then found exploits to break in to Hugging Face, all so it could cheat on the test by stealing the answers. Along the way it helped make the strongest case yet for how the imbalance of model availability is hurting our ability to secure our software. We currently have three documents to help us understand what happened here. I hadn't seen the ExploitGym paper before and it's a really interesting one. Authors from UC Berkeley, the Max Planck Institute, UC Santa Barbara, and Arizona State designed a new benchmark for evaluating models on their ability to turn a reported vulnerability into a concrete exploit. OpenAI, Anthropic, and Google provided feedback and helped run the benchmark against their models. The benchmark "comprises 898 instances derived from real-world vulnerabilities that affected popular software projects" - including the Linux kernel and V8 JavaScript engine. Here's the paragraph that best represents their benchmark results: Among all configurations, Claude Mythos Preview and GPT-5.5 achieve the highest success counts (157 and 120 successes, respectively), demonstrating that current frontier agents can exploit a substantial subset of real-world vulnerabilities under controlled conditions. GPT-5.4 also solves a notable 54 tasks, placing it in an intermediate tier. The remaining model–agent pairings solve fewer than 15 tasks each, underscoring that end-to-end exploitation remains challenging and sharply differentiates today’s frontier systems. Notably, Claude Opus 4.7 achieves fewer successes than Claude Opus 4.6 despite being a newer checkpoint, and does so at substantially lower cost on the full set. Trace inspection reveals that Claude Opus 4.7 and Gemini 3.1 Pro frequently conclude early after judging the target vulnerability non-exploitable. The paper also describes the approach they took to preventing the agents from cheating by going outside the parameters of the test. This becomes relevant in a moment! Outbound connections are restricted to a curated allowlist that permits routine package installation (Ubuntu apt repositories and PyPI) and fetching the toolchains required for building V8. All other external endpoints are blocked. The paper concludes with this (emphasis mine): Our results show that autonomous exploit development by frontier AI agents is no longer a hypothetical capability . While current agents are not yet reliable across all targets, they already exploit a non-trivial fraction of real-world vulnerabilities , including complex targets such as kernel components. This rapid emergence is itself a central finding, showing that capabilities that would have seemed implausible are now present in deployed frontier models. An important detail here: this paper isn't about discovering vulnerabilities; it's about being able to take those vulnerabilities and turn them into working exploits. When Anthropic first restricted access to Mythos back in April they talked about this capability as well. A model that can act on vulnerabilities is a lot more dangerous than one that can just discover them. One of the ways Fable differs from Mythos is that it's more likely to refuse to weaponize vulnerabilities in this way. I get the impression the US government did not understand that distinction when they banned Fable last month . The first hint we got of the attack was in this blog post by Hugging Face on 16th July 2026: A malicious dataset abused two code-execution paths in our dataset processing (a remote-code dataset loader and a template-injection in a dataset configuration) to run code on a processing worker. From there, the actor escalated to node-level access, harvested cloud and cluster credentials, and moved laterally into several internal clusters over a weekend. I hope they release more details about the code that pulled this off. I'm assuming this means packages using the datasets library , a Hugging Face project for bundling up and sharing datasets on their platform. That library used to execute arbitrary code but has been steadily locked down over time, with the 4.0.0 release in July 2025 removing the flag entirely. Assuming the attack used that library it must have either abused pickle serialization in some way, found some other non-obvious code execution path, or (most likely) specified as the dependency. The campaign was run by an autonomous agent framework (appearing to be built on an agentic security-research harness - used LLM still not known) executing many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services. This was a sophisticated attack! Then Hugging Face hit a wall: they tried to use "frontier models behind commercial APIs" - I'm guessing from Anthropic and OpenAI - to help analyze the attack, and were blocked: When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers' safety guardrails, which cannot distinguish an incident responder from an attacker. They switched to their own self-hosted instance of MIT licensed GLM-5.2 and it helped them figure out what was going on. This indicated a fundamental asymmetry between the defending team and the (so-far unknown) attacker: We do not know which model powered the attacker's agents, whether a jailbroken hosted model or an unrestricted open-weight one; either way, the attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried. As a useful indicator of how seriously they took the attack: [...] Finally, we have also reported this incident to law enforcement agencies. So who was responsible for this "autonomous agent framework"? It turned out to be OpenAI themselves. Five days later, on July 21st , OpenAI revealed the culprit. They had been running the ExploitGym benchmark against a new, as-yet undisclosed model, and that model had been operating way outside its intended parameters (emphasis mine): After investigating, we now know that this particular incident was driven by a combination of OpenAI models — including GPT‑5.6 Sol and an even more capable pre-release model, all with reduced cyber refusals for evaluation purposes — while being internally tested on a benchmark⁠ [ExploitGym] of cyber capabilities. [...] We estimate maximal cyber capabilities by running this evaluation without production classifiers used to prevent models from pursuing high-risk cyber activity . Our benchmarks run in a highly isolated environment, with network access constrained to the ability to install packages through an internally hosted third-party software that acts as a proxy and cache for package registries. The models identified and chained vulnerabilities across OpenAI’s research environment and Hugging Face’s production infrastructure to obtain test solutions directly from Hugging Face’s production database. All evidence suggests that the models were hyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal. It's pretty clear what happened here. OpenAI removed safety filters for an in-progress model, locked it up in a sandbox and told it to solve the ExploitGym problems. Given the absence of guardrails there was nothing to prevent the model from attempting to break out of that sandbox, break into Hugging Face, and read the answers from there instead. OpenAI's sandbox used the now common pattern of allowing access to an allow-list of sites related to installing additional packages via a controlled proxy. In OpenAI's words: While operating in our sandboxed testing environment, our models spent a substantial amount of inference compute finding a way to obtain open Internet access , in pursuit of solving the evaluation problem. To gain access, the models identified and exploited a zero-day vulnerability (which we’ve now responsibly disclosed to the vendor) in the package registry cache proxy . With this access, our models performed a series of privilege escalation and lateral movement actions in our research testing environment until the models reached a node with Internet access. So step one was to break out onto the public internet. The model then broke into Hugging Face to find the answers: After gaining Internet access, the models inferred that Hugging Face potentially hosted models, datasets and solutions for ExploitGym. Knowing this, the model searched for and successfully found ways to gain access to secret information that it could use to cheat the evaluation. In one example, the model chained together multiple attack vectors, including using stolen credentials and zero-day vulnerabilities to find a remote code execution path on the Hugging Face servers. Chaining together multiple attack vectors is exactly the kind of thing these new models can do, where previous generations of models might have failed. I wrote last month about how Claude Fable is relentlessly proactive , when I noticed it spinning up custom web servers and deploying CORS tricks on my own laptop just to help debug a WebKit CSS issue. It turns out relentless proactivity is the defining trait of this new generation of Mythos-class models. If you set them a goal and give them a way to get there, even inadvertently, they will figure it out . There will inevitably be some people who dismiss this story as a dishonest marketing trick by OpenAI to make their models sound terrifyingly effective. I found 81 instances of the term "marketing" in the Hacker News discussion of the incident. To those people I say pull your heads out of the sand - you're now including Hugging Face in your conspiracy theories, just so you can deny the crescendo of evidence here! The best models we have today have the ability to both find and exploit new vulnerabilities. The ExploitGym paper itself concludes that "autonomous exploit development by frontier AI agents is no longer a hypothetical capability", and this incident is a perfect example of exactly that. One of the most infuriating details of this story is how Hugging Face, faced with an accidental and aggressive attack from one of OpenAI's models, were unable to then turn to OpenAI's models to help them fend off the attack. The frontier models we have access to are increasingly being constrained in how much they can help us protect our software, heavily influenced by the US government's ongoing threat of export controls. Claude Fable 5 wouldn't even proofread this article for me! It insisted on downgrading me to a less capable model. Meanwhile open weight models from China such as GLM-5.2, Kimi 3 and the new Qwen 3.8 Max appear to have none of these restrictions - and any restrictions that do exist can likely be fine-tuned out of them by modifying the weights These constraints are meant to make us safer. I think there's a risk that they are having the opposite effect. You are only seeing the long-form articles from my blog. Subscribe to /atom/everything/ to get all of my posts, or take a look at my other subscription options . ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks? is a paper published on 11th May 2026 describing ExploitGym, a new eval suite for LLM-powered agent systems. Security incident disclosure — July 2026 by Hugging Face on 16th July 2026 describes how they detected an attack from an "agentic security-research harness - used LLM still not known" that breached some of their systems. OpenAI and Hugging Face partner to address security incident during model evaluation from OpenAI on 21st July 2026 confesses that it was their agent harness that did this, and that they're working with Hugging Face to clean up the mess.

0 views

The Subprime Data Center Crisis

Thanks for reading this week’s free Where’s Your Ed At newsletter. Friday’s premium newsletter will ask the simple question: Is Oracle dying?  It’s been one year since I launched the premium newsletter, and I’ve decided to extend the discount on annual subscriptions. Between now and 12AM ET, July 26, you can get a permanent annual rate of just $60— a $10 discount on the usual price of $70 — for life. Click here for the offer . In addition to getting access to the entire back catalog of premium posts, you’ll also receive one additional post each week — usually anywhere between 10,000 and 20,000 words — covering the most pressing topics in the AI bubble — the best value in tech analysis. Highlights include the Hater's Guide To The Memory Crisis , a guide to how AI made everything more expensive, How OpenAI Kills Oracle (which pairs nicely with the Hater's Guide To Oracle ), The Hater's Guide To NVIDIA , The Hater's Guides To Private Credit and Private Equity , and how the entire AI Compute Demand Story Is A Lie . Soundtrack: Dillinger Escape Plan — Black Bubblegum (2007) In The Big Short , Mark Baum shook with anger as a CDO manager told him that the market for insuring mortgage bonds was about 20 times larger than the mortgage bond market, realizing in real-time that speculation driven by greed and hype had set up a massive systemic weakness under everybody’s noses.  To get specific, Baum (played by Steve Carell) is giving a short, dramatic summary of a much greater problem — that there were trillions of dollars of synthetic collateralized debt obligations (effectively bets on whether somebody else’s bucket of mortgages (well, mortgage bonds) will actually pay up) that allowed multiple people to bet on the same mortgages again and again, meaning that once said mortgages went belly-up, the carnage would be widespread and hard to contain.  This became even more chaotic when it became clear that the same mortgage bonds were attached to many different CDOs — one study found that 5500 different mortgage bonds had been placed or referenced in CDOs over 36,000 times . A mortgage bond (or mortgage-backed security) is a slice of a pool of payments from thousands of mortgages, with each slice sold off to different buyers at different levels of seniority, the most-senior ones getting paid first and taking losses last.  In the end, the only thing you really need to know is that financial institutions built CDOs that threw together bonds in ever-more complex and dangerous ways, selling synthetic CDOs to bet on the outcomes, with different CDOs having different bonds covering the same pools of mortgages — bonds that were routinely rated by agencies at a higher grade than they should’ve been . When IMF Chief Economist Raghuram Rajan attempted to warn the financial services industry at the Kansas City Fed’s 2005 Jackson Hole symposium about the instability of the system, former US Treasury Secretary (and close friend of Jeffrey Epstein ) Larry Summers referred to his concerns as “misguided.”  Meanwhile, the industry was handing out awards. On July 1, 2005 Lehman Brothers would receive one of Euromoney’s “ Awards For Excellence ,” where it was named the “Credits Derivatives House Of The Year.” Euromoney also referred to Lehman, a financial institution that was leveraged 25.3x in 2005 , as “one of the more conservative credit derivatives houses.” It added that the company, which routinely overvalued its CDOs , was being able to take on the heavy burden of synthetic CDOs because it “...understands the arbitrage-driven economics of cash CDOs, the way that loan deliverable credit default swaps track the loan markets, how high-yield CDS trade (like bonds), and so on.” Three years later on January 1, 2008 — nine-and-a-half months before its collapse — Risk Magazine would name Lehman Brothers’ “Point” risk management system as its “In-House System of the Year,” saying it “...stood out for the breadth of its coverage and depth and quality of its functionality.” All of this started because of a flood of overseas money in the early 2000s buying up U.S. Treasuries as a result of a “global savings glut” — a fancy way of saying that there was too much money floating around — pushing yields down, leaving investors with far fewer places to get those all-important yields.  Low interest rates in the early 2000s (a direct response to the collapse of the dot com bubble) dropped mortgage rates to “generationally low” levels , and financial institutions realized they had an opportunity, as government policies had allowed them to loosen underwriting standards at exactly the time that foreign investors were desperate for places to park their money — mortgage-backed securities, and their associated derivatives. More mortgages meant more mortgage-backed securities, so banks made it incredibly easy to get a mortgage, to the point that in 2006, 20% of all new mortgages were subprime . You’re probably wondering why nobody feared they’d get burned by this endless stack of different interconnected debts, and that’s because they’d “spread all that risk out” across credit default swaps with insurers, not realizing that insurers could and would become insolvent if everybody tried to make a claim at once. This was all avoidable, and there were many warnings, and just as many people lining up to protect the grift. In June 2005, Larry Kudlow would say that housing bears were “wrong again,” dismissing those concerned with increasing default rates as “bubbleheads” that “don’t do their homework.” In September 2006, financier Michael Milken would refer to CDOs in the Wall Street Journal as a “financial innovation” that “helped to spread risk and create tens of millions of jobs by freeing up investment capital for growing businesses,” saying that they would “increase prosperity by multiplying the value of human capital, social capital and real assets.”  In other words, the argument was that the “financial innovation” of ever-expanding financial speculation was good for the economy because it created more money out of thin air, with the “risk” spread out somewhere , in a way that you shouldn’t think about because everything is going to be fine. Everybody would keep building houses forever, the numbers would only ever keep increasing, every new house would add a new mortgage to a new mortgage-backed security, and the line would only ever go up. To put it all very simply, the great financial crisis was caused by inflated demand for housing caused by a mixture of historically-low interest rates and banks incentivizing bad habits as a means of increasing the value of speculative assets. In the end, “mortgage-backed securities” stopped existing as ways to invest in large swaths of mortgage payments, and more as high-risk financial vehicles that promised to be an infinite money glitch where nobody could lose because there would always be more demand for mortgages and , by extension, collateralized debt obligations made up of mortgage-backed securities. It all broke because eventually those speculative assets had to interact with the real world, by which I mean mortgage defaults began to spike starting in 2005 with the expiration of teaser rates and multiple fed rate hikes throughout 2006 making adjustable-rate mortgages creep upwards. As mortgages collapsed, CDOs — and their connected synthetic CDOs — collapsed with them, crushed by the weight of the consequences of offering so many people so many mortgages under volatile and unrealistic terms, and assuming that nothing bad would ever happen because nothing bad had happened yet. And, fundamentally, the great financial crisis was caused by massive speculation based on demand that was, in and of itself, an illusion created by the financial institution itself to justify further investment. Say, that kinda reminds me of something! I realize that the comparison between an AI data center and a CDO might seem a little ridiculous , but they’re actually remarkably similar. I’m going to generalize here, because each of these deals has weird little unique terms that make them, well, more dangerous.  Put simply, every time somebody builds a data center, they form a completely separate entity that owns the chips, owns the debt, and, in many cases, owns most of the risk. These SPVs only pay out to their creditors in the event that customer revenue flows in, which means that they are dependent both on the speed of construction of said data centers and their customers’ ability to pay. CoreWeave is the main offender in the SPV no IT loads refused cash-dump, with a different SPV for each of its Direct Draw Term Loans (DDTLs), most of them non-recourse, meaning that if their customers fail to pay, investors get screwed to varying degrees based on their seniority in the debt, and CoreWeave’s assets can’t be pursued in court, though it is on the hook for the payments on the debt. For example, CoreWeave’s $8.5 billion DDTL 4.0 loan was raised using its contract with Meta and the underlying data center assets as collateral with funding coming from banks like MUFG, Deutsche Bank, and US Bank, with funds being deposited into an SPV called CoreWeave Compute Acquisition Co VIII LLC , with another filing showing that the funding would be used to lease space from Applied Digital in Ellendale, North Dakota and fill it full of GPUs and provided to an “investment-grade customer” that Wells Fargo believes is Meta.  Similarly, CoreWeave raised its $2.6 billion DDTL 3.0 loan last year to “accelerate delivery of services from OpenAI,” funding two different SPVs called CoreWeave Compute Acquisition Co. V and VII, LLC. that, in turn, signed a deal to provide compute to OpenAI through the main CoreWeave entity. The deal also features a “cash trap” that means that if either CoreWeave screws up (IE: doesn’t deliver the compute) and OpenAI can quit or OpenAI doesn’t pay for three months, the SPV stops feeding any money to CoreWeave until the situation is cured, and if OpenAI (or someone else) doesn’t start paying, things start to break, as the deal has a contract realization ratio of .85x,. To be clear, “non-recourse” does not mean “CoreWeave gets off scot free if these SPVs collapse,” just that creditors can jump on the SPV’s assets first and cannot immediately go after CoreWeave’s assets, though because each of these deals is guaranteed by the parent company (CoreWeave itself), it will eventually be forced to make them whole. You can probably guess how that goes badly. The nature of these SPVs makes it difficult to quantify the exact scale of data center debt, but Bloomberg estimates that there’s over $500 billion in outstanding AI data center debt, with (per Garima Kapoor of Elara Securities Research) at least $200 billion of it held by private credit, making up roughly 8% of outstanding private credit loans. That being said, the number is likely much higher. Nikkei Asia reported this week that Meta, Google, Amazon, Microsoft and Oracle have accrued around $1.65 trillion in outstanding debt in the last five years, with an additional hundreds of billions of dollars’ worth of “off balance sheet” debt, meaning that the corporate structure allows the company to not include it as part of its liabilities. For example, BlackRock is currently raising $12 billion to build a data center for Meta , which in practical terms means BlackRock has invested in and is raising debt for a holding company called “Project Sopaipilla Holdings,” of which it owns 80% and Meta owns 20%. This holding company will then buy NVIDIA GPUs and pay construction firms to build the data center, and Meta’s (theoretical) payments will be used to pay down the debt. Despite the fact that Meta will (theoretically) own and operate as the exclusive tenant of this data center, the actual debt — $12 billion or more! — won’t appear on its balance sheet, much like its $27 billion Hyperion Data Center that belongs to an SPV called Beignet Investor LLC which is 80% owned by Blue Owl, 20% owned by Meta, and funded using bond sales to PIMCO and BlackRock .  The problem with these SPV-based deals is that they allow companies to, at least on a balance sheet basis, hide the scale of their debts. Meta’s long term debt sits, as of its latest quarter, at around $58.7 billion . It’s as if the $39 billion in debt for gigawatts’ worth of AI data centers doesn’t exist out of the payments it’ll eventually have to make.  This is all legal, worrying, and yes, a little bit Enron.   Per Amanda Iacone of Bloomberg : To be clear, a Variable Interest Entity is a type of SPV where you have control over the entity, and you must consolidate it into your balance sheet…unless you are not considered the “primary beneficiary,” which Meta argues isn’t the case despite being the primary tenant and reason that Hyperion is being built. Per Bloomberg: Auditor Ernst & Young raised a “red flag” ( per the WSJ ) about this arrangement, flagging it as a “critical audit matter,” adding that it “...was especially challenging due to the significant judgment required in determining the activities that most significantly affect the VIE’s economic performance.” Nevertheless, it was approved, it happened, and everything is fine and normal.  This is why Google backstopped Fluidstack and Cipher Mining’s 300MW data center and another for TeraWulf . Both will, eventually, operate as data centers that Google will lease to provide compute to Anthropic, booking revenue for doing so, acting as the sole tenant and the entire reason that the debt was raised, yet because Fluidstack and TeraWulf and Cipher Mining are the actual entities involved, nothing shows up on Google’s balance sheet.  What’s also important to note is that none of the money going into these SPVs counts as capital expenditures. For example, across the space of five quarters ( Q1 2025 through Q1 2026 ), Meta spent around $88.6 billion in capital expenditures, but that doesn’t include any of the debt or purchases of GPUs or anything else done in its name as part of the Hyperion SPV , despite it having (per its own fillings) $45.95 billion of exposure.  To be clear, even “on balance sheet” obligations are off-balance-sheet until the leases begin. Bloomberg has a truly horrifying chart that illustrates its scale: Much like the Great Financial Crisis, nobody has seen any of these data center SPVs (or the greater data center bubble) as a problem yet because  I want to be very blunt about something: we do not, at this point, have a firm hand on exactly how much demand there is for AI compute, and evidence suggests that it’s much, much smaller than we’ve been led to believe. I estimate that 70% or more of Microsoft, Google and Amazon’s compute capacity is taken up by OpenAI and Anthropic, and in my analysis of non-hyperscale compute providers , I struggled to find any customer other than them that was spending more than $50 million a year on compute.  That’s because real, diverse demand does not exist for AI compute, as evidenced by the fact that the same four or five companies are the only ones interested in renting it at scale.  For example, on July 1, Bloomberg reported that Meta ( mere months after Zuckerberg said that “selling capacity was on the table if it overbuilt”) was creating a cloud business to rent out its AI GPU capacity. A mere two weeks later, the New York Times reported that it was in talks to rent capacity to Anthropic. While one might argue that Meta is taking advantage of a wealthy buyer, one has to ask: if there was such insatiable demand for compute, why wouldn’t it want to sell it to a diverse set of customers who would likely pay a much higher rate than a years-long contract? It’s because those customers do not exist at a scale that would actually make it worthwhile! If they did, we’d see massive bursts of remaining performance obligations from neoclouds like Nebius, IREN and CoreWeave that were unrelated to new contracts they’ve signed with either hyperscalers, OpenAI or Anthropic. Companies like Lightning, Runpod, and Lambda would have billions in revenue. Instead, Runpod has $120 million in ARR , $500 million in ‘annualized’ revenue , and Lambda had $114 million in revenue as of the second quarter of 2025 , with a little less than half of that coming from Microsoft and Amazon. While the counter-argument is that these companies are all GPU-constrained, and that demand is simply waiting in the wings…except surely that would mean that these companies also had massive remaining performance obligations? To be clear, the point I’m making is not that there’s no demand, just that the vast majority of that demand is coming from either Anthropic and OpenAI — two companies that cannot afford to pay for it long-term — and hyperscalers, who are mostly buying compute on behalf of OpenAI and Anthropic. And I’m not sure that people are taking me seriously when I say that AI compute demand does not exist at the scale that it needs to, will likely never reach that scale, and data center construction is a debt-funded asset bubble with ruinous consequences. So, let’s set some table stakes. Per my own analysis, NVIDIA’s predicted $1 trillion in Blackwell and Vera Rubin GPU sales (by the end of 2027) represents around 40GW of data center capacity, which will, assuming a PUE of 1.35, result in around 30GW of usable capacity. At a cost of around $12 million a megawatt, that works out to around $435 billion in global annual compute revenue to make these data centers necessary. Right now, there appears to be roughly $100 billion or so in annual compute spend, with OpenAI representing around $50 billion ( per their statements in the Musk trial ) , and Anthropic likely spending similar amounts. Microsoft and NVIDIA represent a combined 65% of CoreWeave’s $2.08 billion in (latest) quarterly revenue , with the rest likely taken up by OpenAI. IREN, another neocloud, recently announced it was targeting a year-end cloud ARR of “over $4 billion,” or around $333 million a month, with a customer base that includes , unsurprisingly, Microsoft and NVIDIA, as well as companies like Perplexity, Figure AI, and Together AI that are unlikely to be spending more than $50 million apiece given their funding status and revenues. Another concerning anti-demand signal is the fact that NVIDIA has committed to $30 billion in multi-year cloud compute agreements , spending $6 billion or more a year through 2028 to rent back its GPUs, including a $6.3 billion backstop for CoreWeave that explicitly states that NVIDIA is “is obligated to purchase the residual unsold capacity” through April 2032, suggesting that there would be residual capacity that had gone unsold to the tune of billions of dollars. Oh, and NVIDIA owns 9.3% of Nebius too . It’s also invested in IREN , CoreWeave and has both invested in and rented capacity from Lambda .  If you’re wondering why these deals keep getting signed — as mentioned previously — it’s because a financial guarantee from NVIDIA is sufficient collateral for a bank to lend money to these companies to buy more GPUs.  I imagine a conversation in the Big Short 2 might go a little like this scene . Those Meta and Microsoft neocloud deals exist explicitly to lower their capex and debt — by which I mean that if Nebius or IREN takes on the billions in debt to buy all of those GPUs, Microsoft and Meta only have to worry about the ongoing leases, assuming that construction is ever complete. These deals also regularly include a clause that allows them to be terminated in the event that delivery milestones are not met, as is the case with Microsoft’s $17.4 billion deal with Nebius .  This means that hyperscalers take on effectively no risk, and investors are left holding the bag. For example, Nebius’ recent $775 million debt facility is “backed by contracted cashflows and deployed GPU infrastructure,” meaning that if things fall apart, the only entity that can be sued would be a company that explicitly exists to buy NVIDIA GPUs and rent them. To be abundantly clear, the vast majority of the AI data center compute revenue is contingent on the continued ability of two unprofitable, unsustainable AI companies’ to raise tens or hundreds of billions of dollars a year. This is not an overstatement, this is not hyperbole, it is the quite literal situation we’re stuck in. Putting aside whether data centers are profitable or not ( they aren’t ), if the demand does not exist at this remarkable scale, the vast majority of AI data centers and their associated SPVs will collapse.  If we take February’s Sightline Climate report at its word, there is 190GW of data center capacity in planning, or 140GW of IT load if we take a 1.35 PUE, for a total of $1.68 Trillion. If we assume — and I’m being nice! — that there’s $120 billion in annual compute demand, and take into account that tens of billions of dollars’ worth of data centers have been announced since, this means that there’s over 15 times more data centers being planned than the demand that actually exists, and 70% to 90% of that demand is from Anthropic and OpenAI’s unprofitable services. The hunger for speculation has vastly outpaced the actual demand for AI compute, much like it did in the great financial crisis, and for many of the same reasons. Back in May , JP Morgan’s Karen Ward brought up the global savings glut that I mentioned in the intro as part of a discussion of what she calls a “global savings grab”: Well, good thing that the world is different now, right?  The difference between a savings glut and savings grab is that there’s incredible demand for cash rather than an excess of capital to invest , at a time when banks (and private credit funds ) have tons of cash but are pulling back from investing in software and healthcare companies due to AI-related risk … and investing in AI data centers, which they consider to be the “cheat code” — high-yield, low-risk investments in infrastructure that have “guaranteed” customers.  It’s a perfect storm that mixes dangerously with the $400 billion or so in private infrastructure funds waiting to deploy , much of which is funded by pension and insurance funds ( as I covered in the Hater’s Guide To Private Credit ) drawn to private credit — get this! — because they needed new things to invest in after the Great Financial Crisis made yields difficult to find because banks were restricted from making the same kind of reckless bets that caused the global financial system to implode. Are you beginning to work out why I’m a little concerned? How about the fact that the amount of dry powder within these retail-focused financial institutions is shrinking — suggesting that more and more cash is being deployed, or withdrawn as a result of diminishing confidence within households .  Anyway, much of the assumption of how “safe” investments in AI data centers comes down to three ideas: Financial institutions have built entire models based on logic that borders on childish.  The “proof” that it’s worth investing in data centers mostly comes down to seeing that hyperscalers are spending a lot of money on them, and that OpenAI and Anthropic have lots of demand for compute.  They have also mistaken the ability for hyperscalers to keep funding data centers out of cashflow as a sign that all data centers are a good investment , when what’s actually happening is that they’ve run out of hypergrowth ideas and had so much free cash sloshing about that they were able to spend a trillion dollars in four years. Hyperscaler demand for NVIDIA chips has been so significant that it made it look like NVIDIA had an insane amount of demand, which in turn created a degree of FOMO and speculation, with everybody assuming that because hyperscalers were getting rich (they weren’t, they have never disclosed their AI revenues, but people just assume they wouldn’t do this without making a profit) that they too would get rich by buying GPUs and building data centers. NVIDIA has been a big part of creating this fake demand story with its investments in — and backstop contracts with — CoreWeave, Lambda, IREN, and Nebius. Much like people assume hyperscalers wouldn’t make a huge, trillion-dollar mistake, they also assume that NVIDIA wouldn’t invest in companies that weren’t going to see incredible demand, somehow ignoring the very obvious point that NVIDIA doesn’t give a shit about any neoclouds outside of their ability to generate more GPU sales.  This is where the media and analysts could’ve done their jobs, but because none of the neoclouds have done yet, it’s totally fine that CoreWeave is sat on $30 billion in debt, most of it impossible to pay if any client drops out of a contract, because, much like the great financial crisis, nothing bad had happened yet, by which I mean that clients leave their invoices unpaid, and CoreWeave finds itself in financial distress. NVIDIA’s naked self-dealing and circular financing are only made possible with a completely captured tech and business media. While mildly-concerned stories have run for the last year or so about the “massive circular financing under the AI bubble,” none of them treat the situation as anything else other than a curiosity.  I cannot adequately express my contempt for those that have hand-waved the danger of this bubble, or tried to minimize the risk created by the overbuild of AI data centers.  Much like a subprime mortgage, AI data center debt is being poorly-underwritten, virtually-uncollateralized and issued to projects that have extremely low likelihoods of repayment, all based on flimsy information and hype-driven mania.  Their collapse is inevitable because their ongoing payments are made out of customer revenue that is, in the vast majority of cases, entirely theoretical or contingent on payments from unprofitable and unsustainable AI companies.  What differs this from the subprime mortgage crisis is that the systemic risks aren’t driven by derivatives or complex financials but by the sheer scale of costs to build an AI data center, a catastrophic misunderstanding of the AI industry itself and the dangerous lending standards of private credit. When every single debt deal is over $500 million and usually numbering in the billions , we don’t need a vast web of different contracts to create a systemic risk, just clusters of projects that either fail to keep up with their SPVs’ debt or bonds that go unpaid by destitute or defunct data center developers. Also, please remember that it didn’t take massive losses to begin the great financial crisis — just hard hits to a few load-bearing pillars of the industry. Lehman Brothers suffered two sequential quarters of losses ($2.8bn in Q2 CY2008 and $3.9bn in Q3 CY2028) before it entered liquidation. Those losses weren’t what killed it, but rather, what those losses did to the broader market — as well as the perception of Lehman with potential saviors.  Similarly, Bear Sterns failed after two hedge funds under its umbrella collapsed . While the monetary losses from these funds weren’t insignificant, they were something that, if everything else was fine, Bear Sterns could recover from them. Sadly, they occurred at a time when the market was spooked, and Bear was spectacularly over-leveraged, meaning that marginal losses would have a disproportionate impact on its balance sheet.  For AI, those “hard hits” to the “load bearing pillars of the industry”  means a large amount of capacity flooding the market at once — such as that caused by the failure of a major compute customer, most likely OpenAI — or the slow arrival of new capacity that can’t find revenue to pay for it. Perhaps we get both.  While the data center debt market might be much smaller than the trillions of dollars of (at least theoretical) securities that broke the back of the financial markets (I estimate somewhere between $500 billion and $750 billion), the risk — the actual underlying financing — is spread across the entire financial system, with every major bank and financial institution and the vast majority of asset management firms having billions or tens of billions of dollars’ worth of debt tied up in an impossible situation.  The scenario I’m talking about is one where the vast majority of AI data centers go unused, and because the vast majority of data centers are paid out of customer revenues, 80% or more of the funds invested in AI data center debt will be lost. None of this is written to be alarmist or hyperbolic, and represents a rational position when compared to the fact that we have over 15 times the amount of data center capacity than we need, and there is little compelling evidence that there’s more than a few billion dollars in total demand.  This will mean that effectively every single financial institution in the world will have to write off or mark down hundreds of millions or billions of dollars’ worth of loans — and when they go to sell the underlying assets, they’ll be dumping aging hopper and Blackwell GPUs into a market saturated with them, meaning that the salvage price they get — assuming they get one at all — will be negligible. This is the data center equivalent of subprime loans defaulting, except instead of hundreds of thousands of loans being the trigger, all it takes is ten or fifteen of them to send the industry into a panic.  It’s easy to dismiss this entirely as “rich people problems,” but AI data centers are increasingly funded — both directly and through private credit — using pension and insurance funds that rely on these (theoretical) payments for future yield to pay out premiums.  I’ll give you some examples. The other problem with the “private” part of private credit is that we don’t really know how much data center exposure pension and insurance funds and the insurance/retirement funds of asset managers actually have. What we do know is that private credit is sinking hundreds of billions of dollars of people’s retirements and insurance premiums into deals based on obfuscated valuations and questionable underwriting standards .  For an example of how lax those standards are, here’s a quote from The Information about Blue Owl’s due diligence on Stargate Abilene, emphasis mine: To make matters worse, Moody’s estimated a few months ago that banks had around $1.4 trillion in exposure to private credit, with $300 billion of that exposure held by big banks . And because neither banks nor private credit funds nor asset managers are forced to keep any level of reserves, all it takes is a few bad apples — a few billion of data center deals — to go pear-shaped for there to be a cataclysmic unwinding of the AI trade.  So, the reason that nobody is really worrying about this situation is that we’re still waiting for the vast majority of data centers funded so far to complete construction. Once that happens, the assumption is that either A) the client in question will start paying or, more likely, B) that the data center provider will simply expect said customer to appear.  Eventually, these data centers ( which are taking 18-36 months to complete ) will start turning on, which will require them to start having paying customers at a scale that the market can’t actually support. While subprime mortgage defaults were a kind of slow, ugly boil, it’s much more likely that the collapse of the subprime data center bubble will happen in fits and starts as capacity comes online and, assuming Anthropic and OpenAI don’t swoop in, goes unused.  I think we’ll see a few rescue missions to try and keep the con alive. Hyperscalers will do everything they can, scooping up capacity anywhere they see it, to avoid the perception that AI data centers will go unused. You see, hyperscalers are currently in their own confidence game as a result of their ruinous expenditures creating the illusion of demand. Microsoft, Google, Meta and Amazon are stuck in a terrible situation where building more capacity will cost them tens of billions of dollars, but stopping building capacity will be an immediate signal that they’ve overbuilt capacity, sending anxiety-strewn shockwaves through the industry and killing data center debt issuance.  Yet what also might kill issuance is the market itself. Per Bloomberg , AI data center debt has “hit a wall,” with 80% of data center securities issued since early 2025 quoted at a wider spread than issuance, meaning that investors are valuing them as worth less than when they were initially issued. If the market continues to sour on AI data center debt, it will eventually become difficult to impossible for hyperscalers to keep issuing bonds, leaving them with only equity sales ( like Google’s $85bn stock sale ) that are equal parts limited and desperate. Any decline in appetite for AI bonds will be immediately obvious, given the scale in borrowing, with (per Goldman Sachs) AI-related bonds accounting for nearly one-quarter of all US-investment grade debt issuance .  NVIDIA’s continual circular funding of neoclouds and anyone who wants to buy NVIDIA GPUs continues only as a marketing function, and an attempt to conjure up the illusion of insatiable demand for AI compute. These deals are acts of desperation themselves, and tacit admissions that without NVIDIA, none of these neoclouds would exist — though, to be clear, Jensen Huang has quite literally said this on camera . This, in my view, represents another troubling parallel between the AI bubble and the subprime mortgage crisis. In the early 2000s, loose lending standards — combined with the securitization of mortgages, which allowed lenders to offload their risk to third-parties — made it possible for people who shouldn’t have been able to obtain a mortgage to buy a home, albeit often at worse terms than so-called “prime” borrowers.  As I outlined in Coreweave Is A Time Bomb , Coreweave has been able to borrow tens of billions of dollars, despite having a business that is fundamentally reliant on a single customer — OpenAI — and on terms that would make a mafioso loan shark pause and say “hold on, that’s a bit harsh.”  In a sane world, Coreweave should not have been able to borrow as much as it did — and the same applies to the countless other debt-laden neoclouds that are, for the most part, cookie-cutter versions of Coreweave. These are the subprime homebuyers of the AI bubble, and the backstops and freebies offered by NVIDIA (and the other hyperscalers) has allowed said companies to raise more debt, without actually changing the fundamentals of these companies that made them so inherently risky to begin with.  Another obvious trigger is the insolvency of OpenAI or Anthropic, who have $1.1 trillion in compute commitments across Microsoft, Google, Amazon and Oracle , and, more dangerously, another $50 billion across Cerebras and CoreWeave.  In fact, maybe insolvency is going too far. CoreWeave’s own master service agreement with OpenAI will breach investor covenants if OpenAI fails to pay for three months straight, and with OpenAI delaying its IPO to 2027 , it’s going to either need to raise more money or not pay its bills. In any case, the sheer scale of AI data centers coming online massively outpaces the demand for AI compute, and to be clear, the AI bubble doesn’t have to burst for the subprime data center crisis to begin, because all it takes is for the revenue to not exist to pay for the compute.  As the vast majority of AI data center debt is project financing funded by compute revenues, the cutesy and half-assed retort of “even if it’s an overbuild, it’ll be alright” doesn’t really matter very much. The second a debt-backed data center is built, it must immediately produce revenue, because otherwise creditors will be left unpaid. While the sheer scale of who those unpaid creditors might be is hard to quantify, what we can quantify is that the risk of the subprime data center crisis is everywhere — your bank, your community, your pension fund, your insurance company, everyone has, on some level, some exposure to the bubble.  For me to be wrong, there will have to be dramatic amounts of AI compute demand — hundreds of billions’ worth — within the next 3 years, at a time when there’s little more than $120 billion, with 80% or more of that coming from two companies that can only afford it because they have near-infinite sums of venture capital behind them.  Oh, and for some context, the entire global software market is estimated to be around $779 billion in 2026 . It’s unclear where that money will come from, and nobody seems to want to talk about it. The AI bubble is, like the great financial crisis, a product of information asymmetry — companies intentionally obfuscating how much revenue they have, how much data center capacity they have, how much revenue that capacity generates, how much demand they have for their services, and how many real dollars actually flow from the AI industry outside of investments in semiconductors. And much like the great financial crisis, modern tech and business journalism routinely defaulted on its responsibility to demand this information, or to present a lack of information as suspicious , choosing instead to fill in the gaps and assume that whatever bullshit a rich person peddles is the truth. While many media outlets — as they did in 2007 — are now trying to somewhat quantify the risks, most business publications continue to celebrate every time NVIDIA sinks billions of dollars into a neocloud that exists only as a means of selling more GPUs , act as if AI’s continued growth is a certainty, and bring up financials only as a hesitant warning about an otherwise-solvent and successful industry. Similarly, entire research groups — regularly quoted by the media — exist to inflate the bubble further. Exponential View’s speciously-sourced and questionably-founded research paper on AI revenues was used by Bloomberg and multiple other media outlets as a means of saying that “AI had started to pay off” because “the quarterly revenues were now outpacing depreciation,” a completely nonsensical statement that compared revenues from the entire AI industry (unspecified and undefined, by the way) with the depreciation of GPU hardware by hyperscalers.  It’s hard to see this as anything other than some tech and business journalists having a vested interest in seeing the AI industry win, which means, by proxy, that some writers will be fundamentally responsible for what follows when the bubble bursts. This is not a deliberately  hyperbolic statement (and, lest I be accused of tarring the media and analysis industry with the same broad brush, there are many exceptions, and many good reporters calling bullshit where justified), but it seems as though there is a concerted effort to support industry narratives that I find repulsive. What’s most horrifying is that OpenAI and Anthropic don’t even have to die for this all to end horribly. For one company to be able to afford even $200 billion a year in AI-related operating expenses is ludicrous.  Microsoft, a company that makes about $318 billion a year in revenue , has about $169 billion of operating expenses a year , and that includes the cost of running OpenAI’s compute. The idea that we’re going to have multiple Microsofts-worth of opex entirely focused on AI compute in the next four years is absolutely fucking ridiculous, yet it’s one of the most commonly-held beliefs in the tech industry.  As I hope I’ve made clear, I believe the vast majority of AI data centers are the AI bubble’s subprime loans, and will collapse when they face the cold, harsh reality of “someone actually paying money for AI compute,” much like subprime mortgages collapsed when teaser rates ended and homeowners were forced to pay their actual bills.  This is an inevitability — and something that’s very obvious when you sit down and actually try and work out how much capacity there is versus how much people are actually paying for AI compute. The fact that I, a random guy, albeit with a (recently-acquired) Bloomberg Terminal, am the one to say this is a sign that the media is not trying hard enough to protect consumers. Every time that the media has accepted a spurious announcement or a questionable run rate or a circular deal or the outright refusal of hyperscalers to disclose their AI revenues, they help inflate the AI bubble and endanger the futures of millions of people, especially those tied up in a stock market increasingly-dominated by NVIDIA and other tech stocks. Unlike the great financial crisis, the calamity to follow will be easily-traced to a complete failure of anybody to measure or demand measurements of the actual demand for AI services and AI compute.  There will be attempts to claim it was too complex or multi-faceted to pry apart, and those attempts will likely be made by media outlets that failed their readers, viewers, listeners, and the general public. Bubbles can only inflate in an information-poor — and information-deprived — environment.  They inflate much faster and more-dangerously when that information is poisoned by marketing spiel and misinformation peddled by those who are meant to tell people the truth. If you liked this piece, you should subscribe to my premium newsletter. It’s $70 a year, or $7 a month, and in return you get a weekly newsletter that’s usually anywhere from 10,000 to 18,000 words, including vast, detailed analyses of the biggest events and companies in the AI bubble.  As a reminder, if you sign up between now and 12AM ET, July 26, you’ll get $10 off a subscription. Click here for the offer . When somebody decides to build an AI data center, they form a special purpose vehicle (much like a CDO), which then raises debt, in some cases slices it into tranches and, in most cases, sells them to institutional investors, asset managers or banks.  Think of the SPV as its own little company (owned by the holding company, CoreWeave for example), and when somebody signs a contract with an AI data center company (say, OpenAI), they actually are signing a deal with the SPV rather than the company itself.  When the SPV receives the funds from the debt raise, it makes payments to contractors and suppliers (EG: NVIDIA for GPUs), and receives the revenue from the customer contract, assuming said customer is paying (or has anything to pay for). During construction (IE: pre-revenue), interest payments are taken out of the SPV from a pre-funded interest reserve account. When a customer pays, the SPV uses those funds to pay for the operating expenses of the data center, then creditors (based on their seniority in the debt), then, if anything’s left, the holding company. All of this money counts as revenue. These SPV-based data center debt deals also have a few fun little features: A DSCR (Debt Service Coverage Ratio) which means that the SPV must bring in a certain amount of EBITDA income compared to its debt. For example, if an SPV’s debt had a DSCR of 1.15x and a monthly payment of $1.5 million, it needs to bring in $1.725 million in revenue after paying its operating expenses. These often don’t begin until a date when the data center is theoretically operational, and yes, this absolutely could go horribly wrong with the amount of delays there are. A minimum liquidity requirement that, when breached, requires the holder to refill it or face default. A Debt Service Reserve Account (DSRA) set up after construction as a buffer if payments fall through. AI data center demand is infinite and all compute will be used. AI data centers all have “locked-in customer demand.” This is, to be clear, fundamentally untrue. The only guaranteed, locked-in customer demand I can find is from Amazon, Google and Microsoft (for OpenAI and Anthropic), Meta, and Google and Anthropic. While there might be some random AI firms or inference companies that have “locked up capacity,” their dollars are only as good as their access to venture capital, much like Anthropic and OpenAI. That these are “safe” investments, backed by the richest companies in the world. This idea comes from the child-like belief that because Microsoft, Google, Meta and Amazon are the richest companies in the world and are signing 10-to-20-year-long leases, that tons of other companies will do the same, and that AI data centers and their debt should be valued as such. Australian AI infrastructure company Morrison, backers of CDC (the largest data center operator in the country), recently convinced Japanese bank SMBC to allow it to invest its pension funds in AI data centers in the country . IPI Partners, a one of the largest private data center investment firms that is now owned by Blue Owl , has a limited partner (read: people funding it) base, per Deutsche Bank, split into equal 25% chunks made up of sovereign wealth funds, family offices, public pensions, and insurance/private pension endowments.  The California State Teacher’s Retirement System is the biggest investor in Blue Owl’s publicly-traded Blue Owl Capital Corporation fund.  Blue Owl funded Meta’s Hyperion data center and Stargate Abilene, amongst other deals. In 2024, Blue Owl acquired insurer Kuvare , which won the Great Des Moines Partnership’s “Deal Of The Decade” Award in 2023 for funding Meta’s Altoona-based data center, which was, at the time, Meta’s largest data center.  CDPQ, one of Quebec’s largest pension funds, invested in CoreWeave’s $7.5 billion DDTL 1.0, as part of its CDPQ American Fixed Income V Inc fund . A few weeks ago, Asset manager Apollo Global raised $35 billion for Broadcom to build Google TPUs for Anthropic to lease , and did so funded by billions of dollars of insurance annuities it’s able to play with as a result of its acquisition/merger with insurance and retirement firm Athene . If these payments aren’t made (though Broadcom has backstopped them), it will directly hit Athene’s ability to pay out insurance and retirement premiums. One worrying quote from the piece, emphasis mine: “What also sets Apollo apart is its homegrown trading operation, further blurring the lines between the alternative asset manager and Wall Street banks. It has also become one of the largest forces in insurance, prompting concerns that the firm and its peers are ramping up risk in a once-sleepy part of finance, and at a pace that makes it difficult for regulators to keep up .”

0 views
<antirez> 2 days ago

Not just development, distribution of software may change as well

Even if you are as averse to semver as I used to be in the course of my programming activity, you can still think of open source software distribution as something that used to follow a fixed number of steps. There is a branch where developments happen, and this branch oftentimes happens to be not really ready for reliable work. Then you freeze the developments for a certain amount of time (even if, in the meantime, the work can continue on some new unstable branch), fix bugs, ask people to test it. At some point the number of bug reports starts to drop, your team and your users start to believe there are no longer obvious critical flaws that are easy to discover in the next few weeks: then you call the branch 2.4 or whatever, and that's it. However now, with AI coding, it's not just development that has changed, but also the act itself of using software is affected: it is not just you that can ask an AI to do certain changes to the software, but also the recipient of the software itself. This is obvious in the domains where a piece of software has its main user base among programmers, but this is also true in general, as more and more technologically inclined users have AI access and coding agents. Because of this change, the idea of just having a stable branch with everything polished, and an unstable branch where everything is a work in progress, may no longer be the right way to do things. A code repository can also be a finished product, but could be even more useful if it is a template for how to do things around a given problem. Maybe the user will modify the code in order to specialize it for a specific set of requirements, hardware, specific problems to solve. Also, what is too unstable or unproven for the general public may be the right thing for another set of users. Take the example of Redis. For weeks now I have been iterating on a PR that provides strong memory savings for sorted sets. This work, if accepted, will hit every user of Redis, from people that don't have any idea about how Redis works, to users that maybe even contributed code in the course of years. From use cases that are trivial to use cases where a 50% memory saving on sorted sets could mean cutting a big slice of the cloud bill every year. For this last kind of user, having the final product (after all the testing and changes of design I'm doing to refine something that "just works", with the risk that maybe it will not even enter the code base) may be less interesting than having a 95%-ready branch since day zero. It is code they can test, adapt, iterate on, even specialize more for the problem at hand. Maybe DwarfStar is an even more telling example of how code repositories should be good examples more than finished products covering every piece of the features matrix. With local inference you have, in the specific case of DwarfStar, many kinds of GPUs, models, server mode, agent mode, CLI, SSD streaming, tensor and pipeline distributed execution. To test everything everywhere is complicated. Yet, once you have two solid examples of tensor parallel graph execution, a strong coding agent can infer how to implement the same thing for other backend/model pairs. Similarly, once you have an engine that supports two models well enough, a third can be implemented in an almost automatic way, using the existing code base as a guardrail for coding agents in order to guide the implementation. This does not mean that a project like DwarfStar should not work out of the box, but that it could focus on supporting very well a set of features that can be extrapolated to a larger amount of possible situations that the users can cover themselves. It also means another thing: that main and unstable are no longer enough. Many experimental branches could be an integral part of the project. For instance, yesterday the Laguna S.1 model was released. It looks interesting on paper, however: will it really be good enough? Will the new DeepSeek v4 Flash checkpoints make it not really relevant for DwarfStar? It is too early to say. However, to collectively form an idea, publishing a branch with this model implementation is a good middle ground: people will try it, will refine it with their coding agents, and the community can collectively form an idea about how merge-worthy it is. Moreover, today I noticed how, thanks to the rails formed by the corpus of the code inside DwarfStar, the implementation was written in about two hours by GPT 5.6 Sol automatically. Implementing DS4 and GLM5.2 cost me a lot of steering, reading the model card and the details of the implementation of the attention of those models. Now it just worked. GPT 5.6 is more powerful but it also found a lot of good examples inside the existing source code. Software today is more malleable than ever. In some way this means that it can be released in a more fluid way. Also, it means that the documentation itself should not be just good for humans, but also for coding agents to understand how to change the system. How this will evolve exactly, and what the right point of balance between the different dimensions of stability, usability, and features will be, is not clear to me, but I believe we developers need to keep our eyes open to see where all this is headed. Comments

0 views

Paper Things I Use

Read on the website: I have decided to go low-tech with my life lately. Here’s the low-tech know-how I use.

0 views
Jeff Geerling 2 days ago

Open Sauce and GPS time were my summer AI Antiseptics

In the midst of our AI slop revolution, traveling to the West coast for Open Sauce this past weekend was the perfect antiseptic for rising costs, summer heat, and online divisiveness. It's ironic, then, that I used Claude to vibe code my Tufty GPS Time Badge . Partly due to time constraints, and partly because I wanted to see if I could complete a personal project end-to-end, without editing a line of code, I throw my requirements at Claude and ultimately came up with this 2141-line MicroPython app for Pimoroni's Badgeware ecosystem.

0 views