Latest Posts (20 found)
Xe Iaso -2 days ago

Giving your Go apps Tigris superpowers

Tigris is S3-compatible, which means you can point the AWS SDK at it and most things just work. The catch is that the Tigris-exclusive features—bucket forking, snapshots, object renaming, and the like—need verbose workarounds because the AWS SDK doesn't know they exist. So we wrote a Go SDK that does. It comes in two flavors: the package is a drop-in replacement for the standard S3 client with first-class methods for the Tigris-specific operations, and is a higher-level client for the common single-bucket case that infers its configuration from the environment so you stop passing the same parameters over and over. You can adopt the Tigris features incrementally without refactoring your existing S3 code, and the simpler API still works against other S3-compatible providers. I wrote up how it works and why we built it over on the Tigris blog.

0 views

Running Python code in a sandbox with MicroPython and WASM

I've been experimenting with different approaches to running code in a sandbox for several years now, but my latest attempt feels like it might finally have all of the characteristics I've been looking for. I've released it as an alpha package called micropython-wasm , and I'm using it for a code execution sandbox plugin for Datasette Agent called datasette-agent-micropython . My key open source projects - Datasette , LLM , even sqlite-utils - all support plugins. I absolutely love plugins as a mechanism for extending software. A carefully designed plugin system reduces the risk involved in trying new things to almost nothing - even the wildest ideas won't leave a lasting influence on the core application itself. My software can grow a new feature overnight and I don't even have to review a pull request! There's one major drawback: my plugin systems all use Python and Pluggy , and plugin code executes with full privileges within my applications. A buggy or malicious plugin could break everything or leak private data. I'd love to be able to run plugin-style code in an environment where it is unable to read unapproved files, connect to a network, or generally operate in a way that's risky or harmful to the rest of the application or the user's computer. My interest covers more than just plugins. For Datasette in particular there are many features I'd like to support where arbitrary code execution would be useful. I've already experimented with this for Datasette Enrichments , where code can be used to transform values stored in a table. I'd love to build a mechanism where you can run code on a schedule that fetches JSON from an approved location, runs a tiny bit of code to reformat it into a list of dictionaries, then inserts those as rows in a SQLite database table. My goal is to execute code safely within my own Python applications. Here's what I need: Web browsers operate in the most hostile environment imaginable when it comes to malicious code. Their job is to download and execute untrusted code from the web on almost every page load. Given this, JavaScript engines should be excellent candidates for sandboxes. Sadly those engines are also extremely complicated, and are not designed for easy embedding in other projects. Most of the v8-in-Python projects I've seen are infrequently maintained and come with warnings not to use them with completely untrusted code. WebAssembly is a much better candidate. It was designed from the start to support all of the characteristics I care about and has been tested in browsers for nearly a decade. The wasmtime Python library is actively maintained and has binary wheels. WebAssembly engines like wasmtime run WebAssembly binaries. Some programming languages like Rust are easy to compile directly to WebAssembly. Dynamic languages like JavaScript and Python are harder - they support language primitives like , which means they need a full interpreter available at runtime. To run Python we need a full Python interpreter compiled to WebAssembly, wired up in a way that makes it easy to feed it code, hook up host functions and access the results. Pyodide offers an outstanding package for running Python using WebAssembly in the browser, but using Pyodide in server-side Python isn't supported. The most recent advice I could find was from October 2024 stating "Pyodide is built by the Emscripten toolchain and can only run in a browser or Node.js". The other day I decided to take a look at MicroPython as an option for this. The MicroPython site says: MicroPython is a lean and efficient implementation of the Python 3 programming language that includes a small subset of the Python standard library and is optimised to run on microcontrollers and in constrained environments. WebAssembly sure feels like a constrained environment to me! I had GPT-5.5 Pro do some research for me , which turned up this PR against MicroPython by Yamamoto Takahashi titled "Experimental WASI support for ports/unix". It then produced this research.md document , so I let Codex Desktop and GPT-5.5 high loose on it to see what would happen: It worked. I now had a prototype Python library that could execute Python code inside a WebAssembly sandbox! The trickiest piece to solve was persistent interpreter state. The WASM build we are using here exposes a single entry point which starts the interpreter, runs the code and then stops the interpreter at the end. This works fine for one-off scripts, but for Datasette Agent I want variables and functions to stay resident in memory so I can reuse them across multiple code execution calls. A neat thing about working with coding agents is that you can get from an idea to a proof of concept quickly. I prompted: After some iteration we got to a version of this that works! In Python code you can now do this: Under the hood this starts a thread, sets up a request queue and then sends messages to that queue for the command, each time waiting on a reply queue for the result of that execution. Inside WASM the MicroPython interpreter blocks waiting for a host function to return the next line of code, which it runs on before calling when each block has been successfully executed. The other piece of complexity was supporting host functions, so my Python library could selectively expose functions that could then be called by code running in MicroPython. Codex ended up solving this with 78 lines of C , which ends up compiled into the 362KB WebAssembly blob I'm distributing with the package. I am by no means a C programmer, but I've read the C and had two different models explain it to me (here's Claude's explanation ) and I've subjected it to a barrage of tests. The great thing about working with WebAssembly is that if the C turns out to be fatally flawed the worst that can happen is the WebAssembly execution will fail with an exception. I can live with that risk. Memory limits are directly supported by wasmtime. CPU limits are a little harder: wasmtime offers a "fuel" concept to limit how many operations a WebAssembly call can execute, and that's the correct fit for this problem, but the units are hard to reason about. I'm experimenting with a 20 million default "fuel" setting now but I'm not confident that it's the most appropriate value. The alpha is now live on PyPI . You can try it from your own Python code as described in the README . I've also added a simple CLI mode in version 0.1a2 which means you can try it using without first installing it like so: You can also try it in Datasette Agent like this: Then navigate to http://127.0.0.1:8001/-/agent and run the prompt: Having complained about immature, loosely-maintained sandboxing libraries, it's deeply ironic that I've now built my own! I deliberately slapped an alpha release version on it, and I'm not ready to recommend it to anyone who isn't willing to take a significant risk. I've put it through enough testing that I'm OK using it myself. I've shipped my first plugin that uses it, datasette-agent-micropython . I've also locked GPT-5.5 xhigh in that Datasette Agent plugin and challenged it to break out of the sandbox and so far it has not managed to. I'm hoping this implementation can convince some companies with professional security teams and high-stakes problems to commit to using Python in WebAssembly as a sandboxing approach and open source their own solutions. You are only seeing the long-form articles from my blog. Subscribe to /atom/everything/ to get all of my posts, or take a look at my other subscription options . Why do I want a sandbox? What I want from a sandbox WebAssembly looks really promising here MicroPython in WebAssembly Building the first version Try it yourself Should you trust my vibe-coded sandbox? Dependencies that cleanly install from PyPI , including binary wheels across multiple platforms if necessary. I don't want people using my software to have to take any extra steps beyond directly installing my Python package. Executed code must be subject to both memory and CPU limits. I don't want to crash my application or the user's computer. File access must be strictly controlled . Either no filesystem access at all or I get to define exactly which files can be read and which files can be written to. Network access is controlled as well . Sandboxed code should not be able to communicate with anything without going through a layer I fully control. Support for interaction with host functions . A sandbox isn't much use if I can't carefully expose selected platform features to the code that it's running. It has to be robust, supported, and clearly documented . I've lost count of the number of sandbox projects I've seen in repos with warnings that they aren't actively maintained!

0 views
iDiallo Today

Why all the PRs?

It's a signal. That's why we get AI-generated PRs. We told everyone, in order to get your resume taken seriously, you need to show your work. When I was getting started in my career, that meant having your own website that you contribute to regularly. So I did that. I built websites, I maintained them. I kept maintaining them even after I got the jobs because that's how I actually honed my web programming skills. Where else was I going to try new frameworks, a new JavaScript paradigm, or try out Ruby on rails? I got the job, and I advised other developers to follow the same path. But then github became mainstream. Rather than just show a finished website, you could actually share the code that runs your project. Share a link to your github project and companies can review your code and directly gauge your experience. But even better, you can show your contribution to open source projects. Not just any projects. Popular projects. The github stars became a metric people look for. A signal that can be used to quickly assign a value to a candidate. But that’s the story told from the outside. I don’t think the github profile link was ever important, unless it was significantly good. Employees focused on their work rarely have the time to maintain healthy github activity. Their experience comes from their day to day job. So for the most part, not much attention was placed on github links other than skimming through those surface level details. When stacks of resumes came on my desk, the best candidates stood out because they had work experience. The good candidates had projects that they could link to, github or elsewhere. But then, the worst candidates had long padded resumes that had elements of every job application tips-and-tricks-article. They had a website, but it was built in a day for the purpose of getting a job, with nothing interesting to say. They had github links, but those often pointed to school projects, homework, or boilerplate code. That’s the vast majority of github links I used to get. People with active and well maintained github profiles were rare. Rare because it actually requires time, effort, and experience. But then we have AI. There was a golang auth issue that I've contributed to on github. It was already a few years old when I proposed a solution that worked for my case. It wasn't universal so it wasn't accepted. The discussion is revived every couple years, each person bringing one more piece to the puzzle. But then recently, someone exploded the thread with comments. And even created a PR to go with it. This was from a user that went from a dormant account to 4000 contributions in a year. It was all AI assisted code. This isn’t to comment on the quality of his code, but he was clearly trying to optimize the metric. Looking at his linkedin profile, he doesn’t work in a software engineering role, and it’s hard to decide if he would be a good contributor if hired. If we were to judge his resume by looking at the github profile, it might catch our attention. But then, there is a problem. There are hundreds, even thousands of people all doing the same thing. They are cranking up their contributions to github projects using AI, so they can have a better chance at getting hired as developers. I understand the job market is rough right now, especially for gen z , and anything to differentiate yourself is a plus. The problem is this is being done at the expense of open source projects. The contributors are not submitting PRs to your project because they are personally invested in it. Instead, they are trying to get their name on the contributors list so that they can use it as a signal in their resume. When we are out here debating if there is any merit in AI generated PRs, or if we should just judge the code, we tend to miss that their gesture is completely hollow. The PR’s author intentions are completely misaligned with the project's maintainers. They are playing a different game. We call it slop, or a waste of time, we ban them and they get really vocal about expressing their first amendment rights. We are directly interfering with their goal of padding their resume. I often ask, why don’t people who create those PRs not just start their own project? One answer I’m starting to believe is, nobody cares about a github profile with a handful of stars. You need to contribute to a popular project. Most if not all AI generated websites look the same, it doesn’t matter how well you customize the prompt. Most greenfield projects from new programmers look the same, the prompter lacks the experience to do anything different. Contributing to open source is a scary thing when you are new. Even when you have experience, it’s a deliberate act. You have to be invested in the work. Just like asking questions on stackoverflow, issues you raised will often get closed . And when they do, you have to learn from it. The value of an open source contributor is not in the volume of work they can perform. If you skim any important projects, you’ll see that the best contributors spend more time discussing the problem than writing code. Their value is in solving problems and contributing to the collective memory of the group. But when you are doing a drive-by PR that may or may not be correct, and you are just trying to get your name on a list, you are providing zero value to the maintainer. Just more work. This is the signal every slop PR generator is after.

0 views

Sprinter Van Phone Mount + Better CarPlay-Compatible Cable Situation

This is the stock look of my 2021 Sprinter van front console area: If it’s not obvious, there’s no great place for a phone. You’ve got the cup holders. Those are actually sorta workable, except for this boss battle: wired-only CarPlay. Wired CarPlay is not ideal, but that’s all this van has. I expect upgrading the whole system would be super expensive or potentially not even possible. Wired isn’t that bad. Wired CarPlay means more immediate response from actions and heck, it charges the phone too. The problem is where that wire needs to go. Up on the dashboard, there is this little cabinet thing with a door that opens up toward the windshield. The ports are inside that cabinet, one of them being the one that has to be used for CarPlay. I’m sure the engineering thought is: plug it in, put your phone in the cabinet, shut the cabinet. And that’s kinda fine. I don’t like fiddling with my phone while driving and CarPlay means I don’t really need to. But it’s still inconvenient. I often forget my phone up there when getting out of the van. If I do need to fiddle with my phone while parked or because something just absolutely has to be done on-device, it’s extra obnoxious to get my hands on it. The answer is this little guy. A 3D printed part from NEXUS. This slides into the cabinet and now the cabinet door doesn’t full close, it closes onto this, leaving little slits to bring the cords out from. We’re ultimately going to move the phone to a mount, and then the question is where and how to mount it. Fortunately NEXUS is on the case again with a 1″ Ball Cubby Adapter . That weird looking thing has nothing at all to do with your butt! It’s a clever device that fits perfectly into the useless weird cubby on the dashboard and provides this general purpose mount. NEXUS doesn’t make a phone mount. I think on purpose? The Winnebago Revel has all these RAM ®  Mounts all over. I had never heard of all this RAM ® stuff before. They make a bunch of mounts and stuff. To me, it all feels like a little step up from the home 3D printing feel of NEXUS. Not quite industrial, not quite hobbiest. I was happy to stay in the kinda happy path ecosystem, so I bought the RAM ®  Mount with the ball. Astute readers will notice… now we have two balls. And, we have two ball mounts! Those two ball mounts won’t actually connect to each other. The magic of the ball mount is that you get this 360-degree(ish) adjustability. I don’t really need that much movability, but I’ll take it. The final bit here is to get the connector to get the two balls together. And it doesn’t occupy any other useful space on the dashboard. Love it. Order list: Kinda pricey for a phone mount. But I think it’s worth it. Its a slightly tricky situation and this solution is (1) long term in that it will fit any future phone (2) opens up the idea for mounting other things with ball mounts (3) allows for more cables out of the dashboard cabinet to come out smoothly. There are other options! There are other/cheaper ones that clip onto the cubby mount that look OK. There are ones that mount into the cup holder nicely. Honestly this one is super minimal and clever. I’m not good at this, which is basically why I just blogged it instead of like TikTok’d it or whatever.

0 views
Unsung Today

More molly guards

Ever since I wrote a post about the molly guard , I have been on the lookout for those, and I think I collected enough to do a little follow-up. First, some classic industrial molly guards from a museum in Germany: = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/1.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/1.1600w.avif" type="image/avif"> = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/2.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/2.1600w.avif" type="image/avif"> = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/3.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/3.1600w.avif" type="image/avif"> = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/4.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/4.1600w.avif" type="image/avif"> This IBM electronic typewriter had a gorgeous perspex molly guard around the power button: = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/5.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/5.1600w.avif" type="image/avif"> Other machines opted for “softer” quasi molly guards that still aimed to prevent you from pressing a button or switch by accident, but without having to get something out of the way first: = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/6.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/6.1600w.avif" type="image/avif"> = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/7.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/7.1600w.avif" type="image/avif"> = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/8.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/8.1600w.avif" type="image/avif"> Even softer? This below is not a traditional molly guard, but the placement of “I’m writing to the SD card” red light was not accidental. Ejecting the card while the camera is writing to it might cause some damage, so the light was positioned right next to the card door and the card itself, making you more likely to spot it and wait: = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/9.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/9.1600w.avif" type="image/avif"> This one is even more clever. You know how some old floppy drives have a handle that lowers the reading/​writing head so that the diskette can be used? That same handle also prevented you from pulling the disk once the head was lowered. It felt so natural, you might not have even realized it’s a molly guard doing its job: = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/10.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/10.1600w.avif" type="image/avif"> On the other side, these following guards are more of a “you really shouldn’t do this” variety – much closer to a disabled state in graphical user interfaces: = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/11.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/11.1600w.avif" type="image/avif"> = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/12.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/12.1600w.avif" type="image/avif"> = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/13.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/13.1600w.avif" type="image/avif"> Let’s jump into software. This is a nice situational molly guard in Finder when you press ⌘O and have a lot of files selected: = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/14.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/14.1600w.avif" type="image/avif"> iPhone’s “slide to unlock” no longer graces the home screen, with one exception – stopping the alarm: There’s something about this treatment that doesn’t sit well with me. I’m not sure what it is: The text not feeling centered? The control being circular? The icon on the slider making it seem like it’s a stop button you can press? Speaking of stuff I don’t love, you might recognize this molly guard from Chrome: = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/16.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/16.1600w.avif" type="image/avif"> This one never felt pleasant to me. You might say “isn’t the point of the molly guard that it doesn’t feel pleasant”? But I think one needs to separate the intent and the mechanics. I don’t mind the intent here, but the styling is ugly, the message kind of confusing – you don’t really have to hold ⌘Q, just press it again – and you also don’t get any feedback during holding. Contrast with this extremely skeuomorphic CD burning molly guard in early iTunes, suggested by one of the readers: And lastly, something I didn’t expect to ever see. Per this issue (page 14) of an alumni magazine of University of Illinois, here’s the actual Molly with her father: = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/18.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/more-molly-guards/18.1600w.avif" type="image/avif"> #definitions #interface design #real world

0 views

JAX backends and devices

There's nothing like writing your own code with a framework to clarify how things fit together! Continuing with my port of my PyTorch LLM code to JAX , I wanted to load up a large dataset: the 10,248,871,837 16-bit unsigned integers in the split of . That's just over 19GiB of data. When I ran that, I got a CUDA out-of-memory error: That makes sense! The allocation it was trying to do is exactly the size of the data I was trying to load. I have an RTX 3090 with 24 GiB, but some is already used up by the OS, various apps, and a model that the code creates earlier on. But in PyTorch land, I was used to things being loaded into RAM by default, and only moved over to the GPU when I asked it to do that. JAX was clearly loading to the GPU by default. How could I stop it from doing that for this case? The load into the GPU was happening inside Safetensors, in code I couldn't directly control. Understanding how to do it helped me understand a little bit more about JAX. JAX has a function that looks relevant: . Without reading the docs, let's try running it. In my virtualenv, with the package installed, I get this: That seems a bit weird! I do indeed have a CUDA device, but I also have a CPU, obviously. Why isn't it showing up? Running the same code in another virtualenv, with just installed -- no CUDA -- gets this: OK, so it did recognise it this time. Feels like it might be time to RTFM. The docs explain things a bit: Returns a list of all devices for a given backend. If is , returns all the devices from the default backend. The default backend is generally or if available, otherwise . OK. So JAX has multiple backends -- named that because they're classes of backend hardware that XLA (the compiler behind the JIT) targets. There is a default one, which is essentially going to be the "best" one available given the hardware configuration and the parts of JAX that are installed. When I had the CUDA version installed, it made the backend default, but when I didn't, it defaulted to (and warned me). And because it only shows the devices on the default backend, when that was , I didn't see the CPU. However, you can specify which backend you want to use with that parameter, so let's go back to the virtualenv with CUDA: Great! So is there some way to list which backends are available? Apparently not -- the recommended way appears to be to try loading devices for the different possibilities, and catch to see which ones aren't available. Yuck. But maybe that's not such a big deal. In PyTorch-land I was very much used to putting code like this near the start of my code: ...then moving models to the device: ...and then moving data to the model's device as needed: What I actually wanted was essentially what JAX does -- have everything on the fastest device available at all times -- but with specific exceptions. In particular, the one that started off this investigation: how would I put this huge array of training data on the CPU's RAM rather than the GPU's VRAM? I had a bit of a false start when I spotted that the function in the Safetensors FLAX API has a parameter, but that appears to be more to do with how it loads up the file -- a backend in a different sense. And anyway, backend is not the right concept in JAX-land, as the backend means just something generic like -- for what we're trying to do, we want to load it onto a specific device . After some digging around, I discovered that JAX has a concept of a default device , which is the one used when it doesn't have any indication of where to put something. It makes sense that this will be on the default backend -- indeed, it looks like it's essentially "the first device in the list that returns for the default backend". There is a config option which you can use to set it; you'd normally use or an environment variable to change it. But what if you only want to change it temporarily? I found this documentation for . The docs are more than a little confusing: Context manager for config option. Configure the default device for JAX operations. Set to a Device object (e.g. ) to use that Device as the default device for JAX operations and jit’d function calls (there is no effect on multi-device computations, e.g. pmapped function calls). Set to None to use the system default device. That near the start tripped me up, as I missed the words "Context manager" just below, and the odd type, and tried this: I still got the CUDA OOM, though, so I reread the docs, spotted the "context manager" bit, swore violently, and tried this: ...which works. It looks like the equals sign in the docs is being used to mean something very different to what you'd normally use it for, and they decided not to actually document the signature of the context manager. Heigh ho. I guess documentation is hard . Still, at least now I have a solution. And as I said earlier, doc grumbles aside, the shape of the code might wind up being a little less fiddly than PyTorch. The default location of things I create is the fastest hardware I have, which is what I want. And for the rare exceptions when I don't want to use that, there is a reasonably simple (now that I know it) way to say where I want things to go. I'll call that a win :-) The only thing I'll need to remember is that when, in my training loop, I want to use subsets of that in-RAM tensor, I'll need to move them to the GPU. looks like the right tool for that.

0 views

2026.23: Power Shifts

Welcome back to This Week in Stratechery! As a reminder, each week, every Friday, we’re sending out this overview of content in the Stratechery bundle; highlighted links are free for everyone . Additionally, you have complete control over what we send to you. If you don’t want to receive This Week in Stratechery emails (there is no podcast), please uncheck the box in your delivery settings . On that note, here were a few of our favorites this week. This week’s Stratechery video is on The SpaceX IPO and Data Centers in Space . Google and Microsoft . Three years ago Google looked hapless, scrambling to respond to ChatGPT, while Microsoft, thanks to their groundbreaking partnership with OpenAI, looked on top of the world. Now Google is pulling away in terms of market capitalization, which makes their decision to issue equity to Berkshire Hathaway a curious one: I try to figure out what it means in The Google Capital Company . As for Microsoft, my first question to CEO Satya Nadella in this week’s Stratechery Interview was a simple one: is he happy with Microsoft’s competitive position? — Ben Thompson YouTubers Take Over Hollywood.  Over the past few weeks, the biggest news in the entertainment business has been the surprising success of two Gen Z YouTubers. They’ve directed the most successful movies in America and beaten (yet another) Star Wars spinoff at the box office. Monday’s Daily Update seizes on that news to explain how we got here and the various factors that explain the YouTubers’ success. We doubled back on the topic on this week’s Sharp Tech to explore not only the implications for Hollywood, but why YouTube, in response to this phenomenon, is unlikely to change a thing.  — Andrew Sharp A Guide to the NBA Finals. The Knicks took Game 1 of the NBA Finals on Wednesday, and I have to say, this week’s Greatest of All Talk preview nailed the contours of exactly what we saw. If you’re looking for what to watch going forward, the intrigue of course starts with 22-year-old Victor Wembanyama. If he can win this title it’ll be the sort of flashpoint that we all remember for the next 50 years. But can he do it? My co-host Adam Mares explained why the Knicks are a tougher Wemby challenge than even the defending champion Thunder were, and that as great as he is, there are real limits in Wemby’s game that will be tested for the next 10 days. Let’s see how he responds. — AS YouTubers Win the Box Office, Goodbye Gatekeepers, The YouTube Bar — YouTubers are ruling the box office, and it shouldn’t be a surprise: succeeding on YouTube is a much higher bar than the gates that currently govern Hollywood. The Google Capital Company — Google has issued equity to Berkshire Hathaway in a deal that signals far more demand and a future where capital is the ultimate commodity. The Nvidia AI PC, Project Solara, Microsoft AI — The Nvidia AI PC feels like a relic of another AI era; Microsoft’s vision for devices at Build was much more compelling. An Interview with Microsoft CEO Satya Nadella About Finding Core Competencies — An interview with Microsoft CEO Satya Nadella about figuring out Microsoft’s role in AI, the relationship with OpenAI, Capex, Software, and a potential new agentic platform. Steph Curry Turns to China — Steph Curry’s move to partner with Chinese shoe brand Li-Ning is smart, lame, and ultimately a refreshing reminder of American strengths. Electric Cars and Meta Subs WWDC Questions Taiwan’s DRAM Failure Seizing The Commanding Heights; Decoding Shangri-La Dialogue; Europe Moots Trade Policy; The PRC Expels a New York Times Journalist Five Questions on the NBA Finals, Wemby and NBA History, Q&A on the Kroenkes, OKC, and Team USA What’s Google Doing With Berkshire Hathaway?, A Bubble Temperature Check, Gen Z YouTubers Take Hollywood

0 views
Andy Bell Yesterday

I did the Standard.site thing

It seems like everyone is integrating Standard.site on their personal websites (and beyond) at the moment. It’s mainly been encouraged by Bluesky’s recent update to the treatment of sites integrated with Standard.site. Sam has a good explainer here . It’s a cool feature! There’s loads of different platforms doing cool stuff, all using different data structures, so the standardisation was really needed. Integrating is fairly straightforward too. If you’re using WordPress, for example, it’s as simple as using a plugin . There’s an Astro plugin too, along with all of your favourites (I imagine). I’ll write up the details as part of my ongoing series . I’ll definitely be getting on that towards the end of the month too as my studio workload quietens down a bit. Was it all a bit of a faff to integrate? Yes. Did I get rate limited multiple times? Absolutely. But do I think it’s worth doing? I think Standard.site is going to do wonders for website discovery, so without a doubt.

0 views

Premium: The Hater's Guide To The AI Bubble 3.0

Last year I wrote one of my favourite pieces ever — The Hater’s Guide To The AI Bubble — and followed it up with The Hater’s Guide To The AI Bubble Volume 2 several months later. Sadly, I’ve realized “volume” is a terrible way to structure something like this, because each volume is more of an update , which is why today’s newsletter will move to a versioning system. The AI bubble is a psyop, a melodrama, a financial crisis, and a mask-off moment for the Business Idiots that run the vast majority of our economy. It is the largest-scale exploitation of ignorance in history, gnawing at the intellectual weaknesses of society by presenting just enough information or just enough proof to substantiate a trillion-plus dollars of investment and manufactured consent for a technology that, based on how many discuss it, doesn’t actually exist. And it’s revealed how many rich and powerful people are either (or both) credulous and woefully ignorant. To be clear, LLMs are real and do some things, but they don’t do any of the things that Dario Amodei is talking about when he says that AI will wipe out 50% of white collar jobs . We’re four years into this joyless slog and people are still talking about AI’s “potential” and what it “will” do and that we’re in the early innings of a technology that, for the most part, is still doing exactly what it was doing at the beginning with refinements that never come close to reaching the vacuous heights of boosters’ promises.  Markets are moved by poorly-written fan fiction by outright scam artists and deceptive hedge fund gargoyles because those selling AI services have entirely disconnected the minds of the markets and the media from reality. This is because con artists like Amodei and Altman constantly discuss what AI will or might or theoretically could do rather than what it actually does , because if they had to do that they’d have to say it constantly loses money and doesn’t have a measurable return on investment . As I said on Bloomberg this week , the markets and the media have conflated capital expenditures for data centers with a thriving AI industry. In reality, 89%+ of all AI revenues and 90%+ of all compute demand comes from two companies — OpenAI and Anthropic — largely based on money-losing subsidized AI subscriptions and unrestrained token burn at organizations run by imbeciles that will go away now that executives are having trouble justifying it because there’s no ROI , in part because AI is too inconsistent and unreliable, and in part because you can’t really measure how much a task will cost .  Now enterprises are already capping their AI spend , with many more are going to follow after multiple companies blew through their annual token budgets in a few months . The sheer volume of the “AI ROI” conversation is remarkable considering that Anthropic and OpenAI only moved enterprises to token-based billing — paying the actual costs of AI — in Q1 of this year.  Remember: the total, actual revenue of the entire AI industry — including OpenAI, Google, Microsoft, Amazon, and Anthropic — has barely reached $100 billion in 2026. That includes every ounce of compute spend, every penny of the $500 million that a single customer accidentally spent on Anthropic’s API , and every cent of NVIDIA’s backstop deal with CoreWeave . More importantly, absolutely nobody is making a profit outside of those selling the bits that go inside a data center.  Both OpenAI and Anthropic lose billions of dollars a year, with no end in sight, though Anthropic did a great job swindling the media by having a single “profitable” quarter thanks to Elon Musk discounting two months of compute . Anthropic has already filed to go public , with OpenAI  allegedly not far behind . Neither of these companies are fit for public investors. Their products are inconsistent, unreliable and only ever seem to get “better” in a kind of wobbly way that can only be measured by increasingly-less-useful benchmarks that they specifically train to beat. Despite many people (and some companies like Spotify ) claiming that AI is writing “most” code, nobody can seem to explain what that means. It isn’t saving money, it isn’t saving time, it isn’t making companies ship better or more-functional products, and the only tangible examples of its effects are that it broke AWS several times and deleted a company’s database . It’s unclear where AI exists outside of coding and the various places companies have shoved it.  I’ve spent years trying to catalogue other, non-coding use cases, and most of what I’ve found are vague descriptions of companies like Goldman Sachs maybe launching agents “soon” at some point to do something maybe and this weird story with Novo Nordisk claiming that it was “integrating ChatGPT’s models to analyze complex data sets” despite them claiming to have done this for years . That’s because generative AI is, no matter how many hats or harnesses or deterministic processes you add, limited by its mathematically-certain hallucinations . These models are probabilistic, guessing at what the ideal output may be, which means that every bit of information they produce is suspicious and every decision they make is brainless, thoughtless and arbitrary. They do not “know” things, they do not have “thoughts,” and no amount of API connections will fix that problem. As a result, nobody has really got a clear answer as to what everybody is doing with AI. Code? Image generation? Using it as a shitty search engine? Using it as a companion? You can’t really rely on it to do anything. When a model hallucinates an incorrect answer to something you know is true it’s a problem that can be fixed — when it hallucinates an incorrect decision with your codebase, that’s fucked everything up to a near-permanent end. This is the ultimate problem with AI. You can try and dress it up with billions of investment and supposed ways to mitigate hallucinations, but it still makes — and will continue to make — mistakes that it has no idea are mistakes.  Well, okay, the other problem is that generative AI just isn’t built to do most jobs. It can generate stuff and summarize stuff at varying degrees of complexity, but the more complex the generation, the more likely it is to hallucinate. The only way to reduce hallucinations is pre-training (shoving stuff into the model at the beginning) and post-training (training it on what “good” looks like), and neither of these actually solve the problem. It is clumsy, inaccurate, unreliable, expensive and cumbersome.  AI cannot do the vast majority of jobs, and the only reason that anybody thinks that it can is that the vast majority of CEOs have no actual connection to the work that enriches them , and because AI can do an impression of something that looks like work, they choose to believe it can do anything . It can burp out a half-functional prototype with the company’s name on it or legitimate-looking legal or financial document, and that’s all it takes for a fuckwit with a high salary and a low IQ to think it’s capable of replacing everybody. If I were wrong, it would actually be replacing people. You’d be able to point to both the data and the proof. You’d have single-person software companies making billions of dollars, hyperscalers would have their companies destroyed by people copying and bettering their software, accountants and lawyers and writers and every other knowledge work career would be dead , not threatened with constant layoffs that are mostly connected to improving profits, but actually dead, untenable, impossible to work in thanks to the “power of AI.” In reality, AI is dramatic only in its mediocrity and the ferocity with which it’s proven how ignorant most authority figures and executives have become. Every boss demands you use it, every app screams at you to try its integration, every news story tells you it will replace you imminently, but in the end it doesn’t appear to do very much beyond generating and summarizing at varying levels of complexity.  The media categorically failed to scrutinize an industry built to exploit it, as I said earlier in the week : The consent has been manufactured and the markets are engorged with semiconductor stocks running because people keep mistaking the availability of debt for actual, real demand for AI compute. The geniuses in private credit and the greater markets saw the amounts that hyperscalers were spending on data centers and the ascent of OpenAI and thought “fuck me up, grandpa,” leading to $178.5 billion in data center debt deals in the US in 2025 and $50 billion in data center construction in April 2026 alone .  Yet it turns out that data centers take anywhere from 18 to 36 months to build , with Microsoft finishing a grand total of zero of the data centers it broke ground on in 2023 , and JP Morgan saying a month ago that 60% of capacity planned for completion in 2027 hasn’t even started construction, with another 7% delayed, per the Wall Street Journal .  And despite the supposed 100GW+ of data center capacity being planned, AI compute demand doesn’t really exist outside of Anthropic and OpenAI, two companies that rely on perpetual flows of venture capital and debt to survive. Between them, they’ve raised over $200 billion in the last six months , and their revenue streams are inherently based on either unprofitable AI startups subsidizing their subscriptions , their own unprofitable subsidized subscriptions, or experimental token spend borne of companies allowing their employees to burn as much as they’d like , which is already coming to an end. At the top of the pile lies NVIDIA, the largest company on the stock market, which sells GPUs that are so expensive that once cash-rich hyperscalers are now having to take on mountains of debt or, in Google and Oracle’s case, dump tens of billions of dollars of new stock into the markets. NVIDIA’s continued growth relies on a dwindling subset of clients, with 54% of its last quarter’s revenue and 64% of its accounts receivable coming from three customers in its last quarterly earnings.  Demand is somehow both incredibly high for data center components but so low for AI compute that NVIDIA has agreed to spend $30 billion over the next six years to rent GPU capacity.  That’s because the AI buildout is being driven by people who haven’t bothered to check whether the demand is real, much like AI is being adopted by people that don’t bother to do any real work, much like AI is sold based on things that it can’t actually do.  Midwits and the incurious will say this is just like the Dot Com Bubble ( it isn’t and won’t leave behind any useful infrastructure ), or Uber ( it isn’t ) or Amazon Web Services ( it isn’t ) because they want to rationalize the waste. In reality, the people running the tech industry are listless Business Idiots throwing as much cash at the problem as possible rather than facing the fact that they’ve backed a dead-end technology because they’ve run out of hypergrowth ideas . Today’s piece is an attempt at a little fun — a raucous, aggressive rundown of the major players and stories of the AI Bubble, both as a refresher for those who already know and a guide for those that don’t. Welcome to the Hater’s Guide To The AI Bubble 3.0. The Rot-Com Bubble — A Guide To How The AI Bubble Got Inflated Why You Keep Being Told AI Is Powerful How The AI Industry Is Almost Entirely Wrappers For OpenAI and Anthropic’s Models How NVIDIA’s Findom Operation Conned Every Hyperscaler How Microsoft’s AI Strategy Has Fallen Off The Rails How Google Is Using AI To Destroy Its Legacy How Amazon Lost The Plot And Became Anthropic’s Paypig  How Mark Zuckerberg Burned $158 Billion To Buy GPUs For Effectively No Reason How SpaceX Became Musk’s Last Gasp Attempt For Exit Liquidity How Anthropic Is The Greatest Exploitation of the Media and Economy In Tech History To Prop Up An Unsustainable Company Run By The Most Annoying People Imaginable How OpenAI Became A Miserable Failson With Too Many Ideas, Unsustainable Economics, and No Plan For The Future How The ROI Conversation Could Burst The AI Bubble

0 views
David Bushell Yesterday

Are you standard.site?

Standard.site provides shared AT Protocol lexicons. Atproto is just spicy JSON and asymmetric cryptography. I’ve tried to explain atproto in more detail before. Bluesky has always supported a few open graph meta tags which I use to generate images for blog posts. That’s part of the social media game; get in people’s faces as loudly as possible. Now the game has changed! I return Monday ready to work and suddenly I start seeing a fancy new “View publication” button appear in my Bluesky feed. I’ve never wanted nor needed a button before but now that people are rocking buttons, what am I supposed to be, a buttonless pleb? I got my own button it looks like this: Mat Marquis, fellow button connoisseur, was quick with a guide to “Implementing Standard.Site” which I hastily copied. Mat used an atproto explorer to edit records which is akin to rawdoggin’ SQL in production. Given the weekly GitHub and NPM malware party this is probably a safer play than running yourself. I’m never going to remember to publish manually though. I have a janky build script and some experience with the @atcute libraries . How hard can it be? My script begins by generating a manifest of pages by parsing markdown before rendering the HTML template. I added a new step that fetches all atproto records in the collection. It cross-references the paths in my manifest. Any unknown path has the record deleted. It then iterates the manifest and either updates the atproto record (if title or description has changed), or creates a new record if none existed. Finally it adds the atproto URI to the manifest for the element. Now my blog is standard.site and I have a fancy button to prove it. Thanks for reading! Follow me on Mastodon and Bluesky . Subscribe to my Blog and Notes or Combined feeds.

0 views

Barry Hess

This week on the People and Blogs series we have an interview with Barry Hess, whose blog can be found at bjhess.com . Tired of RSS? Read this in your browser or sign up for the newsletter . People and Blogs is supported by the "One a Month" club members. If you enjoy P&B, consider becoming one for as little as 1 dollar a month. I’m a programmer-type from rural Minnesota. I grew up on a farm near a small town. Now I live in a bustling city of 27,000 people…surrounded by farmland. In other words, I’m still in rural Minnesota. I studied computer science at a small private college, which led to my 26-year career programming computers. First it was at an insurance company, then it was at a SaaS startup, and now it’s for myself at a little company I run with my business partner. My hobbies are mostly typical: reading, watching movies, and the occasional video game (meaning Fortnite). My favorite sport is baseball, though I’ll watch the occasional other sport. I also try to do a little woodworking, cooking, and, well, blogging. Blogging is a hobby, yes? I decided to start a blog in 2004. Personal blogs were popping up all over, and I was enjoying meeting new people through the comments section in these blogs. I also have a couple non-blogging friends that were doing their thing on Xanga . The blogs I followed were either friends, friends of friends, about the Minnesota Twins (baseball), or about U.S. news and politics. Online I generally use the handle bjhess. That was what my college gave me for my first ever user account. Toward the end of college I was looking for a domain name, and unfortunately there was already a techy person with my first and last name who grabbed that obvious option. (They still have the domain to this day!) So bjhess.com it was, and the name stuck. I blogged via b2evolution and WordPress in the early days, probably at Dreamhost. In the early 2010s I switched over to a self-hosted and customized install of Scanty , and I ran that for a long time. In 2022 I switched to an HTML-only site . That lasted about a year before I and my colleagues built Pika. I don’t have a system or process for blogging. My inspiration is generally from interesting things happening in my life. That can be a vacation, a recent discovery, an experiment that I’m trying, or a feeling that I’m feeling. Most of my posts are written in a single session, with a couple rounds of editing for grammar, tone, and flow. There’s only been one occasion where I asked others to read my writing before posting. I’ve recently tried the “weekly update” format of posts, which to this point has been me adding links and notes to a draft leading up to finalizing the post on Friday or Saturday. I’m toying with updating the draft post daily throughout the week before publishing, but then if I’m doing that I wonder if I should…just post those daily updates daily? Inspiration comes and goes, but I generally prefer to have quiet while writing, whether that’s natural or simulated via headphones. Aside from that basic need, I don’t strongly believe that physical spaces influence my creativity. However, I’ve been noticing that my office is in a state of constant clutter…and I’m starting to believe. Now the question is whether that clutter impacts the mind or whether the cluttered mind leads to a physical manifestation? A little of both, I think. Today, and for the rest of my life, my blog is hosted at Pika . I write my posts directly in the web editor. I would start my blog on Pika, naturally! I believe pretty strongly that most bloggers probably would be better off not rolling their own static site generators or CMS installations. For those that want to play in that world, though, there’s nothing like it. For the rest of us there are a number of small, independent blogging platforms that make things quite a bit easier. They all tend to play nice together, offering exporting and importing options if you ever find a different platform to be a better fit for your style. If I were paying for my Pika account, it would be $60 per year, and my domain is $13 per year. Not bad for a favorite hobby! I pay $9/month for Plausible analytics, though I’m not entirely sure why. As a programmer, I think it’s mainly that I want a place to look to see any weird happenings to make sure nothing is amiss. If traffic to my blog disappeared, I’d be curious if I did something wrong technically to cause it. All’s fair for monetizing. I don’t do it, but I know affiliate links and such make sense in some contexts. Let me dial up my feed reader here. Okay, for a selection… I’m not sure how Chris Glass keeps his daily photo journal going, but it’s great. Rafał Pastuszak does fascinating things at Untested . Adam Keys is usually thinking . Since Luke moved away from my area, I like to read what’s going through his mind on recursion . I travel vicariously through MacPsych. Maique gives me all the photo inspiration . Holy cats, Jamie Todd Rubin is an avid reader . Brendon Bigley provides cool video game news . Annie lends me insight . Davey and Jamie share lives well lived. I also like to keep up with Derek Sivers , Hugh Howey , Craig Mod , and Cabel Sasser (I still need to read the 2025 snacks rundown). Oh, and, boy howdy, Mike Monteiro . Any of the above who haven’t been interviewed would be a great option to interview next! I won’t be shy–I’m working on Pika and I would greatly appreciate it if you gave Pika a look. Our biggest project at the moment is The Pika Pulse , which will be a great help to discover Pika blogs. I think that’s a good thing for the readers of People and Blogs! Mostly, though, I’d like more people to blog. I want people of all ages and backgrounds sharing their experiences at their own domain online. Whether you do that via Pika or any other setup or service (yes, even WordPress), I’ll be excited! See you online! Now that you're done reading the interview, go check the blog and subscribe to the RSS feed . If you're looking for more content, go read one of the previous 144 interviews . People and Blogs is possible because kind people support it.

0 views
Herman's blog Yesterday

The Giant's Cup

I recently completed my first long trail race, set in the Southern Drakensberg mountains with Emma, my siblings and their partners. It spanned 2 days with 30km on the first, and 15km on the second, winding through valleys and around mountains. It was spectacular. I love coming to the Drakensberg, which is a beautiful and unique mountain range, and I'm glad to have the excuse to be here. While I'm generally an active person, I've never been a runner. My first proper run (more than 3km) was in January this year after my brother twisted my (and Emma's) arm into signing up for this race. Since this was not a trivial distance, it meant I couldn't just wing it on my base fitness, and so Emma and I started running 2-3 times a week in preparation. We had 15 weeks until the event and settled on a fairly simple training plan: We didn't stick to this plan exactly, especially since the final 4 weeks coincided with our trip to Japan, but we stuck to the spirit of this plan, and it worked out quite well, with our running fitness increasing quite dramatically over that period without any injuries or overreaching. This is in contrast to other times in my life where I'd try to get into running, start off too ambitiously, become a sweaty, exhausted mess with creaky joints, and ultimately decide that running just isn't for me . Because we capped the increase of speed and distance over time to no more than 10% per week, each run was challenging but not insurmountable. It was also nice to see how quickly running fitness increases when compared to strength, which can take months to see measurable gain (after the initial neural adaptation). One caveat is that I hate cars too much to enjoy road running, and the symmetric, hard surface makes my right knee ache in a way that trail running doesn't. Plus being on a mountain is so much prettier than the asphalt city grid or suburbs. What I enjoyed the most about this experience was how it opened up a whole other world of events and community that is both accessible and affordable. There was suddenly an excuse to get out onto the mountain at least once a week with friends. To go out to parts of the surrounding countryside that I'd never been to before, stand in the chilly morning air while sipping on instant coffee from a chipped enamel cup, then run through it while the sun came up. The event we ultimately completed—The Giant's Cup—was a particularly good one in that it was a destination event, and a non-trivial run with people I love and enjoy spending time with. And while I may never become a Runner (I'm by no means placing), I'm certain that I'll be a runner for the next several decades. Run 5km each Wednesday, usually on the Sea Point Prominade, while increasing the speed gradually each week (it took me a few weeks to get to the initial 5km though) Longer trail runs on the weekend, generally on Table Mountain or in the Cape Winelands, increasing distance by about 1km each week Hop on the treadmill at a reasonably fast pace for about 20 minutes after gym one day a week

0 views
Brain Baking Yesterday

The Archivist In Me Turned This Blog Into a Book

Four years ago, in the article What Happens To My Digital Identity When I Die? , I wrote the following prophetic words: […] Which gets me back to this website. My intentions are to someday publish its contents in the form of a book, which can also be stored at the KBR [Royal Library of Belgium]. This allows the people dear to me to still have access to the silly stuff I write here. Two years later, I claimed that Good Blogging Habits Yield a Book Each Year of more than words. That means compiling a hefty tome to compress all these years of productive blogging into a single physical volume might be a bit more challenging than I initially anticipated. Yet not impossible. So the last months, I’ve kept myself busy by doing just that: turning this blog into a book! The book flipped open on the blog post 'Three Little GameCube Mods' from 05 December 2021. This was a special experiment with a high probability of failure as I wasn’t sure how it would turn out. What should a Brain Baking book look like compared to browsing this website? How should it feel to flip through the pages? Will I be able to squeeze everything in there (nope)? Do I want to publish this publicly or just generate a , send it to the presses just for myself and call it a day? And if so, which service to use, as the past ones I’ve relied on all showed their shortcomings? Luckily, it turned out all right. I call it Brain Baking DX: Blog Archives 2016 - 2026 and it is available at Amazon under ISBN-13 number 979-8197112897 . My first attempt yielded more than 500 pages and I couldn’t find a publishing service that was eager to print something like that for less than . At Amazon, the book costs… . And it’s globally available, should you be crazy enough to want a copy. Be warned, though: the book is mostly unedited , I created this mainly for myself. I sent out a few copies to friends but have no intention of setting up a marketing campaign let alone making money off of it. I intentionally set the price low and receive for every sale. Please do not buy this just to support me. So why a book? As mentioned before, I want my writing to be a bit more permanent than the fleeting medium called the internet. What happens when my VPS is blown up, my backups burned away, and my motivation to restore all this along with it? In Belgium every author of “proper” books (this is debatable nowadays… Is Brain Baking DX a proper book?) is legally obliged to deposit two copies to the Royal Library in Brussels, where the books disappear into the winding depths of the archive deep below the capital. Plus, I like books. I like flipping through this one and rediscovering old writings: it feels very different than clicking through the online archive. Also, since I like to add photos in my blog posts to help shape the atmosphere, preserving these mostly personal photos in the book makes me feel warm and fuzzy inside when I flip through the book and look at them. Some of these photos are snapshots of my life as a kid, my old and new desktop setup, destinations I once biked to, etc. It’s a nice memento to have these included. Why Brain Baking DX ? What’s up with that? Most readers of this blog know that I grew up with a Game Boy and became a big retro gaming nut(case) because of it. The black DX cartridge editions transformed their original Game Boy release into the wonderful world of colours: black carts work with the original Game Boy and the then new Game Boy Color. DX was simply the “DeluXe” treatment to your beloved Link’s Awakening or Tetris . Funny, as Brain Baking DX might be called anything but deluxe: financial constraints prevent me from publishing this book in full colour mode and space constraints prevent me from simply dumping everything that I’ve ever written in here. Perhaps both are for the better. Still, in a way, a printed edition of ten years worth of Brain Baking blog posts can certainly be called deluxe. My method for compiling the book wasn’t as simple as throwing every Markdown article source file at Pandoc to compile a single . I didn’t want to preserve everything I post here: it had to be a deliberate, curated selection. Things I didn’t want in there include: After proceeding to make a first selection, I categorised these into major themes that became the parts of the book: parenting, journaling & writing, work, the web, technology, retro, video and board games, life & philosophy, food & cooking, and living in Belgium. Then I employed my usual Markdown/Pandoc/TeX magic and inspected the results. The front cover of Brain Baking DX. 600+ pages. Ouch. Now what? Maybe it is time to think about the layout: how do I want to present all this text? Clearly, a typical book layout won’t do. I turned down the font size, opted for a two-column layout, selected a more wide book format ( ) and squeezed everything I could out of those margins. As a last resort, I also allowed chapters to start on any page (as opposed to the right page only which introduces a lot of blank pages). I admit I might have overdone it a bit as the top margin is very thin, but all these changes did reduce the page size to a more manageable 470. After ordering a few copies for myself to inspect the result, I was afraid that the text would not be very readable, or the margins where the book would be glued would be too narrow. Fortunately, the end result is surprisingly pleasant to read. The cream paper Amazon provides is a nice match although the paper feels a bit too thin for my taste. Yet a hefty tome like this for is ridiculously cheap so I can’t complain. The fact that the book is printed in black & white does not work against the many photos and screenshots included. Additionally, because it’s Amazon, it allows the book to be distributed and printed virtually anywhere. My copies were printed in Brétigny-sur-Orge in France. Better than China! When you are selecting blog posts to be included into the book, you’ll notice recurring themes you wrote about. For example, I have wasted too many words on physical video game collecting. Instead of just pasting these chapters next to each other, I wanted them to “flow” better in the book so I did rewrite portions to better match the medium. Also, in many occasions, a new chapter (thus blog post) starts with a reference to the previous one. On the site, this is just a link, but on paper, you don’t want to print “in this article”. Speaking of links, a blog or website is an interconnected medium: how to approach this on paper? I ended up putting all LaTeX links in the margin footer on the same page but did a diagonal sweep to remove the excessive ones. On the site, a long link is just hidden behind a click, but on paper, an link to a long URL is not only ugly but will never be typed over or “used” in that way. Also, internal Brain Baking links usually start with —in the end, I decided to keep it that way as prepending everywhere would mean even more text wasted. I did make a note of this in the newly written introduction. Besides the “in this post” link adaptations (don’t do this—it’s also bad for accessibility in your online blog!), I noticed I also had to do something about the images. Because of the two-column layout, the wide figures such as graphs will be squeezed into a barely readable square. You can fix this by manually adding a to the ones you want to be displayed as a full-page spread ( ). But I made another mistake: in many posts, I write something and then add an image to emphasise the statement, ending in a semicolon to point to the image. Yet in a book, you never know precisely where that image will be included! In my future writings, I’ll take these things into account to more easily compile Brain Baking DX II in ten years. This was a lovely month project that rewarded me with a physical artefact of an ever-evolving digital medium, solidifying words, sentences, and paragraphs in a way that perhaps might even envy The Internet Archive. As a hopeless sentimental person, flipping through the book, looking at the figures and reading the text makes me happy. And also embarrassed as there are plenty of contextual and grammatical mistakes in solidified as well. I’m looking forward to revisiting the project in ten years! If you want to attempt something like this for yourself and don’t know how to approach this technically, drop me a line and I’ll be more than glad to help you out. Related topics: / archiving / By Wouter Groeneveld on 5 June 2026.  Reply via email . Research and topics regarding creativity & bread baking: I have other published books that delve into this. Monthly link sharing posts and other posts that are mainly lists or links. Technical posts on programming, coding, Hugo tips, etc. Design mistake posts. Overly negative posts. Anything that has the word “AI” in it (except my more elaborate commentary). Too short posts to be worthwhile printing. Too photo/screenshot intensive posts to be worthwhile printing.

0 views

Attribution in the Browser: Who Really Benefits from Google and Meta’s New Privacy Standard

Google, Meta and the unlikely addition of Mozilla are teaming up to work on a browser W3 specification that would add browser user agent features to track impression, click data and ‘conversion’ data. This data is then sent to respective parties Ad Impression → Ad Networks Conversions → Advertiser So far this is just a duplication of what is naturally tracked by each party, ie just their own resources. The difference then is that this data is forwarded by each party to an attribution service provider (Google/Meta/Mozilla) who aggregates and returns conversion histograms to the Ad network: Ad Impression → Browser Function → Ad Network → Attribution Provider Conversion → Browser Function → Advertiser → Attribution Provider At first glance, this would nearly seem like any expected flow of ad data, but here the “Advertiser” seems to stand in for the Ad Network , working on behalf of the advertiser. Why? Because otherwise the owner of the site would need to manage the attribution selection and call along with then updating the Ad Network. So what realistically will happen? Ad networks will require JS pixels to be dropped on the advertisers site to manage the and attribution process. So the real process: Ad Impression → Browser → Ad Network → Attribution Provider Conversion → Browser → Ad Network (via pixel on advertiser site) → Attribution Provider Firstly, this mostly seems to be a fix for situations where Cookies are removed. How is this problematic for users? As a user I do not mind when an advertiser (eg Nike) tracks what blog I came from to Nike website. My concern is a Meta / Google that tracks every site I was on and went to. So in this way, I think advertisers and users should be aligned. Mobile attribution is based on device fingerprinting. While MMP companies like AppsFlyer (45% app market share) are not mentioned, there could be some potential for MMPs to work on behalf of an mobile advertiser to call and gather attribution from mobile web to app that is *not* just fingerprinting. AppsFlyer has recently released web2app which, despite the hype, has the usual probabilistic and short lookback windows for deferred deep link installs. MMPs would have a strong desire to move from probabilistic to something more deterministic. The problem? The usual, what’s always kept mobile and digital ad measurement separate. A WebView opened by the advertiser app does not have access to the device regular browser cookies. Given that mobile operating systems are run by the ad networks Apple and Google respectively, you could see this some change here if the browser Attribution API comes to pass. Still they would likely have a hard time carving out a space for MMPs to stay between the ad network and advertiser. The API has the potential to support small web publishers but the danger that this is simply co-opted by the ad monopolies to consolidate their positions is real. With the idea of browser tracking ads, why not move the whole process into the local browser, completely cutting out the server calls until a conversion is recorded? This is a popular idea but would add potential avenues for ad fraud where a can be called locally where the ad network ecosystem wide resources to realize this is likely a fraudulent impression would be lacking. I think there is room there for other fraud fighting models to help, but the obvious threat of this type of fraud will likely keep a completely local attribution model from being developed for now. This leads to the current W3 spec and where we are now. I see real positives and negatives to the suggested specs. I’ll be keeping an eye on it in the coming weeks to months (deadline is November 2026 for the working group to finish) and see how it develops, or if this gets added to the Privacy Sandbox graveyard.

0 views
Xe Iaso Yesterday

IPv6 zones in URLs are a mistake

IPv6 is weird. One of the more strange parts of the standard is that every interface's link local addresses are in . If you have a machine with two network interfaces, both of them will be in , so if you have a packet destined to , how do you disambiguate it? The answer is you use IPv6 scopes/zones . The exact format of what goes into a zone is OS dependent, but on Linux it's the interface name and on Windows it's the interface ID. This lets the kernel's routing table know how to handle an address range conflict. On my tower, this would be represented like this: Where is the name of my tower's ethernet device. When you create a host:port bindhost, you normally separate the hostname and port with a colon. IPv6 uses colons to separate hex groups. In order to disambiguate what's the host and what's the port, you typically format the IPv6 address in square brackets, so on port 80 would look like this: And with the right scope it looks like this: Now let's get URL encoding into the mix. From high orbit, you can imagine a URL's format as being something like this: An IPv6 zone would then be part of the hostname, just like with that port 80 example from earlier. So you'd think the URL would be something like this: But if you try to parse this as a URL in Go, you get an error: This happens because URLs can't represent all Unicode values, so any values that don't fit into the grammar of a URL become percent-encoded . This is why sometimes you'll see a in URLs in the wild; that's encoding the ascii space key, which is invalid in URLs. In order to work around this, you need to percent-encode the percent sign in the IPv6 zone: In theory, there is guidance for how to properly handle IPv6 zones in user interfaces in RFC 9844 , but there's no such guidance for URLs . Go also does not seem to follow this RFC in net/url . EDIT: It seems that this behaviour is compliant with RFC 6874 and that this is in fact how it is meant to be done. Our industry confounds me. So in the meantime in order for Anubis to point to IPv6 zoned addresses, you need to encode the with percent encoding. This is horrible, but it seems that this is an edge case that applies to other frameworks, programming languages, and libraries: Maybe some day in the future there will be a better option here. In the meantime my policy of not forking the Go standard library means that this somewhat terrible UX for an edge case is acceptable. I hate it, but what can you do? TL;DR: computers were a mistake. https://trac.nginx.org/nginx/ticket/623 https://github.com/psf/requests/issues/6808 https://datatracker.ietf.org/doc/html/draft-schinazi-httpbis-link-local-uri-bcp-03 -- Browsers don't currently support IPv6 zones because it breaks the concept of an "origin" which is used for many subtle things, this RFC draft attempts to define an zone origin in IPv6 so that browsers have a leg to stand on

0 views
Giles's blog Yesterday

Using Safetensors with Flax

I'm porting my PyTorch LLM code to JAX , using Flax as the neural network layer. For various reasons I wanted to use Safetensors to store checkpoints of the model. It took a little while to get it working; here's the trick I learned. If you look at the Safetensors docs, you'll see that it doesn't mention a JAX implementation -- indeed, searching for "safetensors jax" at the time I'm writing this gives you a link to this GitHub repo by Alvaro Bartolome -- which was last updated in 2023. However, if you look more closely at the docs, they do have a link to the Flax API . I feel this is somewhat misnamed, as it is actually a JAX API. There's no reference (again, as of the time of writing) to Flax in the source -- it's all just JAX code. And in fact Bartolome's library uses it under the hood. There is one problem, though. The API works with simple single-level dictionaries, with strings mapping directly to JAX arrays. For example, the function has this signature: This can cause problems if you're not careful. If you look at the Flax documentation on checkpointing , it suggests that you use Orbax 1 , which has its own API and file format, but then goes on to say: When interacting with checkpoint libraries (like Orbax), you may prefer to work with Python built-in container types. In this case, you can use the and API to convert an to and from pure nested dictionaries. I initially put two and two together -- that and the dictionary-based API for Safetensors -- and got five, and tried feeding one of those "pure" dicts into Safetensors. I got a very confusing error: It's worth digging in to why that happens. The problem is that although Safetensors is expecting a dict of strings mapping to tensors, it doesn't check that that is what it actually gets. And while the dictionaries from are "pure", they are also nested (as the docs say!). Even for the simple model I was working with, I got a structure like this: So, we had strings mapping to dicts, and those dicts mapped from strings to the JAX arrays. More complex models would have had deeper dict structures. Now, internally inside Safetensors, the Flax/JAX API is a simple wrapper. It iterates over the keys in the dictionary it's been provided with, and tries to convert their respective values into NumPy arrays. It does that by passing them into NumPy's function, which accepts things like lists, tuples, and NumPy arrays, and converts them into arrays. JAX's own class exposes an interface that it recognises, so they're converted without trouble. Once it's done that, it passes the result to a lower-level Rust implementation that actually converts everything to Safetensors format. But because Safetensors didn't check types, in my case it was iterating over the top level of the dict, trying to convert the values to NumPy arrays, and got something like this: That is -- because it assumed that the values in the top-level dict were JAX s, it blindly tried to convert them to NumPy arrays. But they were dicts (that happened to map from strings to arrays) -- and if you ask to create an array based on a random object, it happily does so and wraps that object in a NumPy array, with a of . When that is then fed into the lower-level Rust code that is trying to write the file, it encounters NumPy arrays that have a it can't handle, -- hence that error: It all makes sense when you read through the code, but I was a bit perplexed for a while! I think all this might be the reason why Bartolome created his GitHub repo. In the README, he says that: There are no plans from HuggingFace to extend safetensors to support anything more than tensors e.g. , see their response at huggingface/safetensors/discussions/138 . So the motivation to create is to easily provide a way to serialize using safetensors as the tensor storage format However, you don't need to use that library to serialise simple Flax models. Consider how PyTorch models get serialised to Safetensors; my LLMs have keys with names like , , and . They're "flat" dictionaries mapping strings to PyTorch Tensors, similar to what Safetensors wants for these Flax ones, but they use dots to separate different levels, with integers for list items and strings for field names. Looking at the pure-dict structure I had for my model: ...you can see that you could walk the dictionary structure to generate keys like and . That would be easy enough to code up. But -- as Adithya Dsilva points out on GitHub -- you can get there even faster by using . That returns a (non-dict) structure like this: If you iterate over that , you get tuples where the first element is that tuple of strings, like , and the second is a object wrapping the JAX . The tuples mirror the dot-separated string format in the PyTorch-style Safetensors files. objects also implement an interface that can understand, so you can quickly and easily convert the to a regular dict for Safetensors: (You need to wrap in a because if you have a in your model, the item in the tuple will get an integer index rather than a string). You can go the other way pretty easily too; given a model, you can load the saved checkpoint into it like this (because accepts raw JAX s in place of explicit s): A little more work than I'd ideally like, but given that it can be tucked away in general / functions, not too big a deal. Hope that's of use for other people coming across this problem! I'm beginning to feel a bit swamped with all of these libraries with names ending in -ax. It reminds me of the names of the characters in Asterix's village ...  ↩ I'm beginning to feel a bit swamped with all of these libraries with names ending in -ax. It reminds me of the names of the characters in Asterix's village ...  ↩

0 views
Unsung Yesterday

“Then suddenly we were boring, bloated, and not particularly interesting.”

In 2021 and 2022, product manager Steven Sinofsky wrote a… …first-person account of what I saw at the PC revolution from the perspective of joining Microsoft as a newly hired software design engineer fresh from graduate school working on developer tools, through my time as a program manager and ultimately leading Office, and then moving to Windows, and everything in between. Sinofsky called the series Hardcore Software: Inside the Rise and Fall of the PC Revolution . It covers 1989–2012 and somewhere inside over 100 chapters, there is a fascinating six parter about the “ ribbon ” redesign of Office 2007. The first part covers the challenge of the team in 2007, taking stock of Office after almost 25 years of its evolution. (Number of toolbars in 1983: one. Number of toolbars in 2003: 31.) The second part shows great screenshots of all the Office versions from 1.0 until then, and the remaining four cover the Ribbon redesign process. Regardless of how you feel about Microsoft Office today, and whether you consider the Ribbon interface a success, it’s a perfect weekend read as it covers universal challenges of software complexity and change management. It’s such a potent series I’m sure we’ll come back to it. It covers a lot, including – in the first part – wrestling with a definition of bloat or complexity, which in the context of Office was less about the number of functions available, and more about mastery: […] In practice, bloat comes from the fact […] that Office does so many things that customers just assume the product can do whatever they need it to do. Despite that fact, customers have no idea how to make the product do what they need. This feeling of helplessness that leads to frustration. […] Bloat is owning a product that you cannot master. This below is a great observation about the perils of an idea of a “simple mode,” which Sinofsky argues is always a leaky abstraction : We tried reducing bloat by hiding features […], but that only added to the mystery of the product. Mac, Windows, and Office all went through periods of “simple means fewer” and tried mechanisms such as short menus, simple mode, or adaptive toolbars. But that frustrated or confused people. No one really wanted to use a simple mode and there was always one command missing that was needed, so simple mode became a complicated way to do that one thing that made someone’s work unique. It was great to see this argument for a broad definition of a bug, as it slides exactly into my post from a while back : Ages ago in ancient Microsoft history there was a debate on the original apps team about what it means for something to be a bug. Is it a crash? Is it data loss? Is it a typo in an error message and so on? Out of that was created a notion of bug severity, a measure for how serious a bug might be from losing all data all the way to simple cosmetic issues. However, when it came to talking about bugs with product support or ultimately customers the definition of a bug was very simple “a bug is any time the software does not do what a customer expects”. This definition created a discipline of documenting everything reported about the product and always making sure every issue was looked at, even if a code change did not result. The key lesson was how helpful an expansive definition was. There are also observations and research about how users “debug” the product to make it achieve something they know is possible, but they don’t know how: We called the futzing document debugging, and it created a frustration that the product was powerful yet overwhelming. People believed a specific result was achievable but getting from point A to B seemed impossible or unlearnable. And some about the challenges of figuring out what features people use: […] Most people didn’t know or care what buttons they clicked on or menus they chose so long as it was working for them—and that meant when asked, “Did you use X?” most people couldn’t recall. To a skeptical press or IT manager (and they all were) that meant unused features. I should stop quoting and let you read in peace. But, check this out. Lisa wasn’t the only one having linguistic fun: Early keyboard shortcuts were simple, like using Ins(ert) key to copy text from the scrap (clipboard). #case study #complexity #definitions #flow #software evolution What is Software Bloat, Really? A Tour of “Ye Olde Museum Of Office Past” Competing Designs, Better Design Progress From Vision to Beta First Feedback and a Surprise Defying Conventional Wisdom to Finish Office

0 views
Chris Coyier 2 days ago

The New Van

I got something I’ve wanted for years and years! A camper van! I’m a camper van guy now! It’s a Mercedes-Benz Sprinter, and even more technically a Winnebago Revel . I scoured Craigslist, Facebook Marketplace, and RV-specific sites for a long time drooling over these things. I’ve rented a half dozen of them on Ourdoorsy over the years. I’ve borrowed friends. So I feel like I knew what I wanted and I knew the specific price range I could go, so it took a little while to find it. Ultimately one that came up on Craigslist led back to one sitting on the lot at a local spot called Just Used Cars . I liked the size of it. Just a normal length, not “extended”. Plenty of height to stand up in. I like that it’s an actual Sprinter base because of it’s nice poise/stance compared to other van bases. Also it’s 4WD and has good ground clearance which. That wasn’t a requirement for me, but this will make me trust it driving in the winter and up to Mt. Bachelor and such, which will be really nice. Although funnily enough, I’ve already gotten it stuck once out in Pacific City at the beach — even in 4WD Low and using the traction boards on the roof. Sand is rough. I like the tan color as well. Maybe I’ll get some cool decal or have an artist paint the side or something someday. I wasn’t specifically looking for a Winnebago Revel, but that’s just how those roll, and honestly, the Winnebago name sounds nice to me. Long history making campers, obviously. The interest rate on this thing was horrible. It’s 8 or 9% or something. It doesn’t really matter, as my plan is to pay it off in the next few months. My thinking is that it will do great things for my credit score this way. We’ll see. I co-signed for a “normal” new car just recently, and that rate was 3%, which seems fine/good. I started writing a bunch more little stories about the van. I’ve been using it and thinking about it and working on it a ton, so there is a bunch to say. But I think I’ll break those out into smaller blog posts as I go! One quick one: after I bought it, the dealership called and told me the previous owner wanted to talk to me. I approved them giving him my phone number, we chatted, and he came over to see the van. I was able to return to him some things that belonged to him tucked away into crannies all over the van. He was a nice guy who just really really wanted the new owner to understand it . All the little details about how it worked and where you can put things and quirks and whatnot. We spent a few hours going over things. I really appreciated that, and it shows how attached some people can get to these homes-on-wheels.

0 views

Broker-Visible vs Client-Local Parallelism

This post is a little side-quest from my “Kafka Share Groups and Parallelizing Consumption” series. My “Kafka Share Groups and Parallelizing Consumption” series ( part 1 , part 2 ) has been laser focused on how different configurations and behaviors affect parallel consumption in share groups (Queues for Kafka). So far I’ve shown that you most definitely can hold share groups wrong . You could quite easily and inadvertently create a work queue and with the right combination of things going against you, see a small number of consumers dominate, leaving most consumers starved of messages. All the while lag builds and builds. You need to know the settings and what they do. Don’t just rely on the defaults. But it’s worth asking the question: is parallelizing consumption what share groups are for? The answer is no. If your only concern is parallel consumption, then there are other options. Chuck Larrieu Casias wrote a good post on LinkedIn pointing out that people shouldn’t be thinking of share groups as THE solution to parallelizing work (without exploding the partition count). Share groups exist to expose queue-like semantics over a log. Unlike a normal consumer group, a share group lets you accept one record and reject another for retry. A consumer group tracks one committed offset per partition. A share group has to track many individual records independently: which records are available, which have been delivered (to whom), which have been acknowledged, and which should become available again. But just because share groups don’t exist primarily to parallelize work doesn’t mean it’s not a tool that can be used for that purpose. If your messages are independent or you are otherwise ok with loose ordering then share groups could be a simple choice for breaking away from partition count as the unit of parallelism. The central theme I took from Chuck’s post is that parallelism has to be accounted for somewhere . The unit of parallelism can be broker-visible and broker-managed, or client-local and client-managed. Broker-visible/managed can only take you so far. When you need to process 1,000 messages in parallel to cope with the producer rate, what represents those 1,000 parallel units of work? Is it partitions, consumers, virtual threads/async tasks? If the unit of parallelism is the consumer itself then we must scale out serial consumers to scale the parallel processing (with a matching partition count with consumers groups). Every parallel unit of work (consumer) becomes visible to the broker as protocol interactions and state plus one or more TCP connections. If parallelism comes in part from the client itself, the unit of parallelism could be a virtual thread, an async task or even an OS thread. This is invisible to the broker. You need fewer consumers, fewer TCP connections, and less broker-visible protocol interaction/state.  This split of where the unit of parallelism is accounted for, broker-side vs client-side, exists across all messaging systems. It’s not specific to Kafka. A simple calculation for aggregate parallelism is easy: 60000 msg/s * 1s = 60000 60000 msg/s * 5s = 300000 100 msg/s * 20s = 2000 10000 msg/s * 0.5s = 5000 50 msg/s * 5s = 250 Once you know how many messages must be processed in parallel, you can figure out your tactics. The formula tells you how much parallelism you need, then it’s up to you to figure out where that parallelism should live. Let’s use our 60,000 messages per second workload from the share group series. If it takes 1 second to process each message, then we need to support 60,000 messages being processed at any given moment. If each unit of parallelism is a serial consumer, then that means 60,000 consumers! That’s a lot of connections, a lot of protocol state, and a really big consumer group. What if it takes 10 seconds on average to process a message, you’d need 600,000 consumers, and well over 1 million TCP connections! If most of the work is I/O, and the CPU spends a lot of time waiting around then can’t we make a single client do more work? What if one client can handle processing 1000 messages in parallel? Then we’d only need 60 consumers for the “60K msg/s + 1 second processing time” example.  Fig 1. Left: Parallel work across N serial consumers. Right: Parallel work across N parallel-capable consumers. If the ultimate unit of parallelism is visible to the broker as something it must manage, it can get really expensive in resources for highly parallel workloads (no matter which messaging system you use). Managing virtual threads, or even OS threads, is much cheaper than managing one or more TCP connections + metadata per unit of parallelism. This is true of all messaging systems I have ever used. The cost is greater complexity on the client, but if you don’t want to roll your own logic, there are libraries to help here (see Chuck’s post for some). Unfortunately, the ParallelConsumer library is no longer being maintained (though a fork might be in the future). This library not only added internal client-side parallel processing but queue semantics as well (on top of consumer groups). Now that we have share groups, perhaps we need a new library that adds client-side parallelism to share groups. I’m going back to writing Part 3 of my parallelism in share groups series. We’ll be comparing broker-managed vs client-managed parallelism with share groups and consumer groups. 60000 msg/s * 1s = 60000 60000 msg/s * 5s = 300000 100 msg/s * 20s = 2000 10000 msg/s * 0.5s = 5000 50 msg/s * 5s = 250

0 views