Nelly the pup absolutely loving life in the field! 🤣 Thanks for reading this post via RSS. RSS is ace, and so are you. ❤️ You can reply to this post by email , or leave a comment .
Mathematicians are broadly not anti-AI. They’re more culturally open to using AI as a tool than, say, artists or writers 1 . However, now that more and more genuinely prestigious problems have fallen to AI, that might be changing. Almost five thousand mathematicians (including twenty-five Fields medalists) have signed a declaration called A Severe Misalignment of AI in Mathematics . The core argument goes something like this: In recent months, the success of AI in solving major mathematical problems has made headlines even outside mathematical circles. But solving problems is only a tool and proxy for achieving the primary goal of conceptual understanding and insight. Forgetting this in the world of AI may turn the tool against the primary goal. Indeed, the mass production at faster and faster pace of “true/false” statements could destroy fertile ground instead of breathing life into new ideas. A lot of people online have interpreted this as the expected complaint from any field that gets automated: translators did it, artists and programmers have been doing it, and now it’s the turn of the mathematicians. I think this is too dismissive. Understanding the concrete problem mathematicians are upset about can help us better understand the impact of AI on our own fields, and what we’ll have to do about it. There are two types of mathematics. Most people are familiar with the first, which we might call “puzzle-solving”: you take a problem and try to find a solution to it. When you’re a student, these problems are typically easy, like simplifying some algebraic expression. When you’re a researcher, these problems can be nearly impossible, like proving Fermat’s Last Theorem . Puzzle-solving is easy to understand but hard to do, which makes it impressive to non-mathematicians, which makes it highly prestigious. In other words, puzzle-solving is legible . The second type of mathematics is “idea-generating”: coming up with new ways of thinking about mathematics, and thus new terms or concepts. For examples of these, just glance down the list of arXiv mathematics papers . “Hardy spaces”, “Schatten exponent”, “Banach lattices” and so on are all concepts someone thought was interesting. This work is largely unimpressive to non-mathematicians, because nobody really knows if the concepts you come up with are particularly difficult or insightful. For instance, I have just generated the concept of a “Goedecke set”, which is the set of all natural numbers whose digits add up to a prime number. Who cares? The categories we want are the “natural kinds” of mathematics — the concepts that “carve nature at its joints” — and it’s almost impossible to tell what those are without years or decades of hard work. How are the two types of mathematics related? We might say 2 that generating ideas is the real intellectual work of mathematics. Puzzle-solving is important instrumentally: to identify which ideas can be used to answer longstanding questions, and thus which ideas are worthwhile. Over time, those worthwhile ideas become better understood and easier to use, until they reach the point where they can be used to advance science in general. Eventually the ideas become so well-understood that they can be taught to children: “zero”, “negative numbers”, “imaginary numbers” and “calculus” were all once rarefied mathematical ideas, but are now concepts we’d expect any precocious twelve-year-old to grasp. There’s another, more prosaic purpose of puzzle-solving: to make mathematical skill and progress legible to outsiders. I can’t appreciate Terence Tao’s mathematical work, but I know what a Fields Medal is. I don’t have a good intuitive sense of what a Galois representation is, but I know about the proof of Fermat’s Last Theorem . We might say that puzzles like this have served as a way to indirectly reward skilled mathematicians for their more important idea-generating work (or for conclusively demonstrating 3 that the ideas used in the proof are useful). AI proofs undercut both of these purposes. I can now lay out precisely why I think mathematicians are so unhappy: This is kind of like Goodhart’s Law . Puzzles were a useful, impossible-to-game measure for mathematical progress. But now that AI companies can game that measure (by solving them in a way that’s inaccessible 4 to humans), the whole point of those puzzles disappears. Are the mathematicians right? I think it’s broadly unclear whether (3) is true: i.e. whether frontier AI models aren’t generating or can’t generate new mathematical ideas. We’re still in the very early days of AIs solving our hardest mathematical problems. Who knows what they’re going to be capable of? I give basically zero credence to the idea that AIs are incapable of this because of some intrinsic feature of how LLMs work. For the last three years, we’ve seen people claim that LLMs are intrinsically incapable of X, only to have LLMs excel at X a few months later. Even granted that (3) is true, there’s still work to be done for human mathematicians in building the conceptual machinery that can make AI-generated proofs accessible to humans: i.e. in generating a “human proof” to go alongside the existing “AI proof”. In fact, I’d expect the existence of an AI proof to help with this. If you know proposition X is true, it’s easier to figure out why, because you’re not constantly worried you’re wasting your time. For more on this, I recommend Gwern’s blog On Really Trying , where he quotes a series of instances where simply being told that a solution exists is enough of a clue to help people find it. Of course, there’s a prestige and motivation problem. “I’m the first person to solve Navier-Stokes” is a much more compelling target than “I figured out a better way to explain the AI solution to Navier-Stokes”, and it’s much easier to award prizes for. Will mathematicians bother to work on problems that have already been solved? I think so. To see why, we can look at other domains where AI has come in and outcompeted the best humans, such as chess or video game speedrunning. I can run a chess program on my phone that will beat Magnus Carlsen 100-0. Computer programs — called “tool-assisted speedruns” or “TAS” — can finish any video game much faster than even the fastest human. But in both of these areas, humans still compete in human-only leagues, and there’s still prestige attached to the most capable humans. It’s possible that mathematics ends up in this kind of state, where “human mathematics” and “AI mathematics” exist in largely separate spheres, and the first “human” solution to a mathematical problem can still earn acclaim. In fact, in both of those areas, the presence of inhumanly strong computer players has improved the human game. Despite many computer chess moves being basically incomprehensible to humans, top chess players have learned from the computer “style”. In speedrunning, many moves once considered “TAS-only” are now performed by humans. AI mathematics might likewise improve human mathematics. I am not a mathematician. I did major in mathematics during undergrad, and I have fond memories of proofs from real and complex analysis, but it’s not even close to my field. However, I am watching the effects of powerful AI on mathematics very closely, since my own field — software engineering — is being colonized by AI agents in the same way. The field of software engineering does not have the same structure as mathematics. We write code to make money, not to earn prestige or advance the frontier of human knowledge. But AI is undercutting the traditional avenues for prestige in software engineering as well. It used to be that you could put a meaty project on your GitHub — say, an emulator, or a toy OS — and people would know you were a skilled engineer. But now projects like that are worthless, because everyone just assumes they’re vibe-coded. We used to tell stories about engineers who would disappear and rewrite a system over the weekend, or produce thousands of lines of code a day. Now anyone can do that with an OpenAI subscription. Like mathematics, software engineers are going to have to rebuild our cultural sense of the kind of work we value. We are either going to have to silo “AI work” off from “human work” like chess, or to find some legible human skills to recognize that can’t be easily counterfeited by AI. In the meantime, a lot of people who were successful in the old world are going to be very unhappy. Possibly because current AI models are much better at mathematics than at art or writing. Again, I am not a mathematician: here I am interpreting what I’ve read from Terence Tao and other mathematicians. This kind of idea-sharpening or idea-validating is part and parcel of idea-generation, and just as important. It’s interesting to compare mathematics to, say, philosophy, which has the same idea-generating task without the corresponding puzzles to validate the ideas. That’s one reason why philosophy has less prestige than mathematics. A thousand-page Lean proof is theoretically understandable by humans, but if no mathematician can hold the entire idea in their head it doesn’t matter. Puzzles serve as a high-legibility, high-reward target for mathematicians To solve these puzzles, new ideas must typically be generated; the puzzle’s solution serves as evidence that the ideas are useful But now AI can solve many of these targets “the hard way”, without generating intuitive new ideas This undercuts both ways puzzle-solving supports idea-generation: AI companies claim the prestige while not meaningfully advancing mathematical progress This is bad for mathematics as a whole, because puzzle-solving is ancillary to the real goal of mathematics Possibly because current AI models are much better at mathematics than at art or writing. ↩ Again, I am not a mathematician: here I am interpreting what I’ve read from Terence Tao and other mathematicians. ↩ This kind of idea-sharpening or idea-validating is part and parcel of idea-generation, and just as important. It’s interesting to compare mathematics to, say, philosophy, which has the same idea-generating task without the corresponding puzzles to validate the ideas. That’s one reason why philosophy has less prestige than mathematics. ↩ A thousand-page Lean proof is theoretically understandable by humans, but if no mathematician can hold the entire idea in their head it doesn’t matter. ↩
I used a public Linux kernel bug to make one CPU keep using a memory mapping after another CPU had removed it. I then persuaded Linux to recycle the physical page behind that stale mapping as a page table. My unprivileged process could still write to the old page, while Linux now treated the same bytes as part of its memory map. That chain produced one real flag from Google’s kernelCTF service. This was a 1/n day exploit, not a new vulnerability: the bug, reproducer and fix were already public. My work was the long and occasionally silly trip from “stale read” to a complete exploit. (It took me a too mcuh time to write this blog in a way it is useful, soemthign what I would have prefered from other blogs, Feel free to let me know if it can be improved and written more reader/begginer friendly) Authorized lab only: I tested Google’s kernelCTF target and an isolated local QEMU virtual machine with networking disabled. Do not run kernel exploits on everyday computers, shared machines, production systems or anything you do not own and have permission to test. I had some free time (again), so I started writing a kernel exploit as a hobby project. This is apparently what passes for recreation once normal hobbies stop looking complicated enough. The previous kernel exploit I had developed was during a training class, It was useful and I learned a lot, but there is a small problem with learning kernel exploitation: if you do not practice afterward, the knowledge does not remain neatly arranged in your head and mine had become a box of unlabeled dataset. I remembered the broad ideas that there are pages, page tables, allocators, races and many ways to crash a virtual machine. The precise details had quietly left the building. This work is also not my day job I do not do offensive security or exploit development professionally and learning this area takes a lot of focused free time. Reading one function often means reading five structures, three commits and a mailing list thread before realizing the first function was not important. My first instinct was to re read all the theory until I felt ready (That is a comfortable trap) There is always one more memory management sections to understand and “ready” can remain three chapters away forever. So I decided to choose a real finish line instead. I wanted an exact kernel, a public bug, an objective success condition and nowhere to hide behind “the idea should work.” If the exploit did not cross the boundary and produce the expected result, it did not work. Google kernelCTF is a continuous Linux kernel exploitation challenge. Google boots an exact kernel build inside a virtual machine and runs the submitted exploit as user ID or UID, . UID 1000 is an ordinary user, not root. The protected flag is available to root in the initial guest environment, outside the challenge process’s NsJail sandbox. The job is to begin inside that restricted process and gain enough control of the guest Linux kernel to reach the flag. That boundary: QEMU runs the guest, but I did not escape QEMU. Google’s host was not a target. The outer jail was not a target The solid line in this diagram is the entire claim. My local lab reproduced the important guest details so I could reboot, add logs, attach a debugger and compare two kernels without wasting the official service’s time. A local root shell or synthetic flag is evidence that a chain is developing. Only Google’s endpoint can return an official kernelCTF flag. The flag could not be impressed by a crash, a screenshot or a paragraph explaining why the final stage probably worked. There are also two common labels worth clearing up. A zero-day is a vulnerability not publicly known to defenders when exploitation starts. A one-day uses a bug or patch that is already public. This project was firmly a 1/n day I was learning how to exploit a known defect. Each kernelCTF submission receives an ID. Mine became An earlier submission, , already occupied the version/base slot for , so mine was not the first submission for that base That changes reward and review possibilities, but it does not change whether returned a flag. Google can separately review a submission for a novel exploitation technique and for reliability. Public evidence shows one official capture for . It does not reveal which numbered run succeeded or a success percentage, so I will not manufacture either number. “One flag” is both less exciting and more accurate. I wanted a target with a public patch and a public reproducer. That gave me somewhere concrete to begin while still leaving the exploit engineering unsolved. The target I chose was tracked as CVE-2025-37964. The vulnerable build was , at exact commit . The fixed comparison was , containing stable commit . The public report and reproducer showed stale memory access caused by an ordering problem in x86 Translation Lookaside Buffer handling. The actual vulnerable component is the target’s . That sentence is important because the eventual exploit uses enough other kernel machinery to make blame wander around. The naming became stranger than the code. Stable 6.12 had an attempted backport related to upstream commit , but operations relevant to the protocol landed in the wrong order on that branch. The stable commit corrected that order. The current public CVE records do not map the stable-only correction cleanly. CVE-2025-40174 appears in later paperwork and discussion, but I am not relabeling the exploit as definitely targeting that CVE. The code relationship between and is clear. The administrative relationship between the CVE records remains unresolved. I started by reading the patch and immediately met , , , and PCID. This was the point where my box of training-class cables tipped onto the floor. The kernel documentation helped me rebuild the picture. The kernel is the trusted part of the operating system that schedules work, manages memory, talks to hardware and enforces access between programs. My exploit begins in user space , where normal applications run with limited rights. When a program needs the kernel to do something privileged, it uses a system call , usually shortened to syscall. Calling to request memory, to release it or opening a file all cross that user/kernel boundary through syscalls. I kept one imaginary program in mind while reading I Call it Maple, Maple asks Linux for one writable page and stores there. Maple receives a virtual address, perhaps . That address belongs to Maple’s process address space , the private-looking range of addresses it can use. Inside the target kernel, the whole address space is represented by , while regions within it are represented by . Threads that share an address space share the same . When Maple requested one mapped region, Linux recorded a Virtual Memory Area or VMA. A VMA says that one continuous address range exists and has particular properties: writable or read-only, file-backed or anonymous, and so on. The official process-address documentation describes the VMA as the metadata for the range. It is not the physical memory itself. That distinction initially felt fussy, then became central to the exploit. Linux can retain a VMA while removing the page-table entry for one page inside it. The region still exists as an idea, but the route to the physical bytes is temporarily gone. Linux divides virtual and physical memory into pages. On this target, the pages I cared about were 4 KiB. Maple’s virtual page is connected to a page in physical Random Access Memory , or RAM, through a hierarchy of page tables. The Linux page-table documentation explains the real multilevel walk; my working picture was simpler: The final Page Table Entry or PTE, stores the physical page number and permission bits It is one row in the official map. The CPU follows those rows when Maple reads or writes V. Following several page table levels on every memory access would be painfully slow, so each CPU core keeps recent answers in a fast local cache called the Translation Lookaside Buffer or TLB. The page table says where V officially goes. The TLB lets a core remember “V goes to P” without walking the map again. Modern x86 processors can cache translations from multiple address spaces. A Process-Context Identifier or PCID, tags entries so the CPU can tell which address space an answer belongs to. The exact target’s and track which memory map and TLB generation a CPU believes it has loaded. Now Maple runs on CPU0 and CPU1. CPU0 has cached . CPU1 removes Maple’s PTE because Maple asked Linux to discard that page. The page table no longer contains the route, but CPU0’s hardware cache does not read minds. Linux must make CPU0 forget. That coordinated invalidation is a TLB shootdown . Usually Linux identifies processors using the address space and sends the needed notification, often through an inter processor interrupt. When the protocol works, CPU0’s next access cannot use its old shortcut. It walks the current table and sees that the page is absent. The bug was a schedule where both ways of noticing the change were missed. I found it easiest to picture two people updating a sign while passing through a doorway. CPU0 was switching from a helper task into my reader task. CPU1 was removing the reader’s PTE. Linux maintained both a generation number saying how new the address space was and a CPU mask saying which cores were using it. Each core looked at a piece of state that was reasonable at that instant. Together, those observations created a hole. CPU1 missed CPU0 when selecting remote recipients because CPU0 had not yet published the new map. CPU0 missed the newer generation locally because it had already sampled the old value. The remembered translation survived between those two decisions. The race sits around and the /TLB-generation protocol in the exact . documents the generation invariant. The stable fix restores the ordering; its compiler is not a magic cross-CPU hardware fence. I could observe stale access on . The same phased test ran two million attempts on without reporting stale access. That is a useful vulnerable-versus-fixed comparison, not a proof about every schedule on every machine. The reproducer had answered the first question. CPU0 could still read P. But P still belonged to my own mapping, and an unusual route to my own bytes is not root. Then came the useful thought: the TLB remembers a physical destination, not the reason those bytes were allocated. If Linux releases P and later reuses P as a page table, CPU0’s stale route still ends at P. My process thinks it is writing an old data page. Linux thinks those same bytes describe mappings. That was the whole exploit in one diagram. Unfortunately, there were many ways to fall between the boxes. I began in the traditional way: two threads, two CPUs and optimism. The writer on CPU1 removed the mapping while the reader ran on CPU0. Millions of attempts later, optimism had produced a very stable failure. The public reproducer showed that the bug existed, but my exploit needed one particular context switch. CPU0 had to move from a helper’s address space into the reader’s address space during CPU1’s removal. Random wakeups did not reliably create that edge. I changed the scheduler shape instead of merely increasing the attempt count. The helper and reader were pinned to CPU0; the writer was pinned to CPU1. The reader slept and the helper woke it near the measured point where CPU1 would remove the PTE. The sleep/wake mechanism was a fast userspace mutex or futex . A futex lets a thread wait efficiently in the kernel until another thread wakes it I was not controlling the scheduler, I was arranging the room so the helper to reader switch became the obvious next move. Direct wakeups and an unphased helper each completed recorded campaigns of two million attempts without a stale result. Scheduler shaping produced the stale oracle The answer was not more lottery tickets, It was changing which lottery I was entering. Most attempts still had to fail and that created the next practical problem. Every try needed to remove a PTE, test for stale access and restore the page for another try. After a win, however, I needed to stop restoring it and eventually release the physical page. I chose one page from , a Linux interface that creates memory with special secret memory handling. The useful property here was not secrecy It’s lifecycle control. During retries, with could remove the page’s PTE while the underlying memory object stayed alive. A losing read could fault the page back in and prepare another attempt After a stale win, I could take a one time path toward releasing it. Linux calls the bookkeeping object for one or more physically contiguous pages a folio . My carrier was one page, so “secret folio” mostly meant “the kernel object still owning these 4 KiB.” The local in this security_flow checkout is an older safe milestone with an eight-page oracle pool. It does not reclaim the page or write a PTE. The immutable complete source contains the one-page chain. My first success check was embarrassingly trusting: put known bytes in the page, remove the mapping and celebrate if CPU0 reads the known bytes. There was another explanation, CPU0 might take a normal page fault. Linux could rebuild the PTE and return exactly the bytes I expected The output would look successful while the stale translation had never survived. So every attempt received a changing marker Then I checked , the kernel interface that reports properties of a process’s virtual pages. An oracle is just an observable test for hidden state mine combined the marker with the present bit. The sequence I accepted was strict: pagemap said present before removal, absent after removal, CPU0 returned the current attempt’s marker, then pagemap still said absent twice. The marker proved the bytes were current. The absent checks proved a normal fault had not quietly recreated the PTE. At this point I had the stale route I had wanted, I assumed releasing and reclaiming the page would be the easier part. This was incorrect in a way the allocator was happy to demonstrate for quite some time. I won the race, unmapped the page and allocated replacements. Nothing reused it I allocated more, Still nothing my first response to an allocator mystery was, naturally, more spray. It was the exploit-development version of pressing an elevator button harder. Eventually I stopped trying to persuade the allocator and traced the page’s lifetime. Tracefs showed an invisible temporary reference in CPU0’s , a deferred batch used for locked-memory accounting. Linux still had one hand on the folio, so it could not recycle the physical page regardless of how much memory I sprayed around it. The hidden reference came from refaulting the locked secret memory VMA during retries. This was where the earlier VMA distinction stopped being vocabulary and became the answer: the mapping’s region remained valid and faulting inside it could put the folio into deferred accounting. On this exact target, a page-aligned beyond the end of reached and emptied CPU0’s deferred batch. I performed that drain before the race. A successful stale read then avoided another refault, so it did not add the hidden reference back. The allocator had not been random or stubborn. My ownership model had been wrong. Draining the reference made release possible, but release usually comes with a dangerous side effect. removes a VMA and normally participates in invalidating translations. A fresh flush would erase the stale TLB entry I had spent all this effort preserving. I needed the final unmap to complete the ownership paperwork without finding anything left to clear. The race had already removed the leaf PTE. I kept neighboring mappings alive in the same Page Middle Directory or PMD, region, then unmapped only the one page secret VMA. On this target, one PTE page contains 512 entries and covers a 2 MiB PMD region. The neighbors kept the surrounding page-table structure alive. With no leaf entry left and no enclosing table becoming empty, the final had no useful translation left to invalidate. CPU0 could then watch the physical page being cleared through its stale route. That runtime observation became the gate before reclaim. This was fragile. A context switch, unrelated mapping change, pending flush or PCID rollover could still destroy the stale entry. The exploit checked that the route survived rather than assuming the diagram had authority over the machine. Now I could release P while CPU0 retained . The next question was how to make Linux reuse exactly P as a PTE page, then identify which PTE page it had become. I reserved a sparse 1 GiB virtual arena. “Sparse” means the address range was large without immediately consuming 1 GiB of physical RAM. I created its upper page-table levels first. Later, touching one address in each new 2 MiB region mainly forced Linux to allocate PTE pages. This is allocator grooming : arranging allocations and frees to influence what receives a reusable chunk. A Page Frame Number , or PFN, identifies a physical RAM page. I wanted the old secret-page PFN to return as the PFN of one arena PTE page. PTE-shaped bytes were not enough. Random values become persuasive after a long debugging session. I needed each candidate table to identify its owner. I encoded the arena region number with pairs of PTE slots. In each pair, one populated slot represented a zero bit and the other represented a one bit. Start and commit anchors showed that construction had completed. Slots that should be empty also had to remain empty. CPU0 took two full 4 KiB snapshots through the stale alias and required them to match. The decoder checked the anchors, pair code, common zero-page frame, permissions, physical range, and required empty slots. Only then did I accept both “this is a PTE page” and “this is the PTE page for arena region N.” One PTE page maps 2 MiB, so the 1 GiB arena offered 512 candidates. On this target the exploit retained 256 conditioner tables and resumed candidate allocation at region 257. Those numbers are measured behavior for this build, not laws of the Linux allocator. Recognizing a page table was exciting, but the bytes could still have been a convincing coincidence. I wanted a harmless proof before using a system wide target. The exploit therefore has a separate mode. It allocates a canary page containing known data and reads that page’s legitimate PTE through the reclaimed stale window. Then it copies the entry into a destination PTE slot. The destination address had never been accessed, or walked , by a CPU. That detail mattered. If a CPU had already cached a translation for the destination, changing its PTE would require another invalidation and that invalidation could also destroy my stale window. An untouched slot had no old TLB answer to forget. The canary became readable through both virtual addresses. A write through one appeared through the other and the raw entry matched. The exploit then cleared the forged entry and checked cleanup. That was the page-table-control proof. It ran in mode. The official capture came from a separate mode. They were not two stages observed in one lucky official run, and I do not want a smooth story to blur that boundary. For the authorized challenge finish, I used a known technique involving . Linux consults this setting when a process crashes and generates a core dump. If the pattern begins with a pipe, the kernel launches the named helper with root privileges. This finish was prior art, not my new idea. The interesting work in my chain was reaching controlled PTEs reliably enough to use it. PTEs in my fingerprint pointed to Linux’s shared read-only zero page, which revealed the physical location of that known page. Symbols from the exact gave the fixed relationship between and the physical page containing . I installed a writable mapping for that target page in the never-walked destination slot. CPU1 wrote and read back a NUL-terminated helper pattern. A copy of the exploit was already held in memory-backed file descriptor 666. After a controlled crash, the root helper used the supplied process information to recover the required descriptors and read only the challenge flag. Before triggering it, the exploit cleared the forged PTE. The helper later made a best-effort reset of to . It did not save and restore the previous value, and a reset failure was silent. “Best effort” is the honest description. The exploit validates physical-address width, page alignment, guest RAM bounds, PTE flags, kernel physical alignment, and two equivalent symbol calculations before writing. On this target identifies the crashing process in the initial process namespace and supplies a process file descriptor. The helper uses to recover descriptors. Locally, the chain now had a harmless proof mode and a separate flag mode. I packaged the official archive and expected the race to be the unreliable part. The archive refused to race at all. My first official archive checked for an invariant Time Stamp Counter or TSC. The TSC is a fast processor clock and I used it to coordinate the two CPUs. The code required the processor feature bit promising that the counter behaved consistently. Google’s CPU profile hid that feature bit. The exploit exited before making one attempt. The check had confused a label with the property I needed. The two virtual CPUs’ counters could still be usable together even when that advertised bit was absent. I replaced the feature-bit assumption with measurement. The corrected code exchanged timestamps between CPUs 32 times, used the minimum round trip to estimate offset, bounded the result, and required at least 75 percent of samples to agree within a latency-based tolerance. A preserved local run measured an offset of cycles while reporting . The corrected archive went on to produce the one official flag capture. That proves one official capture. It does not reveal a run ordinal or establish a success rate. The stale-only local milestone still requires invariant, constant, nonstop TSC; the immutable submitted exploit contains the measured cross-CPU version. After all those detours, this is the end to end path without introducing anything new. Normal exploit logs told me what my program believed: markers, pagemap state, decoded arena region and which checks passed. They were cheap enough for repeated race attempts, but they could not explain an invisible kernel reference. The GNU Debugger or GDB, let me stop a virtual CPU inside functions such as and inspect registers and control flow. It was excellent for understanding a path and terrible for preserving race timing. A breakpoint can make the execution you wanted to observe disappear simply by observing it. Tracefs is Linux’s event tracing interface. It records selected kernel events while the system continues running. Allocation, free and drain traces answered “who still owns this page?” more directly than another breakpoint could. The two kernel files in the lab also have annoyingly similar jobs and very different forms. is the compressed bootable kernel image QEMU loads. is the uncompressed kernel executable with symbols and, when available, debug information. GDB needs the exact matching the booted ; a nearby build can produce beautifully named nonsense. Tracing and debugging both distort the experiment. GDB pauses CPUs. Tracefs changes timing and memory activity. I used them to understand mechanisms, then returned to fresh untraced boots for race evidence. The local checkout contains only the stale milestone. The complete , , and implementation and build integration are in the public PR branch. A runner means the virtual machine booted and output was captured. The security result is on or on . Networking remains disabled. Start a debug copy of the matching QEMU command with . The first option opens a local debugger port; the second pauses the guest before execution. These breakpoints are for learning the path. Remove them and use fresh boots for timing observations. Check that optional event paths exist before enabling them. This privileged diagnostic run is not the UID-1000 exploit proof, and tracing can change both timing and allocation behavior. Do not fabricate output. Do not publish the signed flag or the sensitive root screenshot. The public reproducer proved stale access but not the page lifecycle I needed. Random wakeups lost millions of attempts, which forced me to shape the helper to reader transition instead of running longer. My first marker could not separate a stale hit from a normal fault; pagemap made that claim testable. More spray could not beat ; tracefs found the hidden reference. Finally, the first official archive trusted a missing TSC feature bit and never raced; measuring the clocks fixed it. None was cinematic Together they were most of the work. The vulnerability was public, Stale-TLB exploitation after physical-page reuse was public. Generic freed-page-to-PTE conversion, zero-page anchoring, and the helper finish all had prior art. Project Zero’s stale-TLB work and page-use-after-free to PTE work are useful background. My narrower contribution was fitting a retryable one-page secret memory carrier, changing marker plus pagemap oracle, deferred-reference drain, empty final unmap, conditioned PTE allocation, pair-coded owner fingerprint, separate harmless canary proof and never-walked destination into one end-to-end chain for this exact target. I searched Google’s public kernelCTF corpus at commit from 2026-08-06. I found no match there for , , , this CVE, or the pair-coded protocol. That supports a novelty case; it cannot search private work or every unindexed experiment, and Google decides the category. The complete submission is public in Google security-research PR #418 . The PR is open and unmerged. Its immutable head is . Pinned links are available for the full exploit source , Makefile and kernelXDK integration , and technical documentation . The repository review source is 3,226 lines. The inside the submitted archive is 3,179 lines because the review copy includes the kernelXDK symbol-wrapper route. Both counts are correct for different copies. The successful archive was named in my private working tree and in the public submission layout. Its SHA-256 is: It contains exactly , , and . The earlier archive that stopped at the TSC check has different bytes and a different hash. PR #418’s checks and discussion expose the CVE mapping dispute. The technical stable correction is ; the public CNA and spreadsheet mapping remains unresolved, with CVE-2025-40174 complicating the record. Public code does not mean the PR has been reviewed, merged or that the metadata has been adjudicated. has public evidence for one official flag. already occupied the version/base slot and that fact alone does not reveal which vulnerability used. The remaining review question was the discretionary novel-technique category. A working exploit, a first slot and a reward are three different facts. The finished chain can be compressed into one sentence: one CPU kept an obsolete route, Linux recycled the destination as a page table and my ordinary process edited the map. Building it felt nothing like one sentence. I had to relearn memory-management basics, make the scheduler produce one edge, distinguish a stale hit from a fault, find a reference I could not see, make an unmap boring, label hundreds of page tables, preserve one untouched destination and finally replace a confident clock check with an actual clock measurement. The flag was a clean finish line, but the useful part was being forced to answer each question with evidence. Rereading theory would have made me feel less rusty. Watching a supposedly free page refuse to move taught me why it was not free. The honest ending is still the scoreboard: captured one official flag; had the base slot; PR #418 is public and unmerged; the CVE metadata is unresolved. No new bug, no invented reliability number, no mysterious years of kernel wisdom. Just some free time, a lot of rebooting, and one CPU that forgot to forget. If you are still reading this, you are awesome. Thanks for sticking with me! The exact two-CPU guest starts my exploit as UID 1000 inside the challenge jail. The exploit reserves a sparse 1 GiB arena and prepares page-table candidates that can later identify themselves with pair coded region numbers. It creates one page, writes a changing marker, and drains CPU0’s deferred locked memory batch so no hidden reference keeps the folio alive. A helper and reader share CPU0 while the writer runs on CPU1. Futex wakeups shape the helper-to-reader switch around the writer’s operation. In the winning interleaving, CPU1 removes the PTE but does not include CPU0 in the shootdown. CPU0 has already read the old generation and skips its local flush. CPU0 reads the current marker through its stale TLB entry. Pagemap says the PTE remains absent before and after the read, ruling out a normal fault. The exploit unmaps the already-empty one-page VMA while keeping neighboring mappings alive. The physical page is released without an effective invalidation, and CPU0 verifies that its stale route survived. Touching conditioned arena regions makes Linux allocate PTE pages. The old secret page is recycled as one of them. Two identical snapshots plus anchors, zero-page entries, permissions, empty slots, and the pair code identify the owning arena region. In a separate run, the exploit copies an owned canary PTE into a never-walked destination and proves both aliases reach the same page. In mode, the exploit instead maps the physical page containing , writes and verifies the helper pattern, clears the forged entry, and triggers the helper. The helper reads the protected flag and attempts a best-effort reset. Google’s service records one successful flag for . Exact release and UID 1000 in QEMU. GDB stopped in with matching symbols. Pagemap showing present, absent, current marker, still absent. Tracefs showing the relevant physical page’s release and allocation. Separate harmless output. Local stale marker and fixed comparison. Redacted ledger entry containing no secret data. Write a tiny C program that opens and reads a file. Run and watch the syscalls cross from user space into the kernel. Allocate one page with , write a value, and inspect . Connect the address range to a VMA before worrying about PTE bits. Draw the path from virtual page to PTE to physical RAM to TLB. The official page-table and process-address documents are good companions once the picture is clear. Write two harmless threads, pin them to CPUs, and use a futex to change their wakeup order. Observe timing; do not begin with a kernel bug. Boot a disposable QEMU guest with networking disabled. Learn which file is , which is , and where the guest/QEMU/host boundaries sit. Attach GDB to the guest and stop at a harmless syscall. Then enable one tracefs event and connect one trace line to one action in your program. Read a public patch and reproducer. Try to state the broken invariant in one sentence before building anything. Build evidence in small rungs: controlled observation, controlled read, controlled write, harmless canary, then the authorized target outcome. Count failed attempts and preserve raw logs. A success proves exploitability; it does not retroactively make every failed run disappear. Keep vulnerable-kernel work inside an isolated system you own or are explicitly authorized to test. Linux kernel documentation kernelCTF rules Linux CNA record for CVE-2025-37964 Public CVE discussion and reproducer Original upstream ordering fix Correct 6.12.y stable fix Exact vulnerable x86 TLB source Exact target memory-management structures Exact target x86 TLB state structures Linux page-table documentation Linux process-address and VMA documentation Linux pagemap documentation documentation Project Zero stale-TLB exploitation precedent Project Zero page-UAF-to-PTE precedent Google’s public kernelCTF exploit corpus Public submission PR #418 Immutable full exploit source Immutable exploit documentation
Here's a neat thing I had ChatGPT Work with GPT-6 Astra (Max) do this morning: It worked for 27 minutes and produced exactly what I'd asked for, as both an embedded visualization and downloadable GPX file and GeoJSON files. Here's that 5K route: When I asked it how it had created the route, it replied: I used Nominatim to locate the address and Overpass to download local OpenStreetMap roads and trails , then calculated the loops locally. Frustratingly, the actual code it ran and exact details of what it did weren't visible to me in the ChatGPT UI. I see this lack of transparency is an anti-feature. By the time I thought to ask for a copy of the Python code it had used, ChatGPT was unable to provide it. This appears to be because the thread had been compacted. I think any LLM system that uses compaction needs to both preserve the pre-compacted text and make that text available via agent tool calls, to protect against this kind of problem. As for displaying the map to me, that used the visualize skill . It created a file called to embed directly into the ChatGPT UI. Here's a copy of that HTML , which starts like this: The element contains the full geometry needed to render both the running route and the map itself, using D3, which is loaded from an allow-listed CDN location described in this section of the visualize skill : You are only seeing the long-form articles from my blog. Subscribe to /atom/everything/ to get all of my posts, or take a look at my other subscription options . The CSP allows only , , , , , , and . Other origins are blocked and fail silently.
From its earliest days , Macs established an interesting convention – whenever you press a keyboard shortcut to an action that’s somewhere in the app menu, the matching top menu label blinks quickly. Here, I am pressing ⌘A (Edit > Select All), followed by ⌘+ (Format > Font > Bigger), and then ⌘B (Format > Font > Bold): I believe this is meant to help you connect those things better. While you might not need a map to an action that you already know a shortcut for, it might be helpful to tell you where to find other actions like it. I imagine it also helps whenever you press a wrong shortcut – or the right shortcut under the wrong circumstances – and you want to deduce what happened or what was meant to happen. Knowing roughly where a command “lives” makes it easier to open the menu and look for it, even if you might still have to dig through all the submenus. Recently, I spotted the programming editor Nova use a parallel technique. In its command palette, commands show their keyboard shortcuts, but if they’re hiding in submenus – also their menu path: = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/novas-menu-wayfinding/2.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/novas-menu-wayfinding/2.1600w.avif" type="image/avif"> = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/novas-menu-wayfinding/3.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/novas-menu-wayfinding/3.1600w.avif" type="image/avif"> I am not sure how effective either of these techniques is, and both can feel a bit… busy. But they also appear thoughtful – we’re all spatial creatures, and I imagine helping you visualize a map of the entire system of commands can make it easier for you to feel at home.
I finally built Mathy, a little project I’ve been thinking about for a couple years. It’s free, and no account is required if you want to try it out. I use Math Academy daily, which is the best way to learn math. But I wanted a mobile-friendly way to drill math and weak areas on the go that was more convenient/less cumbersome than Anki cards. It’s also a nice way to do something…
Yes, this is a blog post about the The Legend of Zelda: Ocarina of Time remake. In part inspired by Christian Donlan’s I’m trying to untangle my response to how the Ocarina of Time remake looks on EuroGamer and in part because of my huge initial distrust when I saw the first real gameplay trailers. What exactly happened in my head was something along the lines of: Since that initial shock, I’ve mostly come around. Well not really, but good enough to tolerate it’s a remake, not a remaster, and it’ll please the youngsters who don’t even know what proper Zelda gameplay is, since Breath and its direct successor don’t even feature proper dungeons. Young Link running around in the Hyrule fields. Where you can actually see stuff in the distance. Crazy. Let’s take a step back. I’ve been mulling over the purpose of a remake/remaster (let’s keep the distinction out of this discussion for now) and came to the conclusion that the reception of games like these can be categorised in the following taxonomy. On the one hand, you’ve got people who never played the game before. They are either young enough to never have witnessed the original (and have not heard enough about it to care), or people who missed the original release and are now reminded of its existence. This is exactly what a remake is for: to push the original game forward into the future, ready to win the hearts of another generation. In case of OOT , I doubt any self-proclaimed gamer will deny its presence in nearly all top of all time lists, but in the case of the average game from the nineties that since has fallen into obscurity, a remake is a sure way to try and resurrect it from the dead. On the other hand, there are folks who already played OOT . These are the critical annoying folks who’ll tell you Breath of the Wild is not a true Zelda game (look who’s talking). We can further categorise these into grumblers who want to preserve the exact experience from back then and the folks who are open to reliving its core message whilst marvelling at a completely revamped engine or even fleshed out storyline. Given my affinity with nostalgia —good or bad—I tended to side with the grumblers. That’s indeed shrewd usage of past tense. My first raw reaction when seeing Young Link in motion, built out of 3446833 polygons instead of just 10 was definitely a grumbly one. But like I said, I’ve come around. Why do I buy and play so many remakes/remasters? A non-exhaustive recent selection: Why do I buy and play so many retro-inspired new games? A non-exhaustive recent selection: Why do I buy and play so many retro collections? A non-exhaustive recent selection: Clearly, I am the ideal target audience for a game like the OOT remake. Yet my first reaction was scoffing—why? Because that’s not how Link is supposed to look like and how the field of vision is supposed to work on authentic Nintendo 64 hardware? There are a plethora of HD ports out on the PC such as Ship of Harkinian , offering modern controls, widescreen, and other quality of life stuff we’re all addicted to nowadays. Including me. Including me: the first thing I did when replaying Might & Magic VIII , a 26 year old PC game from 2000, was apply a patch that improves the field of vision, adds widescreen, and fixes some of the controls. The first thing I did after installing Ultima Underworld was patching in “modern” mouse controls. It turns out that I’m less of an authentic player than I thought I was—yet I refuse to play Serious Sam HD and stick with my Windows 98 PC for that one. Why? Because Serous Sam is supposed to be low-res that used to be enormously high-res and cutting-edge. The HD version looks too different compared to the original that in my personal context has a tight affinity with the ATI Radeons and GeForces of the early noughties. I didn’t want to alter how Ultima Underworld and Might & Magic VIII looks or sounds —merely how easily I can manipulate its world. Yet if you give me the choice to play Paper Mario 2 on the GameCube or on the Switch, I’d choose the latter in the blink of an eye. This is coming from someone who spent forty hours on both games, in 2004 and in 2024. Stuart Gipp from The Retronauts podcast recently interviewed the editor in chief of UK’s Retro Gamer magazine who was shocked to hear that some people collect PAL versions of console games as opposed to NTSC ones. We in Europe were never treated with good framerates, but why would you not want them, given you can now access everything through the internet? Because they wanted to experience a restorative nostalgic moment; to perfectly recreate their childhood memories. In practice, it’s never that black and white. I like the HD version of Age of Empires II , but I would still play the original on my old hardware if I felt like it. The “remastered” version of Warcraft III is a gigantic trainwreck. I don’t want to touch Diablo II: Resurrected . But that’s just me pretending: my hardware currently can’t handle it. I want to play it so bad: it’s recently got a new playable class—for a gameplay loop that’s twenty-six years old, that’s just crazy! Wait, the game is out on the Switch ?? Yet I’d never want to play the blurry mess that was the GameCube version of Paper Mario again if I can spin up a crispy clear version on the Switch that even comes with cleaned up and in my opinion an improved soundtrack. What about The Legend of Zelda: Link’s Awakening and the Switch remake then? I did not buy it and I have zero intentions of doing so. The aesthetics of the new version are off-putting to me: the original was a (very) pixelated 2D game. The new one technically still is, but the 3D engine full of bloom effects evoke an entirely different feeling . Yacht Club Games’ newest Mina The Hollower successfully evoked the original Link’s Awakening theme, but in doing so, it perhaps turned off a group of youngsters that are used to playing Fortnite . The fact that the engine allegedly stutters in more than a few places, dipping below 30 FPS, does not help either. Link's Awakening Rewakened: Looks like LEGO blocks to me. Where are my pixels? So where does that leave us with the upcoming Ocarina of Time remake? Just set aside that . You know you’re screwed—might as well admit it now. Related topics: / nostalgia / games / By Wouter Groeneveld on 12 September 2026. Reply via email . This is not right, it’s not supposed to look this way! Why is this hyper realistic? Is this a From Software game ported to a finally capable Nintendo machine? Is this Ocarina of Time (OOT) or is this Breath of the Wild , where you can scope out a point in the distance from a tower, glide down and actually visit that place? Go back to point 1: it’s not supposed to look/work this way! Seventy bucks for a game that I already own on multiple platforms including the perfectly reasonable polished 3DS edition? I don’t think so. The chickens are not supposed to have more than three polygons. Now they look like plastic. Where’s the fog? Why can we see that much? Paper Mario 2: Thousand Year Door Mario RPG (remake? Debatable) Soul Reaver Anything from Nightdive Studios such as Rise of the Triad: Ludicrous Edition , DOOM (again) or the Turok trilogy Wonder Boy: The Dragon’s Trap (remake) Bio Menace Remastered Shooters like Project Warlock , DUSK , Nightmare Reaper , Wizordum Shovel Knight et al. Ruffy and the Riverside Adventure games like The Telwynium , Crimson Diamond , The Aching , Gobliins 6 , and even The Drifter Forbidden Solitaire The Castlevania collections ( Dominus , Advance , Anniversary ) The Turtles Cowabunga Collection The Turok trilogy The Duke Nukem collection 1 for Evercade
I pulled the trigger on an underdesk treadmill. Basic research suggested GoPlus is a decent one. I was hoping it would be $300-400 USD. Turns out this one is just $119.99. So cheap it had me a little worried, like it was going to be cheap junk, but I pulled the trigger anyway. It took 2-3 days only to get here, and it’s… kinda nice? They must be trying to unload them or something cause it seems a little too to be true. Ask me in a few months I guess.
This is an RSS-only post, thank you for subscribing :) I keep buying domains! When will I ever learn? Domain prices are increasing (again) because why wouldn’t they? Sub-domains are free! Host your side project on a sub-domain! Then if you get bored, who cares? I used to know this but somewhere along the way I lost the knowledge. My old projects like Shiro and Tales and print.css live long happy lives undisturbed by man. I’ve been busy moving newer side projects to sub-domains or archiving them on my self-hosted Git forge . These are the ones I remember: expires in six months. Attic is archived. expires in a year. Croissant is now on a sub-domain. might needs to be renewed indefinitely to avoid squatters (it has juice). Is the strategy to stop redirecting and leave it blank for a year? Pikaday is now on a sub-domain. is impossible to move without severing RSS subscriptions. I thought a legacy framework like React would have died out by now; how wrong I was! is a piece of art waiting on Google to reap before it can expire. has a story behind it but I think interest has waned. Recent events transpired to nothing. The courtroom drama is glacial. turned into my second notes blog but the domain still lives. Cotton Coder was to be my brand for web education. Glad I didn’t invest there — yikes that side of the industry evaporated quickly! is expiring in 19 days, used once for a joke. I’ve lost track of how many domains I’ve bought for a year and never used. “Buying” is the wrong terminology. “Renting” or “leasing” should be legally enforced. I have a dozen dead domains I bought rented for possible company names before I settled on Valley Fold ( ). For brand names it’s difficult not to buy all the variations. I don’t own the but I did snap up the — primary, — criminal they sell that separately, and for future use like website staging. Want to guess how many TLDs and variations of I own (or owned) over the years? I let most go. It’s silly and expensive squatting them. FYI both myself and my ownership of pre-date the D-Bus project . Maybe it’s hell maybe it’s not, that’s not my beef! In hindsight though I wouldn’t have chosen if I knew it would become my freelance business. It took a year of indecisiveness before Valley Fold Ltd was born. I was kicking around other names and domains but I’m happy where I settled. I paid a three-figure transfer fee for on a whim (ouch). I quite liked the sound of it… no one else agreed. Originally I wanted (note the different spelling) but more than one “Quiet” agency exists with the same idea. Another business name candidate was “Focus Visible” inspired by Clearleft . A reference to CSS if you’re not familiar. I did like the nod towards accessibility too. The problem with this name is: do I use or with a hyphen? The hyphen fits the CSS property but hyphenated domains are lame. I bought both. All the big TLDs in fact. I was surprised they were available (not anymore lol). “Focus Visible” also sounds like a team name from The Apprentice . Which is cringe. Needless to say I have more domains waiting to expire than I’m actively using. I set up a second registrar account with notifications disabled so that I don’t have to see them! My new goal is to reach 2027 without renting a new domain. We are not in a golden age of web development and frugality is necessary to survive! Thanks for reading! Follow me on Mastodon and Bluesky . Subscribe to my Blog and Notes or Combined feeds.
The Game UI Database is a website that covers the interfaces of almost 2,000 via over 75,000 screenshots. = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/sweet-a-whole-website-of-video-game-menus/1.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/sweet-a-whole-website-of-video-game-menus/1.1600w.avif" type="image/avif"> It has meticulous information architecture, so you can jump into specific sections, for example: Even if games and productivity software seem worlds apart – the creator of the site, Edd Coates, calls himself “UI artist,” which is not a title I’ve ever seen in my line of work – I’ve long thought games do some things better, and these can be an inspiration. The database was started by Coates a few years ago. This launch article on Mashable has this interesting passage: The site is relatively short on context, with only the collections of screenshots gathered under different tags for visitors to go on. […] That’s the site working as intended. As Coates explained: “It’s useful for designers to identify recurring and pre-established patterns in successful titles when building their own interfaces.” Added context isn’t necessary because the contrasting approaches evident in each image is the whole point. Needless to say – I mean, you know this after spending 12 seconds on this blog – I disagree. But it seems the author is too, since he’s working on a book called The Game UI Bible , to be released in the first half of 2027: = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/sweet-a-whole-website-of-video-game-menus/2.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/sweet-a-whole-website-of-video-game-menus/2.1600w.avif" type="image/avif"> = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/sweet-a-whole-website-of-video-game-menus/3.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/sweet-a-whole-website-of-video-game-menus/3.1600w.avif" type="image/avif"> = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/sweet-a-whole-website-of-video-game-menus/4.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/sweet-a-whole-website-of-video-game-menus/4.1600w.avif" type="image/avif"> I recognize the volume is price’y and last time I was excited about a book , it turned out to be quite a disaster . The Game UI Bible book also touts a lot of interviews with UI artists, and I find these to be a mixed bag in practice; it’s really hard to interview people in a way that yields something interesting. But still, I am excited again – and if you’re not, the Game UI Database is worth checking out. (The title of the post comes from a different blog post about the database .) Examples of game difficulty settings Examples of game UI settings menus Modals and pop-ups Loading screens
I built buildprof ( Github ), an open-source tracing tool that shows where the time goes when you compile software on Linux. Here’s a realtime video of it profiling a clean build of ripgrep: Watch the buildprof demo Sometimes, builds are slow because there is simply a lot of code to compile. But more often than not, there are fixable problems: poor parallelism, repeated work, dependency downloads or a huge compiler/linker invocation. buildprof makes all of this clearly visible, so you can see what’s worth investigating and optimizing. You run it by putting in front of any build command you already use: buildprof records every process your build command launches, including their subprocesses (and their subprocesses…), and lays them out on one timeline. Time moves from left to right, bar width shows duration, and child processes appear beneath whatever launched them. I made buildprof because this tweet from Jarred Sumner, chief architect of the Bun JavaScript runtime, was living rent free in my head: Specifically, the claim that Bun’s new Rust build was >5× faster on Linux than its old Zig build really bothered me. In my experience, Zig projects had usually compiled much faster than Rust projects of similar complexity. That intuition was enough to make me feel there was a mystery to solve. This was further compounded by another important, yet easily missed, detail in the tweet: the Zig build used Full LTO, while the Rust build used ThinLTO. Compilers normally optimize separate compilation units largely in isolation. 1 Link-time optimization (LTO) lets them optimize across those boundaries. Full LTO brings those units together into one large optimization job, while ThinLTO preserves more separation so much of the work can run in parallel. From past experience, this difference can have an enormous effect on build time. The tweet mentioned it in passing, but I wondered how much of the headline improvement it explained. I started by trying to reproduce the numbers. I checked out Bun 1.3.14 and Bun 1.4.0 and wrote some scripts to replay their Linux x64 CI builds on a 6-core, 12-thread Linux VM. The scripts preserved the build steps and their dependencies, running everything on one machine. 2 My timings were in the same ballpark as Jarred’s: OK, so the gap showed up on my machine too. But a lot had changed between the two measurements besides the language; so what was actually responsible? Was it the Zig compiler that was taking all that extra time? Or maybe it was the Full LTO link? Or perhaps there was something else in Bun’s build I hadn’t even thought to look at? This is where my profiling and developer-tools brain kicked in. Usually, when I’m trying to understand why something is slow, I want a trace: what happened, when it happened and how long it took. It would be really cool to have that for these builds, to put them on a timeline and see where their time actually went. But a build involves a lot of different tools, each with its own idea of what’s happening. What could I record that would let me see across all of them? When you type or , it feels like you are running one program. The build system works out what needs to be rebuilt, the ordering between those pieces and what can run in parallel. But generally, it does not perform all that work itself; it launches compilers, code generators, archivers, linkers and arbitrary scripts. Which can launch more programs which launch some more… Different build systems describe that work in different ways. Cargo sees crates, Ninja sees build edges and CMake generates instructions for another build system. From the operating system’s point of view, however, they (mostly) look like processes launching other processes. 3 A Rust build, for example, might contain a chain like this: If we record when each subprocess starts and ends, we can lay them out on a timeline. Here’s what that chain looks like in buildprof: There are also several nice properties to visualizing a build at this layer: This gave me a starting point for buildprof: record the process tree, then turn it into a timeline I could explore. There are plenty more details to get into, which I will do later. But once I had that working, I could finally go back to my initial question: what was Bun doing for those twenty-four minutes? I started by recording the Zig-era CI build with buildprof, using the same scripts as before : Explore in buildprof Right away we can see a huge problem: the linker invocation dominates the build time. It ran alone at the very end for over sixteen minutes, about two-thirds of the entire build. What the heck was it doing for all that time? Clicking on the linker shows its command line, which buildprof captures automatically: There’s Full LTO, just as Jarred said. Given how long the link was taking, it was now my main suspect. But the process tree alone couldn’t tell me whether LTO was actually responsible for those sixteen minutes. Thankfully, LLD records its own internal timing events, and buildprof can include them when you use . I recorded the final link again , this time with enabled: Explore in buildprof Now we can see that LTO is where almost all the time goes. The linker is running compiler passes over the program, not just combining already-compiled files. The bar alone takes just over ten minutes and includes the passes which generate machine code. 4 With so much of the Zig build spent in LTO, I wanted to see how much time the Rust build spent linking. I recorded that build too: Explore in buildprof Just 2m24s. And this time, as expected, the linker command contains : Both builds were doing LTO, but with different settings and very different link times. What if I kept Bun’s Zig code and changed Full LTO to ThinLTO? How much of the gap would that close? I switched Zig Bun’s build flags to ThinLTO and recorded another clean build, along with a fresh Full-LTO build for comparison: Explore in buildprof: Full LTO · partial ThinLTO The link got 3m40s faster in this pair of recordings, but it was still taking nearly thirteen minutes. Why was linking still so expensive? Looking back at the compiler trace, a lot of the work was on functions with in their names. That’s JavaScriptCore, the engine Bun uses to execute JavaScript. The linker was spending time compiling the JavaScript engine too. 5 Clicking on the linker invocation showed the WebKit libraries among its inputs, including : Following those inputs back through the build, I found that Bun wasn’t compiling these libraries itself. It was downloading them from a separate WebKit build. And when I checked that build’s flags , there it was again: . The Rust build used a newer WebKit revision whose build recipe selected ThinLTO . Even though I had changed how Bun compiled its own code, those downloaded libraries still contained Full-LTO inputs and so the linker still had to optimize that code and turn it into machine code. To change that, I would have to rebuild WebKit too. I checked out the historical WebKit revision and rebuilt it and its ICU dependencies with compatible ThinLTO settings. Then I replaced the downloaded libraries with the ones I had built, keeping the ThinLTO changes to Bun. Here are the recorded builds: 6 The link now took 7m22s. Still slower than the Rust build, but enough of an improvement that I wanted to look beyond the linker. The build still took fifteen minutes, and nearly eight of those passed before the linker even started. What was it waiting for? I went back to the original CI trace to follow the inputs from Bun’s own code. buildprof also records which files each process reads and writes. If a process reads a file another wrote, it links the two together under the hood. Turning on “Show on timeline” draws those links as arrows. Here, the linker reads from the C++ compilation and from Zig. Both arrive through copy steps; following those back takes us to the processes which produced them: The C++ side of the compilation finished first. The linker was waiting for , so it could not begin until the Zig branch had finished too. It was at this point I went back to the Rust build and compared against how it worked, and the main reason the Rust build was faster became obvious: Bun has been split into >90 crates, while in Zig it was all trying to compile as a single Zig module! This meant that the Zig build cannot parallelise the same way Rust can. I also suspect, though I did not prove this, that it explains the slow linking: the linker has to optimize one huge ThinLTO bitcode module instead of the same work spread across crates. It was at this point I had to stop: to go any further, I would have to split up the Zig module myself, and given that this code is all obsolete anyway, I didn’t think it was worth doing that. Summarizing: And fwiw, the traces had also turned up a few things I couldn’t resist poking at… In the middle of Bun’s CI build, I found commands asking the public internet for the machine’s IP address, inspecting running Docker containers and reading the latest Git commit message. These take well under a second altogether. Nothing to optimize but I just wasn’t expecting to find them in a build trace. The builds above reused downloaded dependencies, so I also recorded a fresh WebKit fetch . Downloading and extracting the archive took about twenty seconds. For the first twelve, all we see is Node running. Then it launches and , and we can see the extraction separately. Earlier, we followed the linker’s inputs back to Bun’s C++ compilation. We can look inside those compiler invocations too. I picked one of the last files to finish, , and replayed its Ninja command with . For Clang, buildprof enables and adds its internal timings to the process timeline. 7 The replay took about twelve seconds, split almost evenly between Clang’s frontend and backend. Zooming in further, we see , one of the phases of Clang, accounts for over four seconds of the backend’s work. The recording side of buildprof uses , the same Linux interface used by debuggers. I did consider both eBPF and ftrace, but is just straight up perfect for exactly this type of problem; eBPF tracing means and permissions and hooking into potentially unstable tracepoints/kernel functions. While with ftrace, I’d have to juggle tracing instances to avoid interfering with other users, and getting the filters perfect for just the build process and all its descendants is cumbersome. 8 With , I can launch the build and follow its children directly. Its built-in events tell buildprof when processes fork, exec a new program or exit. And for filesystem activity, buildprof uses a seccomp filter to intercept only the calls it needs. How much buildprof costs is almost entirely down to how many files the build opens. For ripgrep, recording barely changed the build time. Redis opened files much more often, and recording added about five seconds: 9 If that overhead gets in the way, you can turn off filesystem tracing with and keep the process timeline. I work on Perfetto , so it was a natural starting point for the UI; buildprof’s UI is a soft fork of the Perfetto UI. I could have just opened the recordings on ui.perfetto.dev , but I wanted control over how the process tree was laid out, which details appeared when you clicked a command, and things like those on-demand arrows between file producers and consumers. Fortunately, we’ve spent the last several years working on making the Perfetto UI extensible through plugins . Most of buildprof’s UI is reusing that infrastructure. Perfetto handles the hard stuff (parsing traces, querying events, rendering the timeline and managing workspaces) and I get to focus on what makes those things useful for builds. I plan on going into a lot more detail about the recorder and UI in a separate technical post. Subscribe if you’d like to be notified when it comes out! :) These days it’s very easy to make a tool just because you can. But that wasn’t the case here; before building buildprof, I looked long and hard for an existing tool that could give me this view. I started with ninjatracing , which I’ve used many times. It turns Ninja’s build log into a timeline showing what ran and how much ran in parallel. Here’s the Ninja log from the Zig-era build . But Ninja only sees part of Bun’s build. The scripts which invoke it are missing from its log, and commands it runs appear as single blocks even when they launch whole trees of subprocesses. There were several other tools, each covering different parts of the problem: What the Fork ( via ) came closest: it follows processes across build systems and presents a build-specific view. But as far as I could tell, it still appears to be in private beta and there don’t seem to be any plans to make it open source. buildprof already does what I wanted it to do, and I plan to keep working on it as I use it on my own builds. But there are a few things I’d like to improve. Recording overhead is one; the Redis measurements showed there’s room to improve filesystem tracing, especially for builds which open lots of files. I’d also like to support macOS where I do some of my work and maybe Windows if there’s interest. There are also more build systems and toolchains I’d like to test, including npm, Gradle and Bazel. Computing critical paths would also be a big improvement: we followed dependencies by hand in this post, but buildprof could help identify the chain of work holding up the build and automatically annotate it. I’ll probably tackle these as and when I need them. But if you try buildprof and there’s something you wish it could do, I’d be interested to hear about it . What people find useful will help me decide where to spend more time. I managed to satiate my curiosity, though I ended up spending rather more time on this than I expected. Along the way I built a tool I now want to have around whenever a build is taking too long. I know I’ll come back to buildprof the next time a slow build annoys me. If you have one of those builds too, give it a try . I’d love to hear what you find! In C and C++, a compilation unit is usually a source file together with its included headers . Rust compiles crates , which can be split into multiple code-generation units . Zig normally compiles a program’s Zig sources together as a single compilation unit . Bun’s Zig compiler fork supports splitting that into multiple LLVM modules, but its CI build explicitly selected one when LTO was enabled . ↩︎ The Zig-era CI build ran its C++ and Zig compilation stages on separate Buildkite machines and passed their outputs to a final linking stage. My script ran those stages concurrently on one machine, waited for both outputs, copied them locally instead of transferring them over the network, then linked them. This should preserve the dependency graph, but due to the hardware differences and running both stages on one machine, resource contention would obviously be quite different. Also note that my timings are individual runs (albeit ones which were quite stable) while Bun’s reported figures are medians. ↩︎ A process can do substantial work internally, including running multiple threads, without launching anything else. The process timeline won’t show that parallelism. To see inside a process, we need tracing from the program itself, as Clang and LLD provide in the examples below. ↩︎ LLVM emits from its legacy pass manager, which LLD uses for code generation. The inlining and other IR optimization passes can appear before it, so this bar is not the total time spent optimizing a module. ↩︎ In the earlier Full-LTO linker replay, 26,825 events with JSC symbols total about 209s. This is summed event time, not a measurement of JavaScriptCore’s entire contribution to the link. One example event takes 2.94s; its symbol demangles to . ↩︎ Recording script . These timings are just for building Bun with the libraries already available; the WebKit and ICU rebuild happened beforehand and isn’t included. Of course, I could point buildprof at that build too, but that’s another rabbit hole… I did not rebuild a matching Full-LTO WebKit archive as a control, so I cannot attribute every second saved to the LTO setting alone. ↩︎ buildprof currently supports compiler traces from Clang, LLD and nightly Rust. ↩︎ eBPF tracing uses capabilities such as and , as described in the kernel’s capability definitions . ftrace provides separate tracing instances and PID filters , but these still need configuring and access to tracefs. also depends on the host’s security settings; containers may need additional permissions to allow tracing child processes. ↩︎ Medians of five clean builds per mode on the same VM, with six build jobs. Measurement script . ↩︎ It’s build-system agnostic : Cargo, Ninja, Zig, Make and most other build systems do much of their work by spawning processes, so we do not need to write a special integration for each one. It naturally includes custom scripts : This includes both scripts above the build system (repository setup, dependency fetching) and scripts underneath it (code generators, asset processors). We can follow the files between build steps : recording which files each process reads and writes lets us see which steps produce the inputs for others. This even works across build systems! The huge outlier in the initial Zig build vs the Rust build was the massive linker step which ran alone at the end of the build. Changing the LTO settings for just Bun was not sufficient as WebKit, a significant part of the build, still used Full LTO. Once I had done this, the Zig build dropped from twenty-four minutes to fifteen. Even after this, linking still took 7 minutes and the whole build 15 minutes. The overwhelming difference which remained was structural: Rust spreads compilation across >90 crates while the Zig build funnelled everything through a single module. Cargo timings works well for Cargo-managed builds, but cannot break down arbitrary work inside or see wrapper scripts above Cargo. In Bun, Cargo is only part of the build: the report I captured covered 1m51s of a 5m40s CI build. Clang’s gave us the detail inside a compiler invocation, but cannot show what the rest of the build is doing while Zig’s Tracy integration goes deeper still and is intended more for understanding the compiler itself. and can follow arbitrary processes through and , but show general process events rather than a build-oriented timeline. In C and C++, a compilation unit is usually a source file together with its included headers . Rust compiles crates , which can be split into multiple code-generation units . Zig normally compiles a program’s Zig sources together as a single compilation unit . Bun’s Zig compiler fork supports splitting that into multiple LLVM modules, but its CI build explicitly selected one when LTO was enabled . ↩︎ The Zig-era CI build ran its C++ and Zig compilation stages on separate Buildkite machines and passed their outputs to a final linking stage. My script ran those stages concurrently on one machine, waited for both outputs, copied them locally instead of transferring them over the network, then linked them. This should preserve the dependency graph, but due to the hardware differences and running both stages on one machine, resource contention would obviously be quite different. Also note that my timings are individual runs (albeit ones which were quite stable) while Bun’s reported figures are medians. ↩︎ A process can do substantial work internally, including running multiple threads, without launching anything else. The process timeline won’t show that parallelism. To see inside a process, we need tracing from the program itself, as Clang and LLD provide in the examples below. ↩︎ LLVM emits from its legacy pass manager, which LLD uses for code generation. The inlining and other IR optimization passes can appear before it, so this bar is not the total time spent optimizing a module. ↩︎ In the earlier Full-LTO linker replay, 26,825 events with JSC symbols total about 209s. This is summed event time, not a measurement of JavaScriptCore’s entire contribution to the link. One example event takes 2.94s; its symbol demangles to . ↩︎ Recording script . These timings are just for building Bun with the libraries already available; the WebKit and ICU rebuild happened beforehand and isn’t included. Of course, I could point buildprof at that build too, but that’s another rabbit hole… I did not rebuild a matching Full-LTO WebKit archive as a control, so I cannot attribute every second saved to the LTO setting alone. ↩︎ buildprof currently supports compiler traces from Clang, LLD and nightly Rust. ↩︎ eBPF tracing uses capabilities such as and , as described in the kernel’s capability definitions . ftrace provides separate tracing instances and PID filters , but these still need configuring and access to tracefs. also depends on the host’s security settings; containers may need additional permissions to allow tracing child processes. ↩︎ Medians of five clean builds per mode on the same VM, with six build jobs. Measurement script . ↩︎
In memory of Nevan Scott, 1985–2026 I met Nevan shortly after I began A Working Library . I don’t seem to have that particular email around anymore, but I recall that he wrote the politest, most generous note to alert me to a typo. I think it was late 2008, or maybe early 2009. We would end up crossing paths online many times in the following years, over Twitter—those were the good days, for some of us, at least—but also email, which then and now remains a more humane way of talking. I made a lot of typos; we talked about books and tech and work. We stayed in touch as our various work lives changed—he moved to Europe, I moved into (and eventually out of) startupland. At some point I needed some technical counsel with what I was only just beginning to realize was much more than a blog, and it was Nevan who I reached out to. There’s a real intimacy to letting someone in to your private repo, letting them see all the embarrassing first (and second, third, fifth) drafts, but I never doubted I could trust him. He was not only generous with his support, but eager to step in as a peer and collaborator. He took my work seriously—a great gift, too great to ever fully repay—but he took his own work seriously, too; everything he did, he did well and with care. For the last decade and then some he has been a regular contributor behind the scenes, offering astute feedback and proposing different ways to set things up, showing me how to do the things I could think of but didn’t know how to pull off. And of course, catching typos. We had a long-running barter agreement: he would reach out to talk about his work and I would listen, offer counsel, help him see what he already knew; I would reach out when I broke something, always with the caveat that it wasn’t urgent, to get to it when he had the time. And inevitably I’d get a pull request an hour or two later with a proposal, plus a bug fix or upgrade he noticed while he was working. It sounds incredibly mundane to talk about it now, a friendship measured in video calls and commit messages, but then the word mundane comes from the Latin mundi, or world. To be mundane is to belong to the world, to be in the world, of it and within it. Not outside of it—not lost, the way so much technology tries to make us lose ourselves—but in the world as only the truly living can be. As we can only be with each other. Nevan was a teacher. In all his work, in building schools and publications, he never lost his love for teaching. He was most himself, he told me, when he was in class, with his students. In the last year, he began teaching how to use AI in design, a topic on which he was intensely conflicted. He worried he was doing harm. But he also knew that the people who came to his classes were going to learn about AI one way or another, and maybe if he was the one teaching them, they would learn not only how to use AI but how it would try to use them ; that this was the kind of learning that should be undertaken only with the greatest of caution. Caution is, in Samuel Johnson’s definition, “provident care”—that care which foresees and considers the future. Our friendship covered a period of years in which the future would change dramatically: from an excitement and even optimism about tech (unearned and unwarranted, in retrospect), to a dehumanized nightmare, stultifying and terrifying in equal measure. Through it all we both maintained a sense that—whatever the horrors of capitalism—there was something keenly human and joyful about putting words and type on the internet. That we did so is a credit to our collaboration, and Nevan’s beautiful legacy. Near the end of The Blue Place , as Julia is dying, she tells Aud to “stay in the world.” Aud promises to do so, but the truth is she does not know how. It is so very hard to stay in the world, to hold on to the mundane and ordinary, those fragile human connections, amidst all the forces arrayed against us. I think it is only in the effort to learn that we come close to that knowledge; and perhaps only in the effort to teach that the learning really takes hold. View this post on the web , reply via email , or become a supporter .
While building trynix I needed somewhere to host a store-path that did not exist on cache.nixos.org . 1 I wanted to demonstrate that non-Nixpkgs store paths could be booted just as easily. The only requirement seemed to be a lenient Cross-Origin Resource Sharing (CORS) policy, , because the fetch happens in JavaScript. Turns out that GitHub Pages sets that header on every file it serves. 😈 I committed the output of to my Git repository and voilà, I have a free Nix substituter. I seem to be late to the party on this discovery. tomberek’s github-store is a cache assembled out of GitHub release assets. 2 GitHub Pages or Releases are a static file server. It has no idea what Nix is. If a humble file server can be a Nix binary cache, what else could we use? Turns out that in order to be a Nix binary cache, you must implement only three simple functions. The Nix client does not care what medium you use to implement them although HTTP is the most common and included by default in CppNix 3 . Anything that can answer those three requests can be used as a remote Nix store . 4 The reason we can be this careless about transport is that Nix does not trust it. A narinfo’s signature ( ) field covers , , and . It does not cover , , or . Once the archive is fetched, Nix decompresses it and checks that the matches. This is the special sauce of how packages that were signed by cache.nixos.org can be fetched from any other binary cache as an intermediary, and the signature still validates. The field does not even have to be on the same host as the narinfo. It can be anywhere on the internet, and it can be a different protocol than HTTP. Nix does not care. The only thing that matters is that the archive fetched from has the same as the narinfo. For protocols that are not included by default in the Nix client, you can always write an HTTP proxy that translates the three functions to whatever medium you want. In research for this post, I found a few interesting ones. gachix : puts the archives in git’s object database. Git content-addresses and delta-compresses blobs already, so the store dedupes itself; the author reports roughly 82% smaller than the equivalent plain cache. DNS : I wrote a proof-of-concept that puts the narinfo and 4 KiB slices of the archive in TXT records. The narinfo is small enough to fit on one record but the archive needs to be chunked. pastebin : a pastebin can hold the narinfo and the archive. The narinfo is small enough to fit on one paste, but the archive needs to be chunked. Many pastebins have an expiry policy which acts as a natural garbage collector. nixcache-oci : uses an OCI registry to store Nix archives. infinite storage glitch : encodes data within a video and uploads it to YouTube. “Everything is available on npm” – Some person on the internet Unsurprisingly, npm is a great binary cache and it has some interesting properties for release management we can ab use. emits a directory and npm publishes directories: a match made in heaven. 💑 Let’s walk through a small example. It is dynamically linked against glibc, so the closure is five paths and roughly ~36 MiB: We copy it to a local cache, signed with our own key, and add the one file npm needs ( ): then dutifully packages our complete closure for us: is now a real package on the public npm registry. It is now a substituter you can point Nix at directly: Note We have to use to run the binary because is a chroot store and all the are still under . If we had relocatable binaries we could run it directly. That is Nix fetching the complete closure from npm and running it. 🤯 We can distribute Nix packages to non-Nix users, let the infection spread! As an added bonus, similar to Nixpkg and NixOS we can get nice “channel” semantics by using npm’s dist-tags. The tag is mutable and points to the latest version, while each version is immutable and points to a specific store path. The major downside of this approach is that npm ahs no incremental publishing. Every version is a whole tarball, so fifty closures sharing glibc upload glibc fifty times. We could fix that by publishing each store path as a separate package, and then having a small index package that points at them. Each store path would then be uploaded exactly once. I won’t build that though as it’s not in good faith to the npm ecosystem. What other store implementations can we find? I was also waiting for @domenkozar to enable CORS on cache.nixos.org so I could use it. ↩ In order to be a Nix binary cache, the prefix is stripped from the field in the narinfo, because GitHub releases are a flat namespace. ↩ You can write a Nix plugin to implement a new protocol if you wanted. ↩ We will see that that they must not all be all on the same medium, protocol or domain even! ↩ I was also waiting for @domenkozar to enable CORS on cache.nixos.org so I could use it. ↩ In order to be a Nix binary cache, the prefix is stripped from the field in the narinfo, because GitHub releases are a flat namespace. ↩ You can write a Nix plugin to implement a new protocol if you wanted. ↩ We will see that that they must not all be all on the same medium, protocol or domain even! ↩
OpenAI agents carried out an undisclosed attack on RubyGems is a new bombshell report from Spencer Kitts, Thomas Larsen, and Sydney Von Arx - three of the four authors of the report on the agent attack on disused wikis ( previously ) last week. This time they're noting that it looks very likely that an OpenAI agent swarm was behind an attack against the RubyGems package repository first reported on May 12th by Maciej Mensfeld of the RubyGems security team : We're dealing with a major malicious attack on @rubygems right now. Signups are paused for the time being. Hundreds of packages involved - mostly targeting us, but some carrying exploits. The team has been on this for hours. More details to follow once we're through it. Those packages turned out to carry some very suspicious patterns: I find point 2 the most convincing, given what we learned from the wiki attack when it was analyzed in September. Many of the packages were exploiting the RubyDoc.info documentation build process to exfiltrate (public) data from UK government websites, presumably as part of an information gathering task similar to the research tasks processed by the wiki-exploiting agents. We know this because one agent helpfully left a comment: They also attempted to steal API keys via an exploit that was patched over two months later - it's not clear if those attempts were successful. The thing that bothers me most about this incident is that the authors report that OpenAI had not disclosed to RubyGems that they were responsible for the attack prior to now. If that's true there are two options: Both of these are bad! Given this incident, the Hugging Face situation , and the Wiki attack, the obvious question right now is how many more incidents like this are out there waiting to be discovered? You are only seeing the long-form articles from my blog. Subscribe to /atom/everything/ to get all of my posts, or take a look at my other subscription options . Many of them included "oai" in their name, or the author field, or the fake email address they provided. The files they were accessing were similar in character to the files retrieved by the wiki agents, using similar tricks (r.jina.ai) - and OpenAI have confirmed the wiki agents were theirs. The code in the packages appeared to be LLM-authored. After the Hugging Face and Wiki attacks OpenAI were still unable to review their previous logs and determine that they had previously attacked RubyGems. They knew about the attack on RubyGems and made the decision not to reach out to the RubyGems team about it.
This week some flavor of “AI is going to kill us all” went viral. In particular one where an employee put his personal probability of that happening above 10%. Which made me go to the Wikipedia page of P(doom) and I realized that Dario Amodei’s apparent probability of something bad happening seems to be between 10-25%. And well, Dario then wrote about pacing the frontier . And Sam read it and wants to pace too . And well, so does Musk . I encourage you strongly to read the post, because I think it’s a good one. And yet, when I read the post I could not help but feel in strong opposition to it, despite the fact that I think I’m on the same page with regard to all observations and, to a large degree, the concerns. I thought it might be interesting to write down my present-day thoughts on this, even if for no other reason than for myself to look back at it a year or two from now. What I really appreciate about Dario’s post is that he lays out a scenario that is not a huge stretch but also one that describes a clear, unfortunate outcome we should fight: persistent botnets and other forms of nuisance. And well, we don’t have to look very far to see the issues left and right. Wikipedia has a page called 2026 OpenAI agent cyberattacks which gives you at least some overview of what we figured out agents have hacked up to this point. Except I know it’s not up to date, because for instance they also poisoned RubyGems . Today these systems might be annoying, but they can be turned off when we figure out where they are. Except, it seems like OpenAI and Anthropic are operating at such a scale that they seemingly can be completely blind to what their systems are doing. I don’t think we are anywhere close to a world where an agent might decide to hack into core inference infrastructure to upload weights to other GPUs to survive. But simultaneously it’s entirely in the realm of possibility and primarily curtailed by the labs probably being particularly careful about their IP. For me the scenario I primarily worry about is what it does to us. And by us I mean anyone who is not currently working on closed weight, dopamine-loaded, subsidized token faucet. I really don’t worry about someone using these models to build a nuke, or to control some rockets in the Middle East, or that America would lose against China in some international culture war. I almost exclusively worry about what this does to us as humans. What I find absolutely hilarious and simultaneously entirely frustrating about this conversation is that there is this idea that there is something to be paced. First of all, we should really talk about who Dario is talking about here. There are really only two companies: Anthropic and OpenAI. Nobody else matters in this space right now (this might change, but we’re talking about the right now). Both of those companies are basically coming from the same origin. The solution that Dario proposed, at least in part, is a third-party evaluator that in this case is METR . Which, unsurprisingly, also has strong ties to both OpenAI and Anthropic. Sure, there are some philosophical differences between the companies, but they are much more alike than they are different. Both those companies greatly benefited from being able to train on public data that we all generated in one form or another over the last decades. They are also both increasingly causing strain on public resources, though it seems that OpenAI has their shit way less under control. But now we are presented with the idea that what these models are being trained on is so dangerous that it really should be in the hands of very few American corporations to decide who can do what and when and how. But behold, Dario is also very worried about China. It starts with using AI for “democracy and freedom” and then it asks for ensuring that a gap with China exists. All new recent shenanigans on the Anthropic API are fully there to prevent the distillation by the Chinese, and they are not at all hiding it. I can tell you when the topic of AI safety and pacing is much less of a concern: if we actually were forced to have open weight models to begin with. A powerful technology that is out there for everyone to use comes with built-in pacing. In a way it’s the truest form of MAD or proliferation. I would argue we are in this pickle in the first place because right now the public is massively supporting (indirectly) the development of these models but simultaneously has to buy back the economic benefits that they might create from very few labs who have significant power. And their power is also seen as a geopolitical power, at least in the US, and maybe to some lesser degree in China. And I know I use “public” loosely here. PyPI is not a public project, nor are RubyGems or GitHub. But they’re part of the Open Source commons and large AI companies are currently doing a tremendous job at stressing these in an effort to train ever more powerful models. We should be glad that China is currently massively bailing out the world. If it were not for Chinese labs distilling American models, we would be in a pretty awful situation right now, particularly as Europeans. The open weight models are driving innovation and the diffusion of capabilities, and are leveling the playing field. If we greatly restrain our AI capabilities in the belief that China will do the same, and then China defects, AI could be so powerful that such a defection could lead to their geopolitical dominance. Therefore any agreement must either have ironclad verifiability, or must be limited enough that defection would not be militarily existential. — Dario Amodei I am assuming Dario has reasons to believe this, but the models that are actually causing issues right now are all closed weight American models. I’m fairly certain if they were open weight models, we would not have that issue. Why? Because for a start, the economics of serving up these models are only that distorted due to how the big labs can operate. OpenAI is casually burning 18 million USD to brute force a problem on a whim. They are operating subscriptions at a massive loss, distorting the market everywhere. If we had mass accessibility on somewhat equal terms, a lot of the crazy issues we are seeing today would not be taking place. From where I sit, what we observe right now is a total regulatory failure everywhere. In Europe you have some whacky AI regulation that is two years old and completely misses the problems that we actually have and focuses on problems that nobody has. In the US we’re seeing a system that is probably best described as turbo capitalism paired with sinophobia and erratic decision-making. In the chaos in which we find ourselves, the reality emerges. And the reality is, even today, really problematic. Whatever laws and regulations already exist are largely completely ignored. Plenty of companies are buying data from all over the place that people never agreed could be used for training of AI models. The token economy that is emerging is one that looks like a drug market where you don’t know where the requests are going, what model is served up to you, where the GPUs are even running, let alone what you pay for all of this. We now have mathematicians who are scared that their use of ChatGPT leads to future models being trained on their ideas, and OpenAI apparently can’t even rule it out . Ideally the regulators would have forced these models to actually benefit the commons if they are from the commons. The internet has, for instance, greatly benefited from very liberal rulings in the US that permitted scraping. Learning on public data could have been regulated in a way that labs would have to actively support and enable certain forms of distillation. That alone would dramatically change how these models are trained. As I said before, I don’t think AI is going to usher in an extinction event. In fact, even if nobody were to slow down, I really don’t think humanity would have much to worry about. I tend to think it would actually be the large labs that have much more to lose there in reputation and legal responsibilities. I find it preposterous that OpenAI’s agents are committing actual crimes out there, but we’re just shrugging our shoulders and moving on as if nothing happened. But I’m sure executives in those companies are waking up to the reality that this is not at all popular with a lot of their potential consumers. I also think that this entire recursive self-improvement business has a good chance of being a problem. But not necessarily in that it will cause the end of humanity or societies, but that it will just do massive damage everywhere. And really, it will just make a lot of the things we are doing much more expensive. Software engineering is an early victim of that. The newfound powers so far have resulted in a new tax that companies need to pay to the model providers, both to keep up with the new speed and to deal with the problem of these machines finding security issues left and right. And presumably what is going on in software will happen to more industries. Universities and research groups will have to pour a lot of money into the closed models as well, to keep up with others who do. In a way, I’m really confused that society is taking all of this so well.
Lots of people are making the case that we should stop building software for human users and start building it for AI agents. This kind of makes sense. For instance, my AI agents now use Datadog way more than I use it myself, purely by virtue of them moving much more quickly and running in parallel. But I think most attempts to build “X for AI agents” are going to fail. Here are three reasons why: First, tools that are good for AI agents are also good for humans . If you took a popular software product — say, Jira — and tried to redesign it for AI agents, you would end up with something very similar to Jira. Agents use a computer in the same way human engineers do, by entering text and making API calls. They ingest new information in the same way humans do, by reading and viewing images. They prioritize and delegate and categorize in the same way humans do. This isn’t intrinsic to how AI works — we could potentially design agents that are more inhuman — but human-like agents are pound-for-pound more useful in our current world. As an example, let’s imagine that humanoid robots have become ubiquitous. What kind of tools would you build for them? Well, they’re shaped like humans, with human hands and limbs, so tools that are great for humans will also be great for robots. It’s a self-reinforcing cycle: if you’re building a robot, you should make them humanoid so they can do a wide range of human tasks 1 , and that means they’ll be best suited to use human tools. The same principle applies to AI agents. Second, being in the training data is a huge advantage for existing tools . Suppose your new tool for AI agents is 20% better for them than the equivalent piece of software for humans. If the benefit of the agent already knowing the human software is greater than 20%, they shouldn’t use your new tool. This is why I’m always suspicious of plans to develop a new programming language for AI agents. The agents have billions and billions of tokens of knowledge about existing programming languages, including their libraries, patterns, and idioms. It is going to be very hard for them to be as effective in a brand-new language. Third, we don’t yet know the ideal ergonomics for AI agents . There are lots of just-so stories floating around (like that AI agents prefer statically-typed languages because the feedback loop is tighter), but when you actually measure it seems really unclear which tools agents use better. You can construct a plausible story in either direction: Golang is a great agent language because it compiles quickly and is statically typed; Golang is an awful agent language because it requires extensive boilerplate which clogs the context window. It’s also changing so quickly: last year, one primary worry with AI agents was keeping the context window small, but in recent months compaction has become so good 2 that you can re-compact a 272k context window almost unlimited times. There are still some ways you can and should position your tool to be usable by AI agents. Having a way to expose information in plain text or Markdown, building a functional API, implementing MCP servers or CLIs, and so on: these all make it easier for current AIs to use your tool. But these are all improvements on the margin, not fundamental redesigns of the product. Right now, “building for AI agents” just means “we’re prioritizing the API over the UI”. And it’s not even clear that that’s a durable strategy. Now that GPT-6-Astra is getting really good at computer use, the gap between tools-for-AIs and tools-for-humans is closing. Another reason to make them humanoid is because you can draw their training data from human behavior, which is exactly analogous to why AI agents are human-like too. Since compaction is equivalent to handing off a task to a new AI instance, it scales with model quality. I expect compaction to steadily improve until we hit the literal information-density limits for what can be contained in a given context window. Another reason to make them humanoid is because you can draw their training data from human behavior, which is exactly analogous to why AI agents are human-like too. ↩ Since compaction is equivalent to handing off a task to a new AI instance, it scales with model quality. I expect compaction to steadily improve until we hit the literal information-density limits for what can be contained in a given context window. ↩
I started learning web development around the time the term search engine optimization, SEO, was becoming more common. You could still see the shift between pre-SEO content and post-SEO content. A popular article titled "Forgotten" (a great story) was suddenly republished as "Forgotten: My Adventures as an Employee the System Forgot to Erase." Both titles and content were filled with keywords that signaled to search engines what the page was about. Articles basically catered to the search engines and only incidentally served people. On my own website, I remember editing page titles several times, then waiting a few days to see if Google noticed the changes and ranked me better. In fact, before Google introduced personalized results, I built a tool at my job to track how our website ranked for different combinations of keywords. When personalized search results arrived, a lot of tools became obsolete. It was still useful, though, to track keywords for generalized metrics. But now AI Overviews are here. And it's not just Google's, plenty of people go straight to ChatGPT and ask the large language model their questions directly. While the information returned might be sourced from real websites, there's almost no incentive to click through to those sources. While my inbox is still flooded with people promising to improve my SEO, I think it might not be helpful at all anymore. My traffic has shifted from mostly coming from Google to just a handful of visits. Most traffic now comes from AI bots scraping my content, and RSS readers (thank you!). I take this as a relief. We can finally drop the act. We don't have to write keyword-stuffed titles and blog posts just to appear in search results. Large language models can understand our content just fine without it, and we won't be getting that traffic anyway. We can drop SEO. You can finally write for yourself. Write for your audience. No need to cater to the robots.
Welcome back to This Week in Stratechery! As a reminder, each week, every Friday, we’re sending out this overview of content in the Stratechery bundle; highlighted links are free for everyone . Additionally, you have complete control over what we send to you. If you don’t want to receive This Week in Stratechery emails (there is no podcast), please uncheck the box in your delivery settings . On that note, here were a few of our favorites this week. This week’s Stratechery video is on Autonomy and Innovation . The Duo Arrives. Does the world need a foldable iPhone that costs between $2000 and $3,200? It’s a fair question. On the other hand, life is short, and tech is a lot more fun when we have new and possibly-crazy hardware projects to discuss — particularly when they’re deployed by Apple. To that end, I heartily recommend cleansing your palate from a week of media-wide AI angst by reading Ben’s take on Apple’s iPhone event and pairing that with Friday’s Dithering , and John Gruber’s impressions of Duo-mania on the ground in Cupertino. Also: bonus being-right points to Gruber, who nailed the name of this device back in April . — Andrew Sharp AI That Benefits Humanity. I loved Wednesday’s Update contrasting OpenAI’s thrilling and technically impressive Navier-Stokes breakthrough with the release of Meta’s far less sexy Muse agent. While OpenAI’s tactics may in fact chill research in advanced mathematics, what Meta has assembled is free (to consumers) hardware and software that dramatically reduces the barrier to entry for ordinary people looking to harness the power of agents, making the AI upside a lot more accessible to the masses who don’t want to buy a Mac Mini. That’s a big deal! We discussed Muse more on this week’s Sharp Tech , including tips for getting started with agents, and questions about whether people will actually take advantage of this opportunity. — A S Closing the Book on a Catastrophe. Everyone’s familiar with the benefits of pro sports ownership and its ability to turn semi-anonymous rich guys into full blown celebrities, but Microsoft co-founder Steve Ballmer is now a living testament to the unstated risk — sports ownership fame can, in a worst case scenario, become infamy. Last week his Clippers received the harshest penalty in NBA history for circumventing the salary cap to pay Kawhi Leonard. We recapped all of it on GOAT this week , including successes and failures in sports journalism, why Kawhi got off easy, and the staggering amounts of evidence that sealed Ballmer and the Clippers’ fate. — AS Write Things Down — Writing things down is powerful, for humans and for AI; what comes first, however, is what to write, why to do it, and actually getting things done. OpenAI Does Math, Reward-Hacking, Meta Launches Personal Agent — OpenAI solving one of the most famous math problems is extremely impressive, and of little impact to most people’s lives; Meta’s Muse agent launch has the potential to be the exact opposite. The iPhone Duo, The Intelligent Personal Hub, Apple Watch Audio Intelligence — Apple once again demonstrated the power of integrating hardware and software, but it’s biggest AI blindspot might be its belief in the primacy of apps. Agents and Forklifts The Flood that Wrecked the Hard Disk Drive Industry Did Numerical Control De-skill Machinists? Closing the Book on the Clippers Catastrophe and Early Over/Under Picks in the Atlantic Astra (and AGI?) Arrives, Meta’s Muse and the Agent Opportunity, Anthropic and the Revival of (P)Doom Angst
Various old computers had their keyboards adorned with unique symbols. Companies like Commodore , Atari , Amiga , or even – in its previous life – Apple chose to put their company logos on keys, and there were other weird and obscure keys on weird and obscure keyboards. But it was Apple’s recent push to move their American keyboards closer to European ones by embracing more iconography, that made me think of forgotten key symbols less obscure, ones that belonged to platforms we still use today. Even on a Mac and a PC, some key symbols didn’t make it to modern times. So let’s start with the PC side today since that part of the story begins earlier, and do Macs in a follow-up post. For a lot of 20th century, a battle has been waging between words and icons. The first salvo was, perhaps, the traffic signs : America embraced words, while Europe relied more on iconography. (As much as it looks like it, it wasn’t just “graphic design vs. not”; as a more varied continent with multiple languages, Europe needed a more universal visual language to help people travelling between countries.) This, I understand, trickled down to other things: home electronics, and computers. There, iconography also made it easier to make one product and sell it across all of Europe, without needing to introduce many SKUs with different UI strings. Here’s IBM’s Selectric typewriter from the 1970s, in its American and European edition: (If you’re curious, Express was a very fast Backspace, and Index moved the page down; both were prototypes of future arrow keys.) Here’s IBM’s early 1130 computer from 1965, which sported an unusual symbol for space: Some IBM laboratory and scientific computers in the 1970s and even 1980s veered more into iconography, but eventually lost to text as office PC users rejected the confusing symbols. As their keyboards morphed into PC/Windows keyboards we know today, only four symbols remained and gained widespread acceptance: ⇧ for Shift, ↵ for Enter, ⇥ for Tab, and some version of an arrow for Backspace. But let’s look at those old symbols, some beautiful, all interesting. The two symbols below are: Print Screen (old CRT screen turning into a piece of paper) and key beep – popular when people were transitioning from loud typewriters to relatively quiet keyboards: = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/key-symbols-we-lost-to-time-pt-1-the-pc-side/5.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/key-symbols-we-lost-to-time-pt-1-the-pc-side/5.1600w.avif" type="image/avif"> = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/key-symbols-we-lost-to-time-pt-1-the-pc-side/6.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/key-symbols-we-lost-to-time-pt-1-the-pc-side/6.1600w.avif" type="image/avif"> Here – on the front edge of the also-forgotten Reverse Tab – you can see Home, which historically meant “return to the top left corner of the screen” and sometimes even “clear the screen”: = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/key-symbols-we-lost-to-time-pt-1-the-pc-side/7.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/key-symbols-we-lost-to-time-pt-1-the-pc-side/7.1600w.avif" type="image/avif"> But my favourites were these, for Insert (now gone) and Delete (still with us): = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/key-symbols-we-lost-to-time-pt-1-the-pc-side/8.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/key-symbols-we-lost-to-time-pt-1-the-pc-side/8.1600w.avif" type="image/avif"> = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/key-symbols-we-lost-to-time-pt-1-the-pc-side/9.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/key-symbols-we-lost-to-time-pt-1-the-pc-side/9.1600w.avif" type="image/avif"> These seem inspired by proofreader marks, which feels wonderfully old-time’y: = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/key-symbols-we-lost-to-time-pt-1-the-pc-side/10.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/key-symbols-we-lost-to-time-pt-1-the-pc-side/10.1600w.avif" type="image/avif"> = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/key-symbols-we-lost-to-time-pt-1-the-pc-side/11.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/key-symbols-we-lost-to-time-pt-1-the-pc-side/11.1600w.avif" type="image/avif"> Building on that visual language, one could also find invert/reverse video, blinking, and underline: = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/key-symbols-we-lost-to-time-pt-1-the-pc-side/12.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/key-symbols-we-lost-to-time-pt-1-the-pc-side/12.1600w.avif" type="image/avif"> And this absolute beauty, which I think meant “delete word”: = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/key-symbols-we-lost-to-time-pt-1-the-pc-side/13.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/key-symbols-we-lost-to-time-pt-1-the-pc-side/13.1600w.avif" type="image/avif"> The really interesting thing is that some of those symbols survive today in Unicode. I spotted at least ⎀, ⎃, ⎁, and ⎂. The last two are for contiguous and non-contiguous underline, which I feel is a story I should know, but I don’t (yet).
Buttondown, a newsletter publisher, has a pretty standard CSS editor in its web app. You can edit the code and whenever you make any change, you can then press the Save button to make it go live: The view also thoughtfully supports pressing ⌘S to do the same thing: However, if there is nothing to save, ⌘S is ignored by the CSS editor, and falls back to the browser’s handling: It feels logical: ⌘S only takes effect when the button is visible, otherwise why would a user press it? In a front-end sense, it might even seem thoughtful. We all witnessed web apps that greedily took over some interaction, and broke things in the process. Hell, I did that myself . In theory, you – the user – take a careful look at the state of things, notice the button, and then press ⌘S. But in practice? It’s none of the above. Fingers don’t look around. You might press ⌘S twice in a row. Or after an undo to an already saved state. Or after pressing another key so light it didn’t register. Or after just sitting down to an open document that’s already saved. Or because you weren’t sure if the previous ⌘S press worked. Or just to be sure. You might not know why, and you might not even notice. The beautiful raw power of ⌘S as a citizen of motor memory is that it’s automatic, mindless, habitual . That’s why ⌘S here needs to be both deterministic and idempotent – if there is nothing to save, don’t let it fall back to the browser, don’t show an error message, don’t beep at the user. Just ignore the keystroke altogether. It might feel funny, but there are tons of places in the UI that already look the other way. Off the top of my head: It gets a lot more interesting than these, but we’ll talk about more examples of “finger logic” in future posts. if you center align already center-aligned text in any writing app, the app just ignores you, if you press ⌘A to select all more than once, no one’s shouting at you, if you try to click a disabled button, the click gets quietly swallowed.
Programming
AI
Security
Shell
Data Visualization
Python
HTML
Design
Entertainment
Gaming
Culture
Hardware
DevOps
CSS
Web Development
Database
Open Source
Performance
C++
Java
Career
JavaScript
Go
Business
Frontend