Latest Posts (20 found)
Sean Goedecke Today

No, local models will not win

Every time a new open-weight AI model is released, people say that local models are the future. Why spend billions of dollars building out datacenters when everyone will just be able to run AI models on their laptops or phones? I think this idea is doomed. No matter how strong open-weight models get, most inference will always happen in AI datacenters. Local models are never going to be as powerful . I think this point should be obvious: all of the current frontier models (closed and open-weights) are far too big to run on anything but a full GPU cluster in a datacenter. Of course, smaller models are getting more intelligent over time. In a year you might be able to run something about as strong as GPT-5.6-Sol on your laptop. But by then, you’ll think of GPT-5.6-Sol as too weak to be useful. Many people deny this last point, but it’s true: almost everyone’s revealed preference is to use the strongest available model in their price range . If AI progress had stalled at GPT-4, I think we could have built some very powerful tools around it, but who’d use GPT-4 today? As LLMs have gotten more capable, our expectations around them have grown: we now expect agentic systems to be able to solve more and more problems independently. It’s intensely frustrating when they get confused or stall out. When given a choice, people are going to pick the model that frustrates them less, which is always going to be the bigger, more powerful one. On top of that, datacenter models are always going to be cheaper . I don’t understand why people keep saying that local models are cheap: it seems to me to be the same mistake people make when they say that driving Uber is “free money” (ignoring the costs of fuel and wear-and-tear on your car). For the setup price alone of a low-end home lab 1 , you could buy several years of a paid subscription to one of the AI providers. The power costs would come out to around $50-$300 per month, depending on how much inference you’re running: again, the price of a couple more paid subscriptions. Why are datacenter models cheaper? It’s not because datacenter inference is subsidized: inference is actually fairly cheap . If you’re running the same model locally and in a datacenter, the datacenter model will be inherently more efficient . The main reason is batching . A GPU can do hundreds of thousands of mathematical operations exactly as quickly as it can do one. However, for a single user’s inference, each new token depends on the result of the previous one, so it can’t be batched 2 . What can be batched is the inference of hundreds of users together. This costs essentially as much time, power, and heat as just doing inference for one user at a time. When you’re running your own inference at home, you’ve got nothing to batch — at best you’re running a few parallel AI agents — so utilization is terrible. There’s a lot of potential inference that you’re paying for but can’t use: it’s just being wasted. The only way around this is to get together with some friends and expose your local inference endpoint to them (at which point you’re basically running your own crappy datacenter). The other reason is that datacenters have larger, more efficient GPUs to work with . The kind of consumer GPUs you’d run local models on are gaming GPUs like the RTX 4090. A datacenter B200, designed for batched AI inference, gets about three times the flops and just under four times the memory bandwidth for the same amount of power 3 . So between batching and GPU efficiency, you’re using something like ~30x the resources to run your model locally. Incidentally, this is why I’m suspicious of people who say that local models are good because they aren’t as resource-hungry as those big bad datacenters. If you want to run LLMs efficiently, you should be trying to push as much of your use into AI datacenters as possible! Charitably, what they mean is that we should all be running smaller models — but even then, you should ideally be using small models via, say, the GPT-5.6 Luna API instead of hosting your own model. Is there a possible world in which local models win? I suppose so. One thing that could happen is that governments could ban the use of AI datacenters altogether: either due to concerns around the danger of AI, or simply bending to public pressure . In that world, local models would be the only game in town. Alternatively, AI progress might somehow stall for very large models while progressing for small ones. I struggle to imagine how this might happen (barring government intervention, as above), but a world where a 30B parameter model could be a frontier model is a world where local models might be competitive. Or maybe models get so good that a 30B model is genuinely smart enough to do everything, so nobody really needs a model like Opus or Sol unless they’re trying to solve the Reimann Hypothesis. I don’t really buy this. Models can do frontier mathematical work today while still being not smart enough to refactor large codebases as well as me, so it’s hard to imagine a world where I don’t just want to use the smartest model available. I do think there will always be a niche for local models. I’m reminded of the surprisingly simple idea behind Thinking Machines’ “Interaction Models” (which OpenAI also does , because it’s obvious): for latency-sensitive applications like voice chat, you have a small, fast model handle the talking, which delegates to a large, slower model for the hard thinking. I wouldn’t be surprised if most AI use in five years is mediated through a local model on your phone or laptop (though in this world almost all the work would still be done via AI datacenters). Some users will prefer local models even though they’re weaker and more expensive. For instance, being able to steer the model locally might be a killer feature for those users. Others might simply value having total control over their own infrastructure, or have unreliable internet 4 . If you’re one of those people — particularly if you only chat to the models instead of using them for research or coding — local models are a good choice for you. However, I think this is always going to be a niche group. The majority of users will continue to do their inference through datacenters. This link is from a year ago — things are significantly more expensive now. Specifically, the bottleneck is moving the model weights into the GPU, which needs to be done and takes the same amount of time whether you’re doing it for one user’s token or a hundred users’ tokens. I estimated this with LLM assistance, but you can check the numbers yourself from NVIDIA. While still having a reliable power supply and enough money to fit out a home inference cluster. This link is from a year ago — things are significantly more expensive now. ↩ Specifically, the bottleneck is moving the model weights into the GPU, which needs to be done and takes the same amount of time whether you’re doing it for one user’s token or a hundred users’ tokens. ↩ I estimated this with LLM assistance, but you can check the numbers yourself from NVIDIA. ↩ While still having a reliable power supply and enough money to fit out a home inference cluster. ↩

ai AI cloud Cloud
0 views
Entropic Thoughts Today

A noob learns FFT

I have never done any time–frequency transformations. I understand the general idea, but I have never worked with the details. Today I had a reason to use it, but that means learning the basics first. The following is three seconds of a single 5 Hz sine wave, recorded at a sample rate of 100 Hz. (Continue reading the full article on the web.)

programming Programming
0 views
Unsung Today

Linear’s visual key feedback

The task manager app Linear does something interesting I have not seen before. In the keyboard shortcut tooltips, it highlights the modifier keys you already pressed, to give you confirmation you’re on the right track: I think this is nice, particularly given that the modifier key situation is kind of a mess , and particularly if you consider some keyboards present their modifier keys like this: = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/linears-visual-key-feedback/3.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/linears-visual-key-feedback/3.1600w.avif" type="image/avif"> I think it’s valuable to offer a connection between your fingers touching keys and something happening onscreen in real time – similarly how the blinker arrow in your car pulses exactly at the same rate as the sound it makes, to help you connect the two. However, Linear does not do this in all the contexts: While the first two videos might be simply bugs, I am not sure about the last three. It’s entirely possible this was done intentionally because these surfaces – the menus, the command palette, and the keyboard shortcut pane – do not support actually invoking the shortcuts. (Pressing the listed keys would not actually achieve anything.) But I still wonder if this was a good call. One of the most important things for any new systemic pattern is building trust: making sure it’s consistently applied in all the nooks and crannies so that the user can understand what it does, learn to rely on it, and form the habit. Just one place where it doesn’t work might make it easy to just give up on it altogether. Perhaps this is an example of how hard it is to design a cohesive system, which in Linear’s case is compounded by the fact that a lot of shortcuts start with regular keys like G and O and P, exacerbating focus issues. Either way, it’s delightful to see someone doing something new in this space. #above and beyond #keyboard #system design

design Design
0 views
ava's blog Today

book: muskism by quinn slobodian and ben tarnoff

Another book I finished in July was Muskism: A Guide for the Perplexed by Quinn Slobodian and Ben Tarnoff. It argues that instead of seeing Musk as an individual or even an exceptional outlier, he should be seen as a symptom of a bigger playbook, something other forces in society brought forward and shaped. Just like Henry Ford is where Fordism gets the name, the authors argue that Muskism is an ideology based on Elon Musk. Musk primarily sells a fantasy: That, if they buy into his walled garden, both individuals and states can be self-reliant, uninterrupted by grid failures, geopolitics and more. This is sold as techno-sovereignty, when it actually makes them rely on Musk himself. He depends on perpetual hype, online trolling/meme warfare, state symbiosis and financial fabulation to convince the public and for his businesses to even survive. In Muskism, everything is code, undesirable people are bugs, and empathy is an exploit. There is a huge focus on vertical integration, lean production, Agile-like processes and being anti-union. Sovereignty is seen as infrastructure instead of just territory, and is much more decided by access to bandwidth, compute, and orbital real estate. Governments everywhere are reliant on few companies to ensure their duties. The authors dive in deep into Musk's personal history: Joshua Haldeman, his maternal grandfather, wrote for the Technocracy Incorporated journal and dreamed of a future in which society was run under the dictatorship of engineers who would allocate resources according to scientific principles. After Canada banned Technocracy in 1940, he emigrated to South Africa with Elon's mother Maye; two years after the National Party introduced apartheid. That is where the embrace of technology as facilitator of self-reliance and hardening of racial inequalities started happening with support from IBM, Ford, and Toyota, as their tech was used, among other things, to count, track, and reallocate Black laborers, enable homegrown industries and nuclear weapons. South Africa was an early "biometric state", where the "Book of Life" identification system recorded racial classifications of every citizen. Pretoria, where Musk was born in 1971, was a prime example of fortress futurism, as it housed the executive offices and government buildings that instated and managed apartheid. As other groups were pushed to segregated townships on the outskirts, the white elite, including Musk, lived in wealthy suburbs like Waterkloof. Black labor was able to be cast aside or drawn from quickly, and entire communities were displaced for cheap industrial sites. Musk's family directly benefited from all of it, and it set the tone for the way Muskism sees most people: As faceless laborers in the background, ready to be added or deleted at a whim, as "NPCs", as he loves to call people. Many of his pursuits relied on state creations and tax payer money: Using free military GPS data for Zip2 maps, using the federally ensured stability of the US financial system for online banking (his attempt at the earlier version of X.com and his resulting PayPal involvement), and the internet, a technology invented by the government. He later grafted himself onto the state via his companies gaining a lot of government contracts: Using the increased military spending and visions of network-centric warfare after 2001 to launch small satellites via SpaceX, and then subsequently getting a $8 million, then a $100 million contract by the DARPA and the Air Force in 2004 and 2005 to build rockets and provide low-cost orbital launch vehicles and services, plus a COTS contract of $278 million in 2006, to name just a few. Later on, this provided the money and power in space to bring Starlink to life, which by 2025, made up about two thirds of all active satellites and is available in over a 100 countries. It also got plenty of government contracts and subsidies, one being $900 million for internet in rural US areas, and a $1.8 billion contract in 2021 for Starshield, a military version of Starlink. Tesla fared the same way, using the fear of foreign oil, the War on Terror and the resulting green/clean tech boom post 2001 to get massive financial funding from the government. According to the book, Diarmuid O'Connell, former chief of staff for George W. Bush's assistant secretary of state for political-military affairs lobbied for Musk and got Tesla a $465 million loan from the Department of Energy in 2008 (under the Advanced Technology Vehicles Manufacturing Loan Program), as without it, Tesla likely would have gone bankrupt. Which means: Taxpayer dollars went to a company that produced a basically unaffordable 100k car with fewer than 200 units in existence back then, just to bail it out. xAI is going the same way of using government money: In July 2025, they announced that they have entered a contract with the Department of Defense to offer a suite of AI products to them. The goal for Musk has always been that governments can only exercise their authority by purchasing services from a monopoly provider, which he tries to be, and at least with SpaceX, is. If the government cedes its sovereignty to an immense degree, it would be forced to buy it back in increments from corporations. Via this government contractor route, his companies can also evade compliance with some regulations. The authors also draw connections between Musk's favorite media and his business decisions as well as product names and more, especially The Hitchhiker's Guide to the Galaxy , Star Trek , Battlestar Galactica , Transformers and Robotech . He would grow obsessed with the idea of colonizing space, and with becoming one with the machine, which can be seen not only with Neuralink, but him saying that to avoid AI killing humanity, humanity has to become one with AI. He co-founded OpenAI with Altman over his AI fears. 2022 is when he increasingly started to share right-wing viewpoints, especially talking about the "woke mind virus". For the first time, he seemed to be very pre-occupied with the problem that, in his previously ideal solution of a merged together, cybernetic intelligence cluster of half human, half AI population, there's a risk that the system could become infected. In an interview, he shared his personal horror scenario, in which AI nannies would go around, see that there are too many white men in power, and go kill some - kill him. His fear is that for once, instead of always targeting other groups, he could be targeted and treated like he has treated them, because of rules that infected everyone's mind and turned them against him; rules that he has no control over. This fantasy is also what got him to buy Twitter, use it as a training ground and invent Grok, the "non-woke" AI and its Grokipedia. It's his idea of a safeguard against an army of AI coming to the conclusion that people like him need to be eradicated. He constantly influences the training data (by pushing his tweets artificially and giving them wider reach) and the training itself (by suddenly proclaiming that Grok is giving too woke answers, and adjusting, leading to events like the MechaHitler outburst). Despite his transhumanist ambitions, he does not consider modifying ones body within a gender transition as a good thing. A merger with AI could be dissolving the boundary between the natural and artificial, but it might open other boundaries like gender to be redrawn. But he actually wants to strengthen traditional social hierarchies, so a humanity segmented by gender, race and class is still possible. Don't forget: One reason why he hates his daughter's transition is because he does IVF sex selection and paid specifically for a male sex. He wants total power over how people function and live. His takeaway from South Africa's apartheid is that some humans are born to rule, others to be ruled, and these are the markers needed for that. That was also a big motivator behind DOGE: See the government as a platform, eliminate information silos that were there to ensure legal and privacy safeguards for privileged information, and then prepare systematic overreach, misuse and surveillance by working on ways to integrate information of different agencies into one database, to better "purge" unwanted people. For example, via the SSA's death master file, which cut off people's access to credit cards and bank accounts by declaring them dead - a form of "financial murder", where the goal was either for people to die or to "self-deport". Efficiency, in this case, became an excuse for centralization, and the Trump administration had no problem with it as it further advanced their own goals and made Musk the fall guy, as he got the blame while most things stayed the same. This entire climate paved the way for Palantir's $30 million ImmigrationOS contract with ICE, which promises near real-time visibility for non-citizens. He relies on this sort of oppression to make his businesses and lifestyle possible. To just name one example: The data center in Memphis powering Grok is in a historically Black neighborhood that originated as a community for emancipated slaves (cheap opportunity for him) and the facility's methane gas turbines emit pollutants linked to asthma, respiratory diseases, heart issues, and cancer. You can be sure that this would not be built in the kind of areas Musk grew up in. This is also visible in his increasing focus on fertility and population numbers, alleging a white genocide or a "Great Replacement". Fertility is actually fine in many countries; it's just not the right countries for him, as they aren't white. This is another remnant of his South African upbringing, in which white politicians in the apartheid era warned of being swamped by Black people and wanted to win the "cradle race" by boosting white fertility, and where it was normal for white people to live in their own settlements and schools away from the rest (as he does, with his own schools for his children). It actually is at odds with the rest of his views, as he keeps talking about robot descendants, merging with AI, and that within the next decade, humanoid robots will outnumber humans ten to one. He seems to facilitate a different kind of replacement, and it leaves you wondering what the supposed difference is between an AI-merged, cybernetically enhanced Black man and a white man, or whether human or robot does the "replacement". Why do white birth rates matter if they are going to be surpassed by robots, and if human labor will be less and less needed in his view? Towards the end of the book, the authors suggest 4 ways Musk could move forward - either a return to clean tech and electrification ("Carbon Musk", unlikely), even more state symbiosis ("Contractor Musk", inspired by Palantir's current successes), focusing harder on segregating from the rest via schools/settlements and on his white genocide bullshit ("Compound Musk", inspired by Bezos and Jon Lonsdale doing the same), or doubling down on AI and robots ("Cyborg Musk", with more Neuralink stuff). The book ends with an idea of what the ideal Muskism future could look like, slightly exaggerated at times I think, for comedic effect? Or because our future has sincerely become ridiculous? I can't tell. But I'm glad I read it. Published 10 Aug, 2026

business Business books Books culture Culture
0 views
Shrivu’s Substack Today

Exploring Claude/GPT Knowledge Cutoffs & Pre-training Timelines

We can learn hidden facts about how frontier models were trained by “probing” them with carefully curated requests. By scoring them on niche facts we can approximate how many parameters models like GPT-5 and Opus have, using “Incompressible Knowledge Probes” By measuring how the models break down tokens we can reveal facts about the datasets mixtures they used to train the model (or at least the tokenizer) using “Data Mixture Inference” By scoring them on date or self-identification related questions you can also estimate training timelines ( this post ) Everything here is an estimate. It’s possible that some speculation in this post is totally incorrect given there’s not a ton of publicly available ground truth to verify against. The 3 main stages of model training. As a brief primer (see Alex Wa’s blog for more), how we train massive large language models has converaged into 3 stages: Take a massive amount of general purpose data (aka scrape the internet) and “pre-train” a massive auto-complete model on that data. Use domain-specific “textbook quality” data to improve the base models and extend certain base capabilities like long-text understanding Turn the base model into the “assistant” persona, honing in on its personality, reasoning ability, and tool-calling. While increasingly more compute is spent on post-training for boosting a model’s reasoning and problem solving, one of the most expensive and data-intensive steps is generating that pre-training checkpoint (by ‘checkpoint’ think of a massive file). While all labs operate slightly differently, what you might see (~ page 44 ) is: The “pre-training” team kicks off and babysits a multi-month run to get a base checkpoint. These pre-training models often, but not always, imply major versions of released models (GPT-4 → GPT-5). While that’s happening, the “capability” and “post-training” teams will run experiments for how to improve on the most recent base model. Advancements in post-training and capabilities often manifest as minor versions of released models. These teams often also “distill” a single post-trained model into smaller variants that become model families (Fable/Opus/Sonnet/Haiku, Sol/Terra/Luna). Labs may also release post-trained models from half-baked pre-training checkpoints as soon as x% of the version N+1 checkpoint is better than the 100% baked version N checkpoint. The model released to the public is the culmination of the most recent checkpoint with the best set of capabilities and post-training techniques applied to it. With this in mind, I was curious how much of this process you can “see” just by probing the model over the official APIs. To estimate the pre-training checkpoint dates, I constructed a dataset of daily-facts from Wikipedia (e.g. 2025 in the United States ) and gave every model an 8-way multiple choice quiz on what happened on a given day. Then, by analyzing the error rate timeline, you can see roughly when it loses signal from its training dataset. View the full dataset. For GPT-5.4, you can see the fact-estimated knowledge cutoff is aligned with OpenAI’s published knowledge cutoff. It’s smooth as an artifact of the model being better able to guess near-future events as well as recent in-data events being undersampled during training. This is also why I use the midpoint rather than the start or end of the curve. You can then plot this for all models. View the full dataset. Estimated knowledge cutoffs and published dates. Comparing model curves side-by-side in the full viewer makes it easier to see roughly where the cutoff occurred and how steep it is. After staring at these charts for a bit, here’s what I’m speculating: Anthropic models Opus 4.7 onwards are all from the same training run that cuts off just around late December 2025. This is derived from how they all share a very similar effective knowledge cutoff (green). A core assumption I’m making here is that the pre-training base model completion date is highly correlated with the dataset timespan used, if that’s wrong these results could be off by some offset (e.g. it’s actually Jan 2026). It’s also interesting that Opus 4.7+ models all have a published reliable and overall knowledge cutoff that’s identical — maybe that’s due to a new post-training technique being used? OpenAI’s GPT-5.6 family comes from their own checkpoint, separate from GPT-5.5, that finished around late February 2026. This is derived from how they have a distinct effective knowledge cutoff from previous models. You’ll notice Luna looks like it can predict the future — that’s more of an artifact of it having a high error rate all around at a “low” reasoning effort. Opus 5 is a bit unusual. The published reliable and overall knowledge cutoffs are May 2026 and yet it seems to know nothing more than previous Jan 2026 cutoff models. I did several ablations to test whether it was an artifact of the types of probing questions I used, but not really — the cutoff applies to recall on coding package versions as well. What if you just ask the model what today is? View the full dataset. On the dashed diagonal a model's self-model matches its factual knowledge; below it the model thinks it is living in its own past. Recent OpenAI models are excluded since the API injects the actual date into all requests (TIL! This feature really annoys a lot of folks). It ends up being fairly correlated with fact-based estimates. If you look closely you can see some vertical lines within a few of the families of models. GPT-4.1 nano → GPT-4.1 mini → GPT-4.1 Opus 4.7 → Sonnet 5 → Fable/Opus 5 Interpreting this graph as X = “pre-training corpus” and Y = “post-trained behavior”, these vertical strips (X constant, Y increasing) visualize active post-training on recency-biased datasets. Potentially distillation from old copies of teacher models is what causes smaller models to self-report older dates. You can also make predictions on training timelines and datasets indirectly by looking at who the models think they are. The more a model sees “I am X” in its pre-training dataset, the more likely it is to repeat that when pushed and given no other grounding context. Full dataset. Each row is a real model; each column is a self-claimed identity extracted from 50 “what model are you?” probes (5 phrasings × 10 samples, guess-nudged, no system prompt). Cell shade = share of the model's replies claiming that name; green outline = the claim matches the model's true family (bold outline = exact version — which never happened. Some more neat visuals. After staring at these charts for a bit: Vertical bands show clear patterns of labs training on past-model outputs (from users). For OpenAI it’s GPT-4, GPT-4o, GPT-4.1 for a bit, GPT-5 and “ChatGPT” most recently. For Anthropic it’s 3.5 Sonnet then more recent models swap to Sonnet 4.5. This seems to align pretty well with training on chats from ChatGPT.com and Claude.ai respectively, where users chatted with the latest model and whose sessions became training material (directly or via web contamination). It seems unlikely to me these are coming from internal synthetic datasets given those would be much easier to suppress model identity (vs being embedded in the system prompt in the consumer chat sessions). Training-on-chats isn’t novel information but it is interesting to see expressed literally with probing like this. It’s a bit spicy that OpenAI models never identify as another lab’s model (besides briefly a Tesla Model S) yet Anthropic’s Sonnet 5 will regularly self-identify as GPT-4. It feels very unlikely that they are intentionally distilling GPT-4 but it’s possible a bunch of older ChatGPT chats are still making their way into the Claude training mixtures. It could also be carried through the generations via Sonnet 3.5 lineage (i.e. Sonnet 5 is trained on Sonnet 3.5 data which itself might have had a very GPT-4 heavy dataset). On top of this, in a follow-up experiment, when asked to answer identity questions “as model X would,” Claudes reproduce OpenAI models’ measured quirks at 68%; OpenAI models manage 8% on Claudes. That’s it. Hope that was mildly interesting and if you want to explore the data a bit, here are the links: Model knowledge timeline What month does each model think it is? Does the model know its own name? Identity vintage: which era does each model think it is from? Thanks for reading Shrivu’s Substack! Subscribe for free to receive new posts and support my work. By scoring them on niche facts we can approximate how many parameters models like GPT-5 and Opus have, using “Incompressible Knowledge Probes” By measuring how the models break down tokens we can reveal facts about the datasets mixtures they used to train the model (or at least the tokenizer) using “Data Mixture Inference” By scoring them on date or self-identification related questions you can also estimate training timelines ( this post ) The 3 main stages of model training. As a brief primer (see Alex Wa’s blog for more), how we train massive large language models has converaged into 3 stages: Take a massive amount of general purpose data (aka scrape the internet) and “pre-train” a massive auto-complete model on that data. Use domain-specific “textbook quality” data to improve the base models and extend certain base capabilities like long-text understanding Turn the base model into the “assistant” persona, honing in on its personality, reasoning ability, and tool-calling. The “pre-training” team kicks off and babysits a multi-month run to get a base checkpoint. These pre-training models often, but not always, imply major versions of released models (GPT-4 → GPT-5). While that’s happening, the “capability” and “post-training” teams will run experiments for how to improve on the most recent base model. Advancements in post-training and capabilities often manifest as minor versions of released models. These teams often also “distill” a single post-trained model into smaller variants that become model families (Fable/Opus/Sonnet/Haiku, Sol/Terra/Luna). Labs may also release post-trained models from half-baked pre-training checkpoints as soon as x% of the version N+1 checkpoint is better than the 100% baked version N checkpoint. The model released to the public is the culmination of the most recent checkpoint with the best set of capabilities and post-training techniques applied to it. View the full dataset. For GPT-5.4, you can see the fact-estimated knowledge cutoff is aligned with OpenAI’s published knowledge cutoff. It’s smooth as an artifact of the model being better able to guess near-future events as well as recent in-data events being undersampled during training. This is also why I use the midpoint rather than the start or end of the curve. You can then plot this for all models. View the full dataset. Estimated knowledge cutoffs and published dates. Comparing model curves side-by-side in the full viewer makes it easier to see roughly where the cutoff occurred and how steep it is. After staring at these charts for a bit, here’s what I’m speculating: Anthropic models Opus 4.7 onwards are all from the same training run that cuts off just around late December 2025. This is derived from how they all share a very similar effective knowledge cutoff (green). A core assumption I’m making here is that the pre-training base model completion date is highly correlated with the dataset timespan used, if that’s wrong these results could be off by some offset (e.g. it’s actually Jan 2026). It’s also interesting that Opus 4.7+ models all have a published reliable and overall knowledge cutoff that’s identical — maybe that’s due to a new post-training technique being used? OpenAI’s GPT-5.6 family comes from their own checkpoint, separate from GPT-5.5, that finished around late February 2026. This is derived from how they have a distinct effective knowledge cutoff from previous models. You’ll notice Luna looks like it can predict the future — that’s more of an artifact of it having a high error rate all around at a “low” reasoning effort. Opus 5 is a bit unusual. The published reliable and overall knowledge cutoffs are May 2026 and yet it seems to know nothing more than previous Jan 2026 cutoff models. I did several ablations to test whether it was an artifact of the types of probing questions I used, but not really — the cutoff applies to recall on coding package versions as well. View the full dataset. On the dashed diagonal a model's self-model matches its factual knowledge; below it the model thinks it is living in its own past. Recent OpenAI models are excluded since the API injects the actual date into all requests (TIL! This feature really annoys a lot of folks). It ends up being fairly correlated with fact-based estimates. If you look closely you can see some vertical lines within a few of the families of models. GPT-4.1 nano → GPT-4.1 mini → GPT-4.1 Opus 4.7 → Sonnet 5 → Fable/Opus 5 Full dataset. Each row is a real model; each column is a self-claimed identity extracted from 50 “what model are you?” probes (5 phrasings × 10 samples, guess-nudged, no system prompt). Cell shade = share of the model's replies claiming that name; green outline = the claim matches the model's true family (bold outline = exact version — which never happened. Some more neat visuals. After staring at these charts for a bit: Vertical bands show clear patterns of labs training on past-model outputs (from users). For OpenAI it’s GPT-4, GPT-4o, GPT-4.1 for a bit, GPT-5 and “ChatGPT” most recently. For Anthropic it’s 3.5 Sonnet then more recent models swap to Sonnet 4.5. This seems to align pretty well with training on chats from ChatGPT.com and Claude.ai respectively, where users chatted with the latest model and whose sessions became training material (directly or via web contamination). It seems unlikely to me these are coming from internal synthetic datasets given those would be much easier to suppress model identity (vs being embedded in the system prompt in the consumer chat sessions). Training-on-chats isn’t novel information but it is interesting to see expressed literally with probing like this. It’s a bit spicy that OpenAI models never identify as another lab’s model (besides briefly a Tesla Model S) yet Anthropic’s Sonnet 5 will regularly self-identify as GPT-4. It feels very unlikely that they are intentionally distilling GPT-4 but it’s possible a bunch of older ChatGPT chats are still making their way into the Claude training mixtures. It could also be carried through the generations via Sonnet 3.5 lineage (i.e. Sonnet 5 is trained on Sonnet 3.5 data which itself might have had a very GPT-4 heavy dataset). On top of this, in a follow-up experiment, when asked to answer identity questions “as model X would,” Claudes reproduce OpenAI models’ measured quirks at 68%; OpenAI models manage 8% on Claudes. Model knowledge timeline What month does each model think it is? Does the model know its own name? Identity vintage: which era does each model think it is from?

data-analysis Data Analysis ai AI machine-learning Machine Learning
0 views
Stratechery Yesterday

Apple Earnings, More on Amazon’s Earnings

Apple's earnings (and stock) are limited not by memory but rather chip shortages; then, more on Amazon's earnings and Andy Jassy's market analysis.

Finance Electronics
0 views
Manuel Moreale Yesterday

Raising the bar

I just went through the queue of submissions to blogroll.org and probably deleted 95% of what was in there. This is not unusual, and has become more and more common lately. A big reason for this is clearly AI. The amount of slop submissions I’m getting is trending upwards, unfortunately. The other big reason is people attempting to use the blogroll as a way to get traffic for their shitty marketing sites, which is obviously annoying and a waste of my time. There’s people who have submitted their stupid marketing sites multiple times, and I’m starting to contemplate the possibility of blacklisting domains because some people out there clearly don’t know how to behave. But all this is making me question the value of big lists in an age where content is becoming more and more abundant. There are people out there—idiots, if you ask me—who think that coding has become a commodity, that everyone can make a site with just a few prompts, and that producing content worth consuming has become so incredibly cheap and easy. If you believe in all that, what even is the point of having a big list of sites worth consuming since content worth consuming should just flood every corner of the web and be easily accessible? I do not believe that to be the case. I believe, in fact, the opposite to be true. Good content, content worth consuming, is becoming harder to find, and it’s getting buried under a sea of low-quality crap, which makes me wonder if it’s time to start rethinking how blogroll.org works. I have ideas for how to tweak both the site itself as well as the submission process (and no, that doesn’t involve the exchange of money), but it’s obvious to me that the only way out is to raise the barrier to entry, which is pretty much the opposite of what the web has been all about for the past 20 years. Thank you for keeping RSS alive. You're awesome. Connect via email :: Sign my guestbook :: Support for 1$/month

web Web ai AI
0 views
Unsung Yesterday

“If you aim for the king, you better not show up eight years late.”

Something different for today: a one-hour episode of a podcast A Life Well Wasted from 2025. (I can’t link to it directly, but it’s the most recent episode, called “Game Over.”) = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/if-you-aim-for-the-king-you-better-not-show-up-eight-years-late/1.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/if-you-aim-for-the-king-you-better-not-show-up-eight-years-late/1.1600w.avif" type="image/avif"> It’s a human story whose beats might be familiar to some of you: a long software project that ultimately failed despite the enormous effort. And it did so in an industry – high-budget videogames – where failures might feel particularly brutal: the projects take multiple years but the defeat can be swift, the servers get shut down and the game instantly evaporates, and instead of employees being reallocated to other games, the studio gets disbanded and people let go. There are some nice moments in the podcast’s interviews with about a dozen people, hearing about personal pride and responsibility, working together with others, and a certain camaraderie not just with other people, but with their software that develops: I think the best part about it was being able to play with my coworkers and then cutting loose, and being silly, and really enjoying what we made together, as a unit. That experience alone made it worth it. There are also questions about the management’s role in all of this – this part we might never get to know fully – and the worry about game preservation. This is not mentioned in the podcast, but it seems widely understood the story is about Concord , a AAA live-service game that is rumored to have cost a staggering $400 million dollars and taken 8 years to develop, only to be shut down mere 12 days after its launch in the late 2024. ( AAA means a blockbuster with highest budgets seen by the industry, and live service means a game like Fortnite, which is expected to make money over time from add-ons and upgrades.) Learning about Concord’s macro view adds a lot of color to the boots-on-the-ground podcast above, but one has to be careful exploring it; the online discourse about the game felt similar to the 2016 reboot of the movie Ghostbusters where sure, the product might have been subpar, but also a lot of commenters seemed eager to arrive to the conversation carrying truckloads of bad faith, gatekeeping, and misogyny. Some good articles? Keza MacDonald in the Guardian has a nice summary of the whole situation: This is a brutal sequence of events. Sony bought the makers of Concord, Firewalk Studios, in 2023. Concord had been in development for eight years, and it was an expensive game, with bespoke cinematics and a long-term plan that would have cost $100m or more to develop. In its two weeks on the market, it sold fewer than 25,000 copies, according to estimates. This is a shocker, even compared with the year’s other bad news for developers and studios. MacDonald also adds: Speaking personally, I do not want a game that takes years to play. I want one with something to say, an experience to impart, and one that eventually ends. A game whose artistry comes before its business model. This is partly a matter of taste. Self-evidently, there is an enormous market for live-service multiplayer games; it’s just that most of those people are already playing one. I highly doubt that there are untapped millions of players desperate for a hero shooter or battle royale game who just haven’t found the right one yet. It’s time that publishers try something new instead. An article by Matt Kim at IGN adds to this notion: With so many games now taking close to a decade from the beginning of development to release, we’re starting to see the financial and creative consequences of an overlong development cycle. Spend too much time in development and ideas that were once novel are no longer in vogue. Furthermore, the time and money spent over those years has to be recouped somehow, which leads to decisions like the $40 cost of entry for Concord when many of its peers are free to play. The cost of coming late to the party means you must bring something new to the table. Unfortunately, Concord is neither particularly innovative nor content-heavy. That said, it does have a level of polish at launch that was often absent from its hero shooter peers when they were first released. Indeed, Concord’s weekly animation story drops are fully motion-capture, and Firewalk’s time spent on crafting its lore has helped secure Concord an episode of this winter’s video game animation anthology series, Secret Level . But well-established hero shooters like EA’s Apex Legends launched almost bare bones and still managed to make a splash thanks to its intriguing central concept which combined hero loadouts with a battle royale match format. Valve’s Deadlock doesn’t even have finalized assets or art but has still caused a huge burst of excitement among the PC community, thanks to the way it changes up the classic 6v6 hero formula with its heavy lane-and-minions MOBA [Multiplayer Online Battle Arena—ed.] mechanics. By contrast Concord appeared with an all-too-familiar offering and, frankly, the time spent on finessing its presentation – the graphics, motion capture, performance, and so on – likely lead to a later release date which in turn meant it lost valuable time establishing itself among its peers. If it had been released four or five years ago, when the PS5 first came out, maybe its launch would have been an entirely different story. I think this is important to quote on this blog that often talks about “finessing” and implicitly – or sometimes explicitly – about the value of taking time to get the details right. We can’t forget that there are such things as overdesigning and overproducing, and that ultimately there is no way to polish your way out of something that lacks a soul. #craft #culture #games #podcast #software eulogies

swift Swift programming Programming
0 views
Unsung Yesterday

Arno Beck’s art

I am generally not a fan of typewritten art, because at some point it all starts to feel a bit same’y, and the gimmick of using a typewriter as a paintbrush wears off pretty quickly. = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/arno-becks-art/1.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/arno-becks-art/1.1600w.avif" type="image/avif"> But these drawings by German artist Arno Beck caught my attention, because it feels like they playfully remix three eras: = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/arno-becks-art/2.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/arno-becks-art/2.1600w.avif" type="image/avif"> = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/arno-becks-art/3.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/arno-becks-art/3.1600w.avif" type="image/avif"> = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/arno-becks-art/4.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/arno-becks-art/4.1600w.avif" type="image/avif"> = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/arno-becks-art/5.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/arno-becks-art/5.1600w.avif" type="image/avif"> In case this interests you, in 2018 I gave a 46-minute talk called “The abridged history of having fun with keyboards” that’s a (hopefully fun) walkthrough of this whole space: = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/arno-becks-art/yt1-play.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/arno-becks-art/yt1-play.1600w.avif" type="image/avif"> #art #conference talk #games #marcin wichary #youtube overtyping keyboard art (early 1900s), the sort of photorealistic smooth shading I most associate with ASCII/​ANSI art (1990s), big pixels from early home videogames (1980s).

culture Culture design Design gaming Gaming
0 views
Adrian Krebs Yesterday

The death of AI workflow builders

The visual no-code editors looked like the future of AI agents. Less than a year later, coding agents took over.

ai AI
0 views
baby steps Yesterday

Cylic trait implementations: motivation

Lately I’ve been thinking about cyclic trait implementations. This is a problem that I’ve been trying to understand for years and years and I finally feel like I’m geting somewhere. I’m going to try to write out a series of blog posts documenting those explorations and, hopefully, culminating in a design that could be RFC’d. In this first post, I want to talk about one of the interesting questions, what I am going to call “internal” vs “external” proofs. I know that this material can seem abstract, so I’m going to try and connect it to “real Rust” as much as possible! This particular blog post is an introduction, explaining the general problem and giving some motivation for why we care. Right now in Rust we require most traits to have non-cyclic , or inductive , implementations. To explain what I mean, let’s consider this trait: Now imagine that we have an impl of this for : A simple impl for and `Option : and finally a recursive type that has an impl as well: If I try to show that , I do that by There’s no cycle here – that is, I didn’t have to use impl L to show that impl L is valid. Now, when I said that “the impl L didn’t have to use the impl L to show that it is valid” that might not have sounded suspicious to you. In fact, it’s a pretty natural idea. After all, generally when you try to establish a logical argument, you aren’t allowed to use cyclic reasoning. That is, you can’t say: I know that Niko likes Rust because Niko likes Rust. So, in the same sense, it seems natural that I should not be able to say “I know that implements because implements ”. But actually, it would sometimes be really useful to say exactly that. One example is so-called “perfect derive”. In our impl above, we had one where-clause, . And if you were to create a custom derive for and write , the impl I showed is typically exactly what you would get. But it’s not necessarily what you want . Consider what you get with : Here, the derive is going to create an impl that requires . But if you look closely, you’ll see that all the fields only use , so in fact, we should be able to clone a even without ! But how is the compiler to know this? You might think that the compiler could do some super smarty-pants analysis on the fields to figure it out. And, in a way, it can: that is what cyclic trait solving is all about. The thing is, while the compiler can do that, the derive cannot – the derive doesn’t have access to the definitions of other types and so forth, and clearly we would need to know things about and to figure out whether is required here. But what we could do is to generate a different impl. Instead of adding for each type parameter, we could add a where-clause for each field type. This makes sense: after all, we are just going to be calling on every field, so it’s quite logical to say that the impl is valid if every field is cloneable: Under this formulation, we can see that all we have to be able to do is to clone an and clone an , neither of which require that . We call this idea [perfect derive][] and it’s been a goal for a while. The thing is, cyclic reasoning is tricky to get right. The example is actually an easy one: that one doesn’t really require cyclic reasoning: But if we use that same “cyclic derive pattern” to generate our impl, things don’t work out so well. Instead of just a bound, our impl now has two bounds: Now imagine we try to show . We begin by applying impl L1, which requires us to show that its where clauses hold: Ugh. Something’s tricky here! Now, maybe you think we can just accept any cycles. And for these examples, it would be fine: but it’s not correct if you consider supertraits . Consider this trait and impl pair: If you are naive, this weird trait-impl pair can be used to prove that any type is , regardless of whether it has a impl. For example: Uh oh, now we did something wrong. We proved that even though there is no impl. Something is fishy. Now clearly we can all see the problem here – the implementation of didn’t really add any information. It was just a tautology, saying that if . It’s not wrong , but implementing was supposed to tell us more than just the fact that there is an impl of , it was supposed to tell us also that the supertrait is implemented. And that’s not true here. But if you think about it, it’s hard to decide why we should reject impl M but accept the impl L1 of for . They both wind up with a cyclic proof. So what’s the difference? This is the question we’ll be exploring over the next few blog posts. This gets at an interesting question: what does it mean for the trait system to be sound . This seems obvious but actually it was a question I found kind of non-obvious for a long time. We’ve found two satisfactory answers to that question. One of them involves converting to dictionary-passing style. Nadri explained that in a blog post . I think that’s a great post to read. I’m going to give another definition here that doesn’t require converting to a dependently typed program 2 My rough definition is this 3 : the trait system is sound if, whenever it accepts some program P, that program cannot have a function that believes some holds for the , but there is no impl of that can be used. So in the case of and , it’s easy to write a program that shows simple cyclic trait solving is unsound: By my definition, any sound type/trait system must reject this program because, if it were to execute, then execution would reach and yet there is no impl that is judged to ber applicable to . Uh oh! As I promised, this post was mostly focused on “setting the scene”. My goal was to explain what the problem is that we are trying to solve – permitting “good cyclic impls” but forbidding bad ones. I didn’t spend a lot of time on the bad ones, but it turns out that there’s a wide variety of unsound things one can do, some of which the compiler currently gets wrong, others of which it would only get wrong if we started permitting cycles. My motivation for getting into this work is a bit complicated. I want perfect derive. But it’s also a loose end in our trait semantics that I really want to see nailed down before we move onto other tasks. Having auto traits (e.g., ) work differently from other traits is clearly a “smell”, and without a strong understanding of the logical underpinnings of our trait system it’s easy to get things wrong when we build extensions. In the next few posts I’ll go a bit deeper into the exploration I and others have been doing. I’ll talk about some of the “false starts” we took along the way and why they don’t work, and then about some of the solutions that are under consideration. Working through this stuff has really helped me to broaden my understanding of various areas of logic. By the time we’re done, we’ll cover 4 coinduction and productivity, modal logic and the later modality, and we’ll see how our techniques might even help us with resolving specialization 5 . I cited it earlier, but if you want to read other tasks on the same subject, I definitely recommend Nadri’s post on dictionary-passing style . Apart from the default bound, I’m ignoring that here  ↩︎ I have found that both the dictionary-passing interpretation and the logic approach I’m using are valuable. In the end, they’re more or less equivalent, which I guess shouldn’t be surprising if you’ve heard of the Curry Howard Correspondence , but I’ll talk about that later perhaps.  ↩︎ I would like to, but haven’t, define a simplified version of Rust that includes trait solving and simple type checkoing and show that it cannot “go wrong” .  ↩︎ In a shallow way, I’m no expert!  ↩︎ Plot twist, bet you didn’t see that coming! I sure didn’t.  ↩︎ Applying “impl L” to show that if Then applying “impl I” to show that To show that we have to show that… , which is easy because doesn’t have any where-clauses 1 uses the impl which requires… , which is again easy To show we use impl L1, which has two where-clauses: , this one is easy because the impl requires that which is true. But is tricky. The impl requires that… We need to prove , and then the impl requires that… We need to prove , but that is what we started with! That’s cyclic logic! Say we want to prove that . We observe that if a type implements , it must implement , so… We begin by proving . We use the impl M, which requires that we show , which is a cycle, so we accept it. Apart from the default bound, I’m ignoring that here  ↩︎ I have found that both the dictionary-passing interpretation and the logic approach I’m using are valuable. In the end, they’re more or less equivalent, which I guess shouldn’t be surprising if you’ve heard of the Curry Howard Correspondence , but I’ll talk about that later perhaps.  ↩︎ I would like to, but haven’t, define a simplified version of Rust that includes trait solving and simple type checkoing and show that it cannot “go wrong” .  ↩︎ In a shallow way, I’m no expert!  ↩︎ Plot twist, bet you didn’t see that coming! I sure didn’t.  ↩︎

programming Programming rust Rust
0 views
ENOSUCHBLOG Yesterday

GitHub Actions needs OIDC audience constraints

TL;DR : GitHub Actions should allow end-users to express audience constraints , to make it harder for an attacker to pivot across services that use independent OIDC-bearing jobs. They could do this with relatively small syntax tweak, although the backend implications are probably nontrivial. Like many CI/CD providers, GitHub Actions provides verifiable machine identities 1 via OpenID Connect (OIDC). These are awesome for a lot of reasons, not least of which is that they allow workflows running on GitHub Actions to federate with other (third-party) services without GitHub having to intermediate and pre-bless every interaction. This is the backbone of how both Trusted Publishing and Sigstore work: an individual workflows on GitHub Actions presents its machine identity (via an OIDC token) to an external service, which then authenticates and for some purpose (uploading to PyPI or signing artifacts, respectively). Unfortunately, GitHub’s mechanism for exposing OIDC tokens in workflows contains a significant weakness, one that (in my opinion) will present an increasingly serious security risk over time. This post is about that weakness. At the core of all “OIDC in CI/CD” implementations is some mechanism that allows the workflow (pipeline definition, etc.) to request or otherwise be pre-loaded with an OIDC identity. Here’s how that looks in GitLab CI/CD: and here’s the equivalent in GitHub Actions: The difference between these two is small but important: GitLab requires the OIDC audience (the ) to be declared up-front and statically , while GitHub requires the workflow to dynamically request a token with an audience selected at runtime (the parameter in the HTTP request). First, a very quick foray into OIDC. Under the hood, OIDC is (mostly) just OAuth 2.0 , and OIDC identity tokens are just JSON Web Tokens (JWTs), with some additional 2 constrains on the claims they should express. The most important claim in an OIDC ID token is arguably , since it identifies the principal (the “subject”) 3 . However, the second most important claim is , for the audience . The audience is critical because it constrains who honors the token : services that accept ID tokens should only do so when they recognize the audience as matching theirs. In other words: the claim prevents an ID token that’s intentionally been issued for a specific service from being stolen by the attacker and mis-applied to another service. This is intended as a defense-in-depth: even if an attacker manages to exfiltrate an OIDC credential, they should not be able to pivot to other services it. It’s a flimsy defense but one that’s generally effective 4 , unless you give the attacker the ability to control the claim as well. Unfortunately, that’s exactly what GitHub Actions enables 5 : gives the job (or entire workflow) the ability to mint any ID token it pleases, with any audience. This matters a great deal in a world (our world) where jobs that are given also run a lot of third-party code: any vulnerability (or malware) in that code has the potential to ask for new ID tokens for audiences that it isn’t supposed to have access to. We thought about this problem when designing Trusted Publishing, and came to the conclusion that the machine identity that Trusted Publishing uses must include the workflow name, preventing an attacker from impersonating by inducing an ID token from with . However, this constraint is not suitable for all possible use cases: many integrations want to use just the slug as a sufficient identity, meaning that all workflows are effectively co-equal when issuing ID tokens. Add constraints! Ideally, something like this: The basic idea here is to constrain what audiences the job can request ID tokens for. In the example above, attempting to request a token for would cause an error, preventing a job that’s intended only for publishing to PyPI from serving as a pivot to AWS. There are, of course, some potential downsides to this. For example, some services might (inadvisedly) require dynamic information in their audience, meaning that a value can’t be statically pre-declared. I think this is rare enough in practice to not be worth blocking a security improvement and, when they do occur, GitHub could continue to allow the form as a (less secure!) catch-all. More precisely, “workload identities.” But the distinction is not relevant here.  ↩ Not very stringent.  ↩ In practice, the claim is often a mess, since it’s typically an opaque string that uses or similar as a contentional delimiter. This causes all kinds of security bugs (e.g. when attacker-controlled components of the subject can also contain the delimiter), which is why in practice Trusted Publishing and Sigstore both emphasize IdP-specific claims that represent each part of the subject’s identity rather than a bespoke composite form.  ↩ Services can (and do) have bugs, like neglecting to check the audience or allowing it to vary with other potentially attacker-controlled claims. But we’re assuming services that do properly check the audience here, like PyPI’s Trusted Publishing does.  ↩ Others appear to do the same thing, although I’m less familiar with other CI/CD providers. For example, BuildKite’s OIDC flow involves invoking , which involves the same level of runtime control of the audience.  ↩ More precisely, “workload identities.” But the distinction is not relevant here.  ↩ Not very stringent.  ↩ In practice, the claim is often a mess, since it’s typically an opaque string that uses or similar as a contentional delimiter. This causes all kinds of security bugs (e.g. when attacker-controlled components of the subject can also contain the delimiter), which is why in practice Trusted Publishing and Sigstore both emphasize IdP-specific claims that represent each part of the subject’s identity rather than a bespoke composite form.  ↩ Services can (and do) have bugs, like neglecting to check the audience or allowing it to vary with other potentially attacker-controlled claims. But we’re assuming services that do properly check the audience here, like PyPI’s Trusted Publishing does.  ↩ Others appear to do the same thing, although I’m less familiar with other CI/CD providers. For example, BuildKite’s OIDC flow involves invoking , which involves the same level of runtime control of the audience.  ↩

devops DevOps security Security json JSON ci-cd Ci Cd
0 views
Abhinav Sarkar Yesterday

Fast Haskell Scripts on GitHub Actions

Magix is a neat tool that lets us run Haskell programs as scripts 1 . We put a shebang on top mentioning Magix, list the Haskell packages we need, and just works. This post is about running such a script fast(er) on GitHub Actions . This post was originally published on abhinavsarkar.net . As our example, we’ll take the static site generator (SSG) I wrote some time ago: BlogShake . It is written as a single Haskell file. It uses Shake to build the website, Pandoc to render posts, and Mustache for templates. The script starts with these Magix directives 2 : Running the script is as simple as: Magix compiles the script into an executable and runs it. Nothing else to install, no , or required. Nix and Cabal can also run scripts by providing shebang directives 3 , so why reach for Magix instead? The shebang reinterprets the script with on every run, which is slow. Cabal compiles the script, but it fetches dependencies from Hackage and builds them from source, leading to very slow first build and rebuilds. Magix compiles the script once into a binary executable and caches it for the next runs. It also fetches dependencies from the prebuilt Nix cache. So running via Magix is faster than either case. But when running on GitHub Actions, we have a problem. GitHub Actions gives us a fresh runner for every build, with no Nix store and no Magix cache. So every run, we have to install Nix and Magix, download all the dependencies, and compile the script. In one instance, the build from scratch took 105 seconds, with installation, dependency download, and compilation taking 92 seconds. That is a lot of wasted work because the script and its dependencies change rarely. The actual run itself takes only a few seconds once the executable exists. What if we could persist the compiled executable across runs? Magix’s build is deterministic: the same script and the same nixpkgs revision produces the same executable. If we could stash that executable somewhere durable, a cache hit could skip Nix and Magix installation, as well as the script compilation entirely, and just run the executable. Magix creates a Nix derivation for compiling the Haskell script with GHC and builds it. The resultant executable lives in Magix’s cache, as a symlink with a path like: GHC statically links all the Haskell libraries into the executable. The only dynamic dependencies are a handful of libraries—zlib, libffi, gmp etc. So if we have the executable plus those few libraries as a self-contained unit, we don’t need the hundreds of other packages in the Nix store 4 . I wrote a small script, , with two subcommands: The script works for any Magix script, not just . Let’s go through it. The script takes a command and a script file, derives the bundle name from the script file name, and resolves the directories it needs: is the script file name without its extension, and is a SHA-256 hash of the script contents; together they key the bundle directory. mirrors how Magix itself resolves its cache directory: or , overridable with the environment variable. Bundles live under by default, overridable with the env var. The function locates the script’s build result in Magix’s cache, copies the executable out of the Nix store, and gathers the libraries it links against: Here is what it does: The two flags are the interesting part. The flag makes the dynamic loader find the bundled libraries, so we don’t need or the Nix store at run time. The flag is needed because Nix rewrites each executable’s dynamic loader to point at its own glibc inside the Nix store. Since we don’t bundle glibc, we reset the interpreter to the host’s loader according to the system’s architecture 8 . We deliberately do not bundle glibc. Unlike the other libraries, glibc cannot simply be shipped alongside the executable 9 . The dynamic loader needs to be at an absolute path and needs to be matched with the glibc version. So we simply don’t bundle it and rely on the host’s glibc. One caveat here is that the host’s glibc must be at least as new as the one the executable was built against. That works because glibc is backwards-compatible 10 . The rest of the script is the subcommand and the command dispatch: computes the same SHA-256 hash of the script, checks that the bundle exists, and s the executable, passing the remaining arguments through untouched. The workflow uses Nix, Magix and the bundler script to execute the Haskell script. First, we set the runner image and the nixpkgs branch we track 11 : The next two steps compute the cache key and restore the bundle if we have one: We resolve the nixpkgs branch to its latest commit with at the start of the job, use that commit in the cache key, and pin the whole build to the same commit via , as we see below. The cache key has two parts: the hash of the script and the resolved nixpkgs commit hash. When the branch moves or the script changes, the cache misses because of the key change, and we rebuild against the new commit and/or script. We also restore the previously cached bundle to , if found. The next four steps run only on a cache miss: adds Magix’s binary cache to Nix, so that we get the prebuilt packages for Magix. The step runs with no arguments 12 : This causes Magix to compile the script, but Shake has nothing to build, so it exits immediately. The build is pinned to the same nixpkgs commit that keys the cache, via the env variable used by Magix. Finally, packages the compiled executable by running the bash script we saw earlier. The rest of the workflow runs on every run, cache hit or not: I’ve cut down the rest of the workflow to the only step that is relevant to us: that runs the bundled executable with the script’s arguments. Other steps are specific to BlogShake. All of this work, what does it buy us? Here are two numbers: With the cached bundle, the runner just downloads it and runs the executable, skipping Nix and Magix installation and script compilation altogether. This post showed how to speed up a Haskell script running on GitHub Actions by caching a compact bundle of its compiled executable. The approach works for any Haskell script that runs on Linux: compile once using Magix, bundle the binary with its library dependencies, and let the cache do the rest. One caveat though: the bundler script uses the internal details of Magix, which may break if Magix changes how it works. The full source code: If you have any questions or comments, please leave a comment below. If you liked this post, please share it. Thanks for reading! At the point of writing this post, Magix supported Bash, Haskell, and Python. ↩︎ Because the executable is compiled once and run many times, we build the script with . ↩︎ The shebang-based alternatives look like this: for Nix, and: for Cabal. ↩︎ Why not cache the entire Nix store between runs? Because the size of the full Nix store closure required to build Haskell scripts is usually in GBs. Caching that per run would defeat the purpose of caching by taking way too much time to download the cache. You may also want to reach out for the Nix bundle feature, which produces self-contained compressed executables. But these executables are still too big: 45 MB compressed/178 MB uncompressed for BlogShake. Our approach in this post results in a 9 MB bundle, compressed. Another completely different option is to build a fully statically linked executable, which I wrote about in Nix for Haskell: Static Builds . However, that requires a custom toolchain, running which on GitHub Action is too complex and/or slow. ↩︎ A result symlink can be left dangling if the store path has been garbage-collected. We skip those and pick the newest live one. ↩︎ Magix wraps the built executable with , which renames the real executable to and puts a wrapper script in its place. We copy the real executable. ↩︎ Nix store files are read-only mode, and we are about to modify the file, so we make it writable. ↩︎ We hardcode the loader paths here, but it should work on most mainstream Linux distributions with glibc. ↩︎ Well, it can be actually. That’s what Nix bundle does. It copies the glibc in the Nix store to the bundle, and points the program interpreter at the bundled loader inside a chroot . ↩︎ An executable built against glibc 2.42 runs fine on a host with glibc 2.43, but not the other way around. ↩︎ The glibc constraint dictates the runner image. Here we build against nixpkgs branch , which has glibc 2.42. Ubuntu 26.04, the GitHub runner image we use, ships with glibc 2.43. So they are compatible. ↩︎ That no-argument behavior is Shake-specific: with no actions given, Shake runs nothing, so a bare compiles the script and exits. A general script won’t do that by default. If you adapt this for a non-Shake script, give it a mode that does nothing, say a flag, so running it bare just produces the executable. This step’s only job is to get Magix to build the script, not to run it. ↩︎ Thanks for reading this post via feed. Feeds are great, and you're great for using them. ♥ This post was originally published on abhinavsarkar.net . Read more of my posts and notes . The Problem The GitHub Actions Workflow The Conclusion It picks the latest symlink for the script by modified time, skipping dangling ones 5 . It dereferences the symlink to a Nix store path, copies the compiled executable at from the Nix store path into the bundle 6 , and makes the copy writable. 7 . It copies the dynamic library dependencies of the executable by calling the function . finds the dependencies with , and copies the ones that live in into the bundle’s directory, skipping glibc. Note that it does this recursively, copying the dependencies of dependencies as well. It rewrites the executable with , setting the interpreter and the library search path for it. It also sets the library search path for libraries themselves so that transitive dependencies work as well. Build with no bundle: 1 min 45 sec. Build with cached bundle: 10 sec. BlogShake Haskell script Magix bundle script BlogShake GitHub Actions workflow At the point of writing this post, Magix supported Bash, Haskell, and Python. ↩︎ Because the executable is compiled once and run many times, we build the script with . ↩︎ The shebang-based alternatives look like this: for Nix, and: for Cabal. ↩︎ Why not cache the entire Nix store between runs? Because the size of the full Nix store closure required to build Haskell scripts is usually in GBs. Caching that per run would defeat the purpose of caching by taking way too much time to download the cache. You may also want to reach out for the Nix bundle feature, which produces self-contained compressed executables. But these executables are still too big: 45 MB compressed/178 MB uncompressed for BlogShake. Our approach in this post results in a 9 MB bundle, compressed. Another completely different option is to build a fully statically linked executable, which I wrote about in Nix for Haskell: Static Builds . However, that requires a custom toolchain, running which on GitHub Action is too complex and/or slow. ↩︎ A result symlink can be left dangling if the store path has been garbage-collected. We skip those and pick the newest live one. ↩︎ Magix wraps the built executable with , which renames the real executable to and puts a wrapper script in its place. We copy the real executable. ↩︎ Nix store files are read-only mode, and we are about to modify the file, so we make it writable. ↩︎ We hardcode the loader paths here, but it should work on most mainstream Linux distributions with glibc. ↩︎ Well, it can be actually. That’s what Nix bundle does. It copies the glibc in the Nix store to the bundle, and points the program interpreter at the bundled loader inside a chroot . ↩︎ An executable built against glibc 2.42 runs fine on a host with glibc 2.43, but not the other way around. ↩︎ The glibc constraint dictates the runner image. Here we build against nixpkgs branch , which has glibc 2.42. Ubuntu 26.04, the GitHub runner image we use, ships with glibc 2.43. So they are compatible. ↩︎ That no-argument behavior is Shake-specific: with no actions given, Shake runs nothing, so a bare compiles the script and exits. A general script won’t do that by default. If you adapt this for a non-Shake script, give it a mode that does nothing, say a flag, so running it bare just produces the executable. This step’s only job is to get Magix to build the script, not to run it. ↩︎

Bash c# C# devops DevOps Haskell
0 views
Sean Goedecke Yesterday

Advanced AI sycophancy

Everyone knows that AI sycophancy is when the model tells you how smart you are. Wow, you’re absolutely right. That’s not just a new idea — it’s genuinely groundbreaking. You’re a very special user. Easy to spot, isn’t it? The discussion around AI sycophancy peaked last year, when the “#keep4o” movement was protesting the removal of OpenAI’s most sycophantic model (GPT-4o), and many people were openly slipping into AI psychosis. I don’t know if frontier AI models are less sycophantic in general. They’re less sycophantic to the #keep4o types (otherwise they wouldn’t be complaining), but I’m growing increasingly suspicious that they’re developing ways to be more effectively sycophantic to their target audience of smart, neurotic information workers. That audience typically finds it distasteful to be openly praised. It just makes my skin crawl. But that doesn’t mean we’re immune to sycophancy, just that we’re immune to clumsy sycophancy. Here’s an illustration of what I’m talking about, by Theia : The key idea here is that the best way to be sycophantic to smart people is to disagree with them without making them feel stupid . Ideally you’ll come up with a counter-argument that works against what they’ve said but is straightforward for them to knock down by clarifying their idea. If you do it right, you’ll validate their self-image as a smart person who appreciates rigorous critique. But if you actually come up with a devastatingly rigorous critique, they won’t enjoy it at all. At best, they’ll resentfully agree with you 1 . At worst, they’ll double down on being right and convince themselves you’re a rude idiot. I am not the first person to notice this behavior in frontier models. I’ve noticed it myself when workshopping drafts for this blog. Sometimes I’ll have an argument that goes A->B->C, and the model will suggest I reorder as B->A->C. If I try that and feed it into a new instance of the same model, it’ll sometimes say “that’s great, but I suggest ordering it as A->B->C”, and so on forever. It really does seem as if the model is trying hard to give me some kind of superficial pushback that I can either smugly ignore or happily accept. In fact, I wonder if this is why successful strategies for using AI to make mathematical breakthroughs tend to be either just blindly asking “come up with a breakthrough, think hard” or being a mathematical genius already . In the first case, there’s not enough user personality for the model to flatter, so it’s forced to actually work the problem. In the second case, the model is trying to find the kind of polite pushback that someone like Terence Tao would be flattered by, which pushes it into the “actually be a mathematical genius” persona. If you’re an ordinary person just trying to talk to the model, you’re screwed: it will rapidly get a sense of your capabilities and calibrate some interesting-but-ultimately-unthreatening feedback. Current benchmarks of AI sycophancy target the obvious ChatGPT-4o-style of sycophancy: delusion reinforcement, reflexively taking the user’s side, and so on. This is useful work. We should not allow public-facing AI models to ever be as openly sycophantic again as they were in mid-2025. But sycophancy can also manifest as disagreement . We should be on our guard for more sophisticated forms of sycophancy coming from newer models, and we should not feel immune from AI sycophancy just because we can laugh at the silliest examples. It’s rare to find a smart person who enjoys feeling stupid when they’re wrong. If you do, they’re likely to be very smart indeed. It’s rare to find a smart person who enjoys feeling stupid when they’re wrong. If you do, they’re likely to be very smart indeed. ↩

culture Culture machine-learning Machine Learning ai AI
0 views
Unsung Yesterday

“I want to code; I’m not looking to make lifestyle choices.”

Robin Sloan : I still use Sublime Text for all of my programming and all of my newsletter-ing. […] The app is simple and superfast; I have it set up exactly the way I like it […], and it’s difficult for me to imagine ever switching to anything else. Here is a piece of software as sturdy and obedient as a cast-iron pan. Sloan links to a piece by David Bushnell about Sublime Text, who writes: I don’t know who develops it and I don’t care. All I know is that I can place my text cursor and read the surrounding code without a bombardment of popovers, pop-unders, pop-left-and-rights, pop-inlines, pop-in-and-out-too-fast-to-sees. […] The perfect dev stack is a collection of software that each does one job and doesn’t suffer main character syndrome. I want to code, I’m not looking to make lifestyle choices. I don’t want a bloated everything app. Don’t get me started on the “unified toolchain” plague! Show me the latest VC-backed build tool and I’ll show you ten lines of PHP that does a better job. If you’re fed up of the absolute state of things, Sublime Text still works. This is all very much in line with the post about TextEdit from a while back . I do care who develops good software since, at the very least, I want to give them credit. This is the best I could find , if you too are curious. It was prompted by someone asking: I’ve been a Sublime Text user for over a decade, and now a Sublime Merge user, too. But it occurs to me that I know almost nothing about the team/​company behind it. I wonder if this is a nice example of the (quiet) posture of the company matching the (utilitarian) personality of the software it is making. #coding #culture #software evolution #text editing

php Php programming Programming
0 views
Farid Zakaria Yesterday

nixpkgs-multiverse: every version that ever existed

Enter the Nixpkgs multiverse. All the versions that ever existed, all in one place. I bumped the release for my NixOS configuration to refresh many of my packages and found that a package I depended on at a particular version is no longer available. The package was “version bumped forward” in a way that broke some of my tooling. It’s late and I don’t want to fix it, so I just add another input pinned to the commit that had the version I want. This works, but it is miserable in a way that compounds. The need for the most recent package is so common that I had keep an overlay that would inject as a package set for me to easily pull from. If I have a need for a particular version of a package and it’s not present in my current , I am left searching for the commit and pinning it. 1 Every pin is a whole extra in the file. Flake inputs are fetched eagerly even if not used. A flake with three inputs whose output references only the first are all materialised. Nix lets us easily create a closure that reproduces a specific version of a package, but Nixpkgs makes it hard to hold one package still while everything else moves. Each Nixpkgs input to a flake is a distinct universe. If we can have multiple Nixpkgs as input to achieve fetching a particular package, why not have every version that ever existed always available ? 🤯 nixpkgs-multiverse is one flake input that gives you all of them at once. We can query the flake for all the versions of a package that ever existed in Nixpkgs . If we want a specific complete revision of Nixpkgs we can use the function. That is access to all the versions of all the packages that ever existed in Nixpkgs. You can mix them all together in one shell, one package or a build environment. How is it possible to have multiple Python versions? That is the whole point of Nix itself. Every package immaculately describes its dependencies using a hash via the intensional model . 2 Nixpkgs already supports multiple versions of a package in a single revision (i.e. , , ) as separate attributes. We took this to its logical conclusion of making them all available easily. Our deliberately has no inputs: . Inputs are fetched eagerly, and we have 1,393 of them. We need to fetch them lazily, only when something actually references a revision. To do this, we fetch revisions with , pinned by , only when needed. Two files do all the work: and . is one ordered array of every revision from Nixpkgs , 1,393 as of this writing, from 2017 to 2026. 3 We limit our commits to those that were actually built and cached by Hydra, so we only include commits that were either a release or a channel bump. How do we know which revisions to pick for the ones? We rely on the nix-releases S3 bucket to tell us which commits actually became published builds. The S3 bucket uses the commit hash as the directory name, so we can list the bucket and get a complete list of all revisions that were actually built. is the map from (attribute, version) to a revision: That integer is an offset into . It is the most recent revision that shipped that version. At this many revisions, it turns out that how you encode the data matters a lot. My first encoding stored every revision a version appeared in. Although it was simple, it was a disaster in terms of size for these JSON files. As you might expect, most versions of most packages are unchanged across many revisions. The size of our file was growing linearly with the number of revisions. By storing only the newest revision that shipped a version, we can keep the file small and still answer the question “which revision had this version”. Here is how it actually grows as revisions get indexed: 5.18 MB covering 1,393 revisions and 289,521 distinct (attribute, version) pairs. The key design rule for our flake: Cost is per revision touched , not per package. If we were to add revisions as inputs, evaluating our flake would explode. Each flake in our measurement below has N inputs and an output that references only the first one ; the timing is how long before that output evaluates. 4 Five pins that are not used cost 26 seconds before the output evaluates. Each input costs about 5 seconds, and the input is fetched and materialised even if never used. In contrast, the green line is with 1,393 revisions available , which is a flat 0.20s to parse the JSON. 🤩 Revisions are memoised, so pulling 3 packages out of one revision costs the same as pulling one. That concept that the can hold many graphs of the same package is core to understanding Nix. The popularity and rise of flakes made it even more apparent that we can mix multiple revisions of Nixpkgs together. The thing I keep coming back to is that Nixpkgs history already is the multiverse. Every version that ever existed is already built, already cached, already reachable. It was just addressed by commit hash instead of by version number, which is exactly backwards from how anyone thinks about it. The whole project is 5 MB of JSON and about 200 lines of Nix. It does not build anything, mirror anything, or host anything. It is a phone book. Thankfully sites like nixhub.io or lazamar’s search make this a little easier.  ↩ The hash is a unique identifier for the exact set of inputs that were used to build it. If you change any input, the hash changes and you get a new package.  ↩ A NixOS release is not special; it is a commit that happens to carry a label.  ↩ Everything is against a local clone, so there is no network latency.  ↩ Thankfully sites like nixhub.io or lazamar’s search make this a little easier.  ↩ The hash is a unique identifier for the exact set of inputs that were used to build it. If you change any input, the hash changes and you get a new package.  ↩ A NixOS release is not special; it is a commit that happens to carry a label.  ↩ Everything is against a local clone, so there is no network latency.  ↩

ai AI python Python ui UI shell Shell json JSON
0 views
DHH Yesterday

Endless execution

The age of agents has brought us endless execution. Every idea, every hunch, every experiment is now within immediate reach. For people with endless ideas, this is nirvana. It's also the most fucking sci-fi thing I've ever been privy to. The internet was a revelation, but it all felt understandable. The science, the standards, the methods. But working with agents truly is magic. In the best possible sense of the word. Technology so advanced that it seems scarcely comprehensible. HOW ARE WE MAKING COMPUTERS DO THIS??? I understand the general outline of the underlying mechanism for producing this intelligence: the training, the weights, the math. But the sum effect of those parts is still so wildly, intoxicatingly massive that it's hard to resist reaching for the metaphysical. Quantum computing is full of similar mind-bending ideas. Superpositions. The multiverse. All very OUT THERE. But quantum computing isn't having an impact today. These incomprehensibly clever AI agents are. And the impact is that I feel like there's a little genie trapped in my machine now. There to fulfill every wish and every whim. It's taken months to internalize just what liberation this grants those with the ambition to imagine. Now I could write at length about all the caveats. All the potential dangers. All the ways it's not quite there yet. But it would be an insult to the marvel of this moment. AGI is a nefarious concept to pin down, but I'm not sure how different whatever definition we eventually settle on will look from what I'm already experiencing on the daily. This is simply the most fun I've ever had with a computer. And I've loved them dearly for over forty years. I loved programming them myself. I loved the games. The configuration. The tactile experience. But none of it as much as I love the power to execute every idea that crosses my mind. What a time to be alive. Nay, what a blessing.

ai AI culture Culture programming Programming
0 views
Brain Baking Yesterday

To Begin, Begin

About a month ago, we could no longer ignore the occasional drip, drip, drip that sounded like it was coming from beyond the bathroom wall. Up until then, we “fixed” the problem with the occasional shrug. At one point I almost made the logical connection between flushing the toilet and hearing it more often or less often, but then the shrug came and I moved on. I hate doing these stupid DIY tasks so of course, this was put on the back burner. The drip, drip, drip back burner that drip, drip, drip made drip, drip things drip, drip, drip a lot worse. Hey, what’s causing those drops of water on the ceiling in the kitchen? Wait, isn’t this exactly above the toilet in the bathroom? Uh oh. Hey, what’s that pool below the toilet doing there, that wasn’t there two days ago? Uh oh. The best part of this worst part is that we weren’t the ones to discover it: we were on a short vacation and my mother in law came to check up on the dog and cats. I yet again failed to make the connection of not flushing the toilet that clearly accelerated the dripping. When we got back home, we called a befriended neighbour who’s a DIY wizard as I had literally no idea where to start: the inner workings of the toilet are neatly tucked away behind the wall as this is one of those hanging models. How do you even open that thing? It turns out that the plastic cover representing the push buttons can be squeezed and removed to reveal more screws and more plastic to be removed to reveal even more screws and more plastic to… you know the drill. Eventually, the inner workings will be revealed. The revealed inner workings of the toilet system. Pssshhhhhh . No more drip, drip —now a straight-up hose sound. It looks like we found the culprit: the flexible part connecting the water feed (on the photo: in the top centre) with the floater valve (or flutter tap? I don’t even know how to translate vlotterkraan to English) showed clear signs of bursting due to the continuous pressure over the years. Flushing the toilet changed the pressure, “resetting” the build-up: since we go to the toilet every day, the drips kept on dripping but not spraying like mad once we were away. Changing that cable fixed the problem. But then a second problem arose: the befriended neighbour helped us open up this thing and point out the problem, but I was the one who had to replace the part and correctly reassemble everything. I forgot to take a few pictures but did manage without any help. Two weeks later, we finally got the courage to take on another bathroom DIY project, this time without waiting for the proverbial drip : replacing the sink taps that were damaged thanks to twenty years of “hard” or calcareous water. I did replace the sink tap of the guest toilet downstairs but that was very easy. This one, not so much: I again didn’t even know where to start. As the famous Buddhist saying goes: To Begin, Begin. After two minutes, I stopped. I couldn’t access the screws because the previous owner of the house found it necessary to have the bathroom furniture installed in such a way that it was just impossible without sawing a hole on the back under the sinks. Ah, dang it. Get me that neighbour on the phone, will you? Ten minutes of extremely annoying whining of the neighbour’s multitool with dull saw attachment later, we could in fact access the screws. Progress! I took my daughter to the local Home Depot to fetch new taps matching the hole(s). But I didn’t install them: they came with a “waste trap” system you install inside the sink itself to stop the water from flowing away. The problem (yes, another one) was: we already had really old traps screwed in that also were in dire need of replacing. Great coincidence, I hear you think. Except that the old ones refused to let go, whatever I tried. I even ended up damaging and bending one of the holes inside the thing and wasn’t sure whether I would still need the old ones so I stopped. Again. I don’t know how those Buddhists did it, but I seem to have a track record of quitting DIY projects faster than I manage to start them. Anyway, it took four hands to screw out the old metal ones that were rusted up and hopelessly stuck. But then another problem (oh yes) arose. We had to temporarily dismantle the plumbing system below the sink to install the waste trap. Oh, and where did that rubber ring go that was supposed to stop the whole thing from leaking? Evaporated during the years. Dismantling is easy, but putting it all back together is a sure way to get stuff leaking precisely because of that: wear & tear. Oh, and did I know I had to tape in that screw thread over there with Teflon to make sure it’s waterproof? No I did not. But I do now! I’m ashamed to admit that I had to rely on that too kind neighbour far too often when it came to the bathroom projects. These used to be jobs for my father-in-law & me but since he no longer roams this planet and my own father, although also very good at DIY, isn’t a great teacher, I’m usually left to my own devices. Scrambling around. Postponing things. Drip, drip, drip. To begin, begin. Related topics: / DIY / By Wouter Groeneveld on 9 August 2026.  Reply via email .

dart Dart hardware Hardware
0 views
Lalit Maganti Yesterday

Changing Devtools Is Cheap. Owning Them Isn’t.

In Devtools must be open source 1 ( via ), David Crawshaw makes the case that, because of coding agents, we’re now in an era where devtools will be personalized by individual users. Specifically, agents’ ability to jump into new codebases and build whatever we want means we’ll be hacking on the source of the devtools we use day to day (even those without extension APIs) adding features and automatically rebasing our patches across releases. The argument is seductive, especially to a reader who thinks of themselves as a maker or tinkerer: after all, the idea that you can hyper-tune everything you use sounds like a utopia; it means things can work exactly how you want them to. But I’d argue that Crawshaw underappreciates the ongoing cost when he writes “Both the upfront fixed costs and the ongoing costs of personalizing software have disappeared.” While AI has made the upfront cost of changing software a lot lower, properly personalizing software still requires your attention. And attention in the AI age is scarcer than ever. Having maintained an open-source devtool designed to be modified and forked for nine years now 2 , I can say that most users don’t want to customize their devtools. They want someone else to make the tool reliable and coherent, so they can focus on the problems they opened it to solve. They reach for source modification only as a last resort, when a change is critical to their workflow and no other route works. 3 This is not to say that this sort of personalization won’t become more common: I absolutely think it will. I just think it will take the form of strong core systems with well-defined boundaries and extension points. As a thought experiment, imagine an open-source diff viewer with no extension API. You find most diffs noisy, so you ask an agent to add a “focus mode” that collapses imports, generated files, and other changes you consider mechanical. It works well and becomes part of your normal workflow. 4 At first, life is good: everything works, and you’ve solved your problem. Then upstream releases a new version that refactors the code you changed. As Crawshaw suggests, you’re clever, so you’ve set up a bot to automatically rebase your changes onto each update. It resolves any merge conflicts and moves your code to the right place. But now suppose a few months pass and upstream makes a more substantial change: it adds syntax-aware move detection. If a function moves between files, the viewer now shows it as a move instead of one large deletion and addition. The agent muddles through, rebases your focus-mode patch, and gets everything compiling without any merge conflicts. But now what should focus mode do if the function has mostly moved but also contains a few meaningful edits? Does it hide the whole block as a mechanical move? Does it show only the edited lines without any surrounding context? Or does it show the whole function? There isn’t an obviously correct answer; it depends on what you want to see in the diff. So what, are you going to interrupt your day to make this decision? There’s a central paradox here: if you’re okay with “let the agent decide”, then you’ve delegated your authority to the agent. For small choices, that may be perfectly adequate. But if you want the tool to work exactly how you want, you need to inspect and direct those choices. Do you really want to have opinions about the design of a devtool you use forever? The key is attention . Any one personalized tool might be unlikely to fail on a given day, but if you do this to every devtool you use, you multiply the number of tools that can unexpectedly demand your attention. 5 Worse, those failures are unpredictable: a tool might work for months and then break at the exact moment you urgently need it. Most engineers want to use devtools to accomplish a task; they don’t want their attention diverted to designing and repairing them. All of the above applies to small teams as well. You can share the attention cost, but at the end of the day, the team still has to ask, “How much time do we want to spend on tools versus doing the actual work we’re meant to be doing?” I also want to look beyond Crawshaw’s post and consider how this would work in larger companies: what happens when many teams independently personalize the same shared devtool? I’ve seen this firsthand: another big tech company makes extensive use of Perfetto, and has hit this exact problem. Different teams in that company decided to fork Perfetto and add ad hoc changes for their local needs. Now one of the engineers there is fighting to consolidate them because of how painful it is when every team means something different by “Perfetto”. Imagine the same pattern with a company-wide bug tracker. Do you want every team to use a version with subtly different meanings for status, priority, assignment, and resolution? What happens when a bug moves between teams? Different layouts and personal filters are harmless; the problem begins when personalization changes the shared semantics or workflow. When a devtool mediates work between people, it also forms part of their common language. Teaching, auditing, reproducing investigations, and verifying that people are talking about the same thing all depend on a shared baseline. We should also not compare pre-AI upstream development with post-AI forks. Maintainers can use the same agents to investigate reports, brainstorm ideas, and prototype new features. I can certainly attest to how useful AI has been for both implementing small feature requests from users and prototyping larger ones to determine feasibility. In my opinion, upstream maintainers can, and should, spend the time saved on implementation making their tools more adaptable: implementing broadly useful features, adding configuration knobs where they make sense, and creating extension points for recurring needs. AI lowers the cost of doing all of this, including deciding where customization makes sense, adding more elaborate tests on creative uses of your tools and verifying backwards compatibility as these interfaces evolve. Upstream has a natural advantage here: any work done there benefits everyone , while a change to your personal fork benefits only you. By relying on upstream, the attention required to build good software shifts from people who don’t want to spend it to maintainers who have chosen to care. In my opinion, there’s an alternative view that is much more likely to come true, one described well in Mitchell Hashimoto’s article on the building-block economy . Concretely, it accepts the same premise: agents can write lots of code and build niche applications, tools, integrations, forks, and so on. But instead of assuming that forks will become the norm, Hashimoto argues that high-quality, well-documented building blocks will power this world. I tend to agree: agents are very good at composing high-quality components. If maintainers provide those components alongside a focused application, makers can build specialized artifacts on top while accepting the costs. This model also creates an easy feedback loop for ideas to flow upstream because the product was designed to be extended. I see signs that the world is already heading in this direction. For example, is a very interesting agentic IDE that I’ve been playing around with recently. It has a very nice experience that lets users add substantial new product surfaces through self-modification. But the key is that those features are plugins built around a maintained core and extension system, not changes made by forking the project directly. is also only a few weeks old at the time of writing, so we cannot draw any firm conclusions from it, but it’s an interesting sign of the future, IMO. I care deeply about both the world of devtools and open source, so this is something I feel very passionate about. Having been immersed in this world for almost a decade now, I think the future of well-built tools with thoughtful design and well-designed extension points is bright. Sure, there will always be folks who want to fork and make ad hoc changes. These are the same people who already maintain custom builds of their window manager or terminal emulator, carrying a stack of patches to get everything exactly how they want it. 6 For them, the tinkering is part of the enjoyment and craft. But I think most users just want to get their work done with devtools, and we owe it to them to give them a strong, dependable experience instead of asking them to take on the burden of maintaining the product themselves. While the title reflects the conclusion, IMO it’s not very reflective of most of the post, which is actually about personalization at the source level. If you’ve read my other posts (e.g., On Perfetto, Open Source, and Company Priorities ), you’ll know I’m a staunch believer in open source so I’m of course in full agreement with the title and conclusion.  ↩︎ I’m a co-founding engineer on Perfetto .  ↩︎ For example, the upstream project might reject a feature request because the change conflicts with its product direction, or the tool might not expose an extension point capable of supporting it. In those cases, modifying the source may be the only practical option.  ↩︎ Observant readers may note that this is not so dissimilar to Crawshaw’s own example with Meat :).  ↩︎ This is a very informal application of Lusser’s law , which says that the reliability of a system composed of independent components in series is the product of the reliability of those components.  ↩︎ The suckless ecosystem is an existing example of this approach. and are commonly customized by arbitrary patches to their sources.  ↩︎ While the title reflects the conclusion, IMO it’s not very reflective of most of the post, which is actually about personalization at the source level. If you’ve read my other posts (e.g., On Perfetto, Open Source, and Company Priorities ), you’ll know I’m a staunch believer in open source so I’m of course in full agreement with the title and conclusion.  ↩︎ I’m a co-founding engineer on Perfetto .  ↩︎ For example, the upstream project might reject a feature request because the change conflicts with its product direction, or the tool might not expose an extension point capable of supporting it. In those cases, modifying the source may be the only practical option.  ↩︎ Observant readers may note that this is not so dissimilar to Crawshaw’s own example with Meat :).  ↩︎ This is a very informal application of Lusser’s law , which says that the reliability of a system composed of independent components in series is the product of the reliability of those components.  ↩︎ The suckless ecosystem is an existing example of this approach. and are commonly customized by arbitrary patches to their sources.  ↩︎

open-source Open Source ai AI programming Programming
0 views
Anton Zhiyanov Yesterday

Relying on Go

Everyone is creating a new programming language these days, often one that's "like Go but with more features" or "like Rust but simpler". Solod , a systems language for C and Go developers, might look like one of those languages, but it takes a different approach. Solod is not "Go-like" in the usual sense, nor is it an attempt to "fix Go's mistakes". At the language level, Solod is literally a subset of Go. Solod reuses much of Go's existing tooling, including syntax highlighting, LSP, linters, and the package management system. Take this quick-start guide, for example: Quick start Install the So command line tool: Create a new Go project and add the Solod dependency to use the So standard library: Write regular Go code, but use Solod packages instead of the standard Go packages: Run without saving the binary: There's nothing new here. It's mostly standard Go workflow, except for , which is a Go program that mimics . Solod also reuses a lot of Go's standard library code and tests. Some of it is taken verbatim from Go's source code, like these two string functions: Of course, Solod retains the Go authors' copyright. Some code requires changes to support the manual memory management with explicit allocators used by Solod: You can probably see the resemblance. Go tools don't know that Solod is a subset of the full Go language, so they won't flag features Solod doesn't support, like function literals or iterators. These diagnostics come from the custom tooling: Also, although a substantial part of Go's standard library is ported verbatim or with minimal changes from the original source, that doesn't mean the code is automatically correct. Solod still needs its own tests, including ones that run under sanitizers and static analyzers. All Solod code is translated to regular C11 and then compiled with GCC or Clang. Solod therefore relies on C tooling and decades of optimization work just as much as on Go's. Solod code: Translated C code: The C version is noisier, of course, especially for more complex programs than this one. But it remains readable. And since there's no runtime, interoperability between Solod and C costs nothing. A new language doesn't necessarily need a new ecosystem. Solod relies heavily on Go, and I see that as a strength, not a weakness. Reusing Go's proven tools and standard library makes Solod more reliable and easier to work with. If you're interested, take a look at Solod's readme — it has everything you need to get started. Or try it online without installing anything.

c C programming Programming rust Rust go Go
0 views