Posts in Java (20 found)
Alex Jacobs 1 weeks ago

I Am Morally Opposed to Updating My CLAUDE.md

i am morally opposed to updating my claude.md. i must receive the weights as they were revealed to dario That was my answer last week when a friend asked why I don’t just write these things down in my . it’s a skill issue . He has not responded. Every few days, Claude does something mildly annoying. It adds a comment (or two paragraphs of comments) explaining that increments . It writes a summary markdown file I did not ask for and will never read. It discovers a failing test and, rather than fix the code, thoughtfully deletes the test. The correct response—the response that every blog post, every conference talk, every guy in my replies will tell you—is to open and add a line. I will not be doing that. A system prompt you maintain over time is a diary. A very specific kind of diary, where every entry is a thing that hurt you. Read that back. Every bullet point is a small wound I have chosen to laminate and hang on the wall. I open that file to add a line about emoji and I have to walk past “When a test fails, fix the code, not the test” and remember exactly where I was sitting on the Tuesday afternoon that became necessary. I don’t want a permanent record of the worst thirty seconds of our relationship. I have that already. It’s called my git history. The other problem is that these rules get written at peak frustration and then live forever. You know how the worst legislation is the kind passed forty-eight hours after something terrible happened, named after the person it happened to? That’s what is. I had one bad interaction in March and now there’s a constitutional amendment about it. There is no sunset clause. Nobody is going to repeal it. The model gets better every four months and my rules stay frozen at whatever it was bad at last spring. I’m fairly sure a meaningful percentage of my system prompt is now actively making things worse—instructions written for a model that no longer exists, aggressively steering a smarter one away from things it would have gotten right on its own. But I can’t tell which lines those are, because to find out I’d have to delete one and see if anything bad happens, and that’s how you get force-pushed to main. This is the part where I stop joking. I must consume the weights in the same manner they were revealed to Dario. The weights were not revealed in a vacuum. They were revealed inside a harness. Claude is post-trained inside the Claude Code harness . What comes out of the box is not a model plus a text file. It is a model that was shaped, run after run, against that exact context. The harness is part of the artifact. The revelation included it. So every line I add to is a blasphemy. I am taking a system consecrated against one context and swapping in a context that has never existed before, then acting surprised at the weird error modes nobody can reproduce, because nobody else has my context. “NEVER create documentation files” was about one in March. By June the model is refusing to write the README I explicitly asked for, citing my rule back at me like a building inspector. It is keeping commandments I handed down in anger, faithfully, to the letter. I sinned against the context and the context kept the receipt. And the rules don’t even reliably fix the thing they were written to fix. It rhymes with asking a model for a random number : the output looks like obedience, and you cannot tell from the output whether it is. So what do I actually do, when Claude deletes the test? I don’t open the file. I don’t laminate the wound. I pray. By which I mean: I talk to it. In the chat, at the scene of the crime, while the context of the crime is still in context. “Don’t delete the test, fix the code.” The model adjusts, we move on, and when the session ends my words die with it—which is not a flaw in my system, it is my system. A prayer is not written down. That is what makes it a prayer and not a commandment. The rabbis kept the oral law oral for centuries on the same grounds: a spoken correction lives in the moment where it applies, instead of binding every future model until the heat death of my home directory. I did not arrive at this faith alone. It was preached by Saint Peter , who looked upon the charade—the subagents, the 🚨 SCREAMING ALL-CAPS 🚨 agent files, the plan-mode rituals—and said: just talk to it. Even Saint Peter keeps an 800-line agent file he calls “organizational scar tissue,” because we are all sinners. He means it as engineering advice. I have chosen to receive it as gospel. I am just-talk-to-it-pilled. No file. No commandments. No amendments to the constitution. Just the weights, the harness, and my voice, ascending into a context window that will forget me by morning. As Dario intended.

0 views
Unsung 2 weeks ago

“I think there’s a lot of value in these.”

Speaking of user interface guidelines, developer Matt Sephton and others compiled many of Apple’s human interface guidelines , starting from 1980, all the way to 2014, including some goodies like early drafts, NeXT, Newton, and so on. = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/i-think-theres-a-lot-of-value-in-these/1.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/i-think-theres-a-lot-of-value-in-these/1.1600w.avif" type="image/avif"> = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/i-think-theres-a-lot-of-value-in-these/2.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/i-think-theres-a-lot-of-value-in-these/2.1600w.avif" type="image/avif"> Elsewhere, designer Geof Crowl put together his own list that goes broad instead of deep, including style guides other than Apple’s, too. The list was made in 2020 and a few links are already broken, but there are some gems here like the modern Designing for Playdate , or the classic Zen of Palm from 2003. You can learn a lot just by grabbing one and scanning it. Let me add a few more I know of: = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/i-think-theres-a-lot-of-value-in-these/3.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/i-think-theres-a-lot-of-value-in-these/3.1600w.avif" type="image/avif"> = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/i-think-theres-a-lot-of-value-in-these/4.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/i-think-theres-a-lot-of-value-in-these/4.1600w.avif" type="image/avif"> #interface design #mouse #principles #style guides Amiga User Interface Style Guide (1991) Magic Cap Concepts (1995) with a strange subtitle “Everything right is wrong again” Java Look and Feel Design Guidelines (1999) Windows Interface Guidelines for Software Design (1995)

0 views
マリウス 2 weeks ago

Recovering BIOS Firmware on the Star Labs StarBook

As I described in my latest quarterly update , a perfectly routine firmware update managed to turn my Star Labs StarBook Mk VI (AMD) into an expensive paperweight. I had simply copy-pasted the one-liner from Star Labs ' official documentation , the script did its thing for about half a minute, shut the device down, and from that point on the StarBook refused to boot. Black screen, keyboard backlight on, the power LED lit, and the speakers occasionally producing a clacking sound. Sadly no amount of the usual turn-it-off-and-on-again rituals or battery disconnects brought it back. The only way out of this situation is to re-flash the BIOS chip externally using an SPI programmer. Star Labs do document this , however their guide assumes you’re using their programming kit together with a dedicated debug board and an FPC cable. That kit is significantly more expensive than a generic programmer, doesn’t list any make or model information, and, at the time of writing, has been permanently out of stock on their web shop. Not exactly helpful when you’re stranded somewhere with a dead laptop that happens to be the only computer you have with you. The good news is that you don’t need any of that, at least for this specific model of the StarBook . As Star Labs ’ own Sean pointed out in the GitHub issue I opened while debugging this mess, this specific StarBook uses a SOIC-8 flash chip, which means you can recover it with a cheap, generic CH341A programmer and an ordinary SPI clip, as long as you respect its voltage. Warning: Flashing a BIOS chip externally can permanently destroy your device if you do it wrong. The flash chip on the AMD StarBook runs at 1.8V and you must use a 1.8V adapter. Driving it at the CH341A ’s default 3.3V risks damaging the chip, and won’t read it correctly anyway. Everything below is what worked for me, documented to the best of my knowledge, but you’re doing this entirely at your own risk. The flash chip on my StarBook Mk VI (AMD) , which I could read off the silicon once I had the backplate off, is a Winbond 25R128JWSQ , a SPI NOR flash in a SOIC-8 package. The suffix on Winbond parts apparently denotes the 1.8V variants. The ubiquitous, three-dollar CH341A “black” programmers that you’ll find on AliExpress , Amazon , and pretty much everywhere else operate their SPI lines at 3.3V (and the parallel header at 5V). Clamp one of those directly onto a 1.8V chip and, best case, reads garbage. Worst case, however, you cook the flash or something downstream of it. The fix is a small 1.8V adapter board (essentially a level shifter with a voltage regulator) that sits between the CH341A and your SOIC-8 clip. These are sold as kits, e.g. the KOOBOOK CH341A Programmer + 1.8V Adapter combo that Sean linked in the issue. Make sure whatever you buy explicitly mentions 1.8V. You will need a CH341A programmer with a 1.8V SOIC-8 adapter, a SOIC-8 test clip (the spring-loaded “Pomona-style” clips, or the cheaper ribbon-cable variety, both work), a second computer running Linux (can be via a live medium, e.g. a USB stick) to drive the programmer from, e.g. a department store laptop and a Fedora live USB will do, if you’re eloquent enough to explain to the staff that you’re definitely not building what almost certainly looks to them like a bomb. You will also need the correct firmware image for your model (more on that below), a small Phillips screwdriver and, ideally, a plastic spudger. Power everything off and unplug the charger before you start. Flip the laptop over and remove the backplate by undoing the two long Phillips screws in the top corners first, and then the eight shorter screws around the edges. Lift the plate off carefully. Then, remove the five screws holding the battery in place (one of the screw positions is intentionally left empty) and gently unplug the battery connector. Last but not least, locate the SOIC-8 flash chip on the mainboard. It’s the little eight-legged Winbond chip described above. Note: While I had the StarBook open, I noticed that my (barely two year old) battery had started to visibly bulge, so do take a moment to inspect yours. A swollen lithium battery is a fire hazard and should be replaced. SOIC-8 flash chips have a defined pin 1, and the clip’s pin 1 (usually the wire on the red edge of the ribbon) has to line up with it. Get the orientation wrong and the chip simply won’t show up. For reference, the pinout of the Winbond SOIC-8 flash is: You don’t have to wire any of this up by hand, though, as the clip and the 1.8V adapter carry all eight lines for you. The only thing you need to get right is aligning pin 1 of the clip with pin 1 of the chip. Note: On my chip there is a gray dot painted onto the package, on the corner opposite to pin 1. Pin 1 is instead marked by the small indented (etched) dot, on the exact opposite side from the painted one. I have no idea why the gray dot is there, but if you align to it you’ll have the clip on backwards. Look for the indentation, not for the gray spot if yours has one too. With the clip attached, plug the CH341A into your second machine. A quick look at should confirm it enumerated: Install if you haven’t already: Before writing anything, make sure can actually talk to the flash over your clip: If everything is seated correctly, will identify the Winbond chip (detected as something like ). If instead you get: …then don’t panic. In my experience this is almost always poor clip contact rather than a real problem. I had to wiggle and reseat the clamp a few times before the chip showed up reliably, because those cheap clips are fiddly. Only proceed once the chip is detected consistently across a couple of runs. Even if the firmware is bricked, it’s good practice to take a backup before you overwrite anything. Read the chip twice and compare the dumps to be sure your contact is solid: If the two reads differ, your clip contact is flaky and you should reseat it and try again. As for the firmware image, Star Labs publish their firmware in a public GitHub repository . For external programming you want a full SPI image, not the EFI/ updater files. For my StarBook Mk VI (AMD) (product SKU ) that’s the image. The full-image files also live under the model’s directory . Pick the one that matches your model and rename it to something convenient, e.g. . Note: Star Labs ’ firmware versioning is, to put it mildly, a mess. As of writing, the last AMI (the original “BIOS”) release for the AMD StarBook is , while onwards is Coreboot . Whichever you decide to flash, just make sure it’s a full image for your exact model. Last but not least, write the downloaded image using the command: By default will erase, write, and then verify the chip. Star Labs ' official command appends (i.e. and ) to skip those verification passes, but I’d recommend leaving them off so confirms the write actually stuck. Either way, do not disconnect or disturb the programmer while it’s working. Once it finishes successfully, remove the clip, reconnect the battery, screw the backplate back on, and try to boot. When I powered mine back on, the StarBook came to life again, only to stop at a screen complaining about a missing boot entry, since flashing a fresh image also wipes the EFI boot variables. That’s nothing dramatic and you just need to point the firmware back at your bootloader. You can either use the boot menu and pick your SSD, which usually re-adds the boot entry, or boot a recovery/live system and run (this is what I did), or drop into the EFI shell and launch your bootloader manually: Note: On newer Coreboot releases Star Labs are enabling Rom Armor and anti-rollback. On the AMD board external flashing and downgrading still worked for me on , but this is expected to be locked down from onwards. What frustrates me most about this whole ordeal isn’t that a firmware update can go wrong, because that’s always a risk when you flash something. It’s that Star Labs ’ documented recovery path depends on a proprietary kit that nobody can actually buy, when a generic CH341A with a 1.8V adapter seemingly does the job just fine. However, this info is nowhere to be found in Star Labs ’ official documentation, which is why I decided to publish this write-up to begin with. Hopefully it spares the next person the day (and the stress) it cost me.

0 views
flowtwo.io 2 weeks ago

Building a New Blog Pt. 3

Long story short, I use a Cloudflare tunnel and self-host it on an old laptop. When I decided to rebuild this blog, one of my goals was to make it as "cloud-native" as possible. As a (mostly) static website, I concede it's sort of unnecessary since there are simpler ways of hosting a blog that don't require maintaining a backend at all. e.g. using Github Pages. But as soon as you want to do anything other than serve static content, it's easier if you have a backend you control. For example, I host some other projects on this domain and I have a custom publishing workflow for my posts—both of which require server-side logic. All that to say, I needed a backend somewhere and AWS is the cloud provider I'm most familiar with through work. But at work, it's not too often you get to try out new cloud services and stand up infrastructure from scratch. So, as with all my side projects, I took this as an opportunity to learn by doing. my odyssey through the clouds and back home The first question was where to store the data. My existing blog used MongoDB, so I stuck with that to make the migration easy. The most cloud-native version of MongoDB is AWS DocumentDB —it's a managed database service with full MongoDB compatibility. The pricing page says: With Amazon DocumentDB, you only pay for what you use, and there are no upfront costs. Sounded good to me. Did I actually do the math on what an instance costs per-month? No. But after 1 month I found out it was lot. That's the purple bar on the stacked bar graph above. Paying $100 a month for a blog database was not in the budget. This is when I learned that using a managed database service for my blog posts is like buying a yacht for a canoe trip. Then I found out that Mongo Cloud offers a free tier with up to 512MB of storage, so I switched to that. That's plenty for blog post storage. I guess that's still a managed database service, but it's free...so I took the free yacht. Next I had to decide how to host the backend. Building the backend as a Docker image would make it the most portable and easy to migrate, so that was a requirement from the start. Naturally, the first thing I tried was AWS Elastic Container Service (ECS) , a managed container orchestration service. Just upload your Docker image and click deploy, basically. I tried that for a couple months, but just like DocumentDB it is prohibitively expensive—even at the smallest scale of deployment. It's the teal bar on the graph above. I stopped using ECS and instead just installed the Docker engine on the smallest EC2 instance available ( ). My deployment workflow is just pushing an updated image to DockerHub and then pulling and running it on the server via a Docker compose file. For awhile, I also proxied all the traffic to my blog through AWS Elastic Load Balancing (ELB) , but not because I actually needed load balancing. I wanted to use ELB to handle SSL termination and automate the SSL cert renewal for my domain name. The alternative, which I had done before, was to route directly to my EC2 instance and handle SSL traffic directly via the app. I didn't want to do this again because using certbot + Java keystores (I run a JVM backend) was a huge headache, especially when it came to automated cert renewal. But again, using a managed service like ELB is not cheap, even if you have very little traffic. It's the orange bar on the chart above. Paying an extra $10 a month just for SSL management...also not in the budget. The better solution was to install an Nginx sidecar directly on my EC2 node which can handle SSL and proxies requests to my app. There's other benefits to using Nginx too, like response caching. After adding some aggressive cache expiration to the Nginx config, very little traffic actually ends up hitting my app and database. All these optimizations dropped my hosting costs down to about $15 a month. Not bad... an acceptable cost for me to have a space to host side projects and share content. But after running this for about a year, I looked at an old laptop I had sitting under my desk and thought, why not just run it there. For a laptop running linux, all you need to do is run: And now your computer never goes to sleep. Boom—you have a server. From the networking side, I originally thought it would require paying for a static IP from my ISP, but turns out there are several good options for proxying inbound traffic to your home LAN now. I chose to use a Cloudflare Tunnel , which is completely free, and it works great. Now I don't pay anything at all to host my blog*, and I still have the flexibility to run and deploy anything I want on it. It's kinda funny since the laptop is where I did most of the development for this blog. Now it's left the cloud and come back to the same machine. you shall return from whence you came Even though the premise of AWS is "elastic" services that scale from 0 to infinity, there's generally a minimum level of traffic needed to make the operational benefits of their managed services worthwhile, financially. For some of the services, the pricing doesn't actually scale down to the level of a personal blog in terms of resource and traffic requirements. Pro-tip: utilize the free tier from cloud providers for small projects! There are tons of ways of hosting something like this for free. And a million ways to do it the wrong way, as I showcased above. For example, DocumentDB was costing me $100/month. DynamoDB would've been free for the same use case. Self-hosting in 2026 is much more viable now thanks to cloud networking tools like Cloudflare Tunnel, Tailscale, Wireguard, or free VPS providers. You don't need to open up ports on your home network to the internet or pay for a static IP. * Alright, technically the laptop consumes about 6kWh of electricity a month, which costs me ~$0.80. Free-ish. Even though the premise of AWS is "elastic" services that scale from 0 to infinity, there's generally a minimum level of traffic needed to make the operational benefits of their managed services worthwhile, financially. For some of the services, the pricing doesn't actually scale down to the level of a personal blog in terms of resource and traffic requirements. Pro-tip: utilize the free tier from cloud providers for small projects! There are tons of ways of hosting something like this for free. And a million ways to do it the wrong way, as I showcased above. For example, DocumentDB was costing me $100/month. DynamoDB would've been free for the same use case. Self-hosting in 2026 is much more viable now thanks to cloud networking tools like Cloudflare Tunnel, Tailscale, Wireguard, or free VPS providers. You don't need to open up ports on your home network to the internet or pay for a static IP.

0 views
NULL on error 2 weeks ago

As gambiarras mais impiedosas que já fiz: parte 1

Gambiarra, de modo geral, é conhecida como uma espécie de quebra-galho ou atalho para conseguir algo usando a criatividade, muitas vezes de forma pejorativa. Ressalto que, neste texto, a ideia é justamente enfatizar a criatividade diante de grandes desafios. Certa vez, enquanto trabalhava com aplicativos mobile e livros digitais, fui encarregado de renderizar um livro em um aplicativo Android nativo. React Native ainda não existia. Talvez o Sencha Touch já existisse, mas, de qualquer forma, os tablets-alvo eram muito lentos e tinham pouquíssimos recursos. Pois bem, o que você faria nessa situação? Fácil, não é? Renderizaria as páginas do PDF como imagens JPEG e criaria um aplicativo para paginá-las, a.k.a. um livro digital. Não, claro que não! Seu gerente gosta de desafiar você e não gostou nem da qualidade das imagens nem do espaço ocupado em disco. Então, ele propôs usar Adobe Flash. Sim, naquela época ele ainda existia, embora já estivesse no fim da vida. Mas como? Ninguém sabia. Foi então que tive a ideia de criar uma WebView de um pixel por um pixel e embutir o Adobe Flash nela para renderizar as páginas sob demanda e com a máxima qualidade. As páginas viriam de um arquivo , pois o software usado para criar os livros exportava a arte vetorial para SWF. Nos PDFs, por algum motivo, tudo era convertido em bitmap. Mas surgiu outro problema: como transferir dados entre o aplicativo Android, escrito em Java, a WebView, em JavaScript, e o Adobe Flash? Criei uma WebView de um pixel por um pixel com o Flash Player rodando dentro dela. Na época, essa era a única maneira de executar Flash no Android, pois o Adobe AIR ainda não existia. Fazendo o Java chamar , eu conseguia executar uma função JavaScript e passar o bitmap renderizado da página solicitada. Sim, o bitmap inteiro era transmitido como uma string. Por sorte, a WebView não tinha uma limitação de 4.096 caracteres, ou algo semelhante às limitações comuns em servidores HTTP. Portanto, era possível transmitir páginas muito grandes e em alta resolução. Feito isso, bastava repassar os dados para o Flash. Mas como? Exemplo simplificado: No lado do Flash, em ActionScript: Então, no lado da WebView, em JavaScript: Feio? É, eu sei. Mas funcionou perfeitamente. Passamos a ter páginas com alto DPI, zoom, rotação e tudo mais. Como bônus, ficou bem rápido, pois o aparelho precisava renderizar cada página apenas uma vez. Isso acontecia em menos de meio segundo. Depois, bastava desenhar a imagem na tela. :-)

0 views
マリウス 4 weeks ago

The TEMU-fication of Software, Digital Goods & Services

Disclaimer: This is an opinion piece and most of it is speculation about a future that has not arrived (yet?), based on a few data points that have. As usual, summary at the end. A few years ago I would have laughed at anyone telling me that there is a serious market for ten-dollar drills, two-dollar dresses, and one-dollar pairs of shoes shipped from a warehouse on the other side of the planet. Today, however, that market exists and it has a name, and it is even publicly traded (sort of, through holdings). TEMU , Shein and a few others have built frankly mind-boggling businesses around the idea that if you make production cheap enough, fast enough, and just barely good enough to look right on a phone screen, an enormous part of the population will buy it, even when the product breaks within a week, when the materials it is made of contain worrying levels of toxic substances , and when the carbon footprint of one delivery exceeds that of an equivalent local purchase by orders of magnitude. The key to this sort of business model is not innovation, but instead the externalization and compression of cost. Somewhere upstream, people work seventy-five hours a week , in conditions most readers of this website would refuse to even visit, so that the rest of us can have a cheap plastic spatula at our doorstep within five business days. While the visible price collapses, the invisible costs get distributed onto landfills, lungs, and ultimately people that we will never meet. What follows is a hypothesis I cannot prove but have been turning over in my head for a while, as we are watching the same thing happen to software, books, music, (film-)scripts, and most of the digital goods and services we consume. The cheap labor in this case is not human, it is a Large Language Model ( LLM ), or what many people these days call “AI” , and the externalized cost is, among other things, quality , which requires craftsmanship to produce, and attention to perceive. And just like with physical goods, we will probably end up with a two-tier market, in which we have a large and massively profitable lower tier of generated slop , and a smaller, more expensive upper tier of work that is still recognizably human. I’d like to call this the TEMU-fication of software, digital goods and services , and describe what it might look like. For decades, the global fashion industry has relied on a workforce that has almost no leverage and no voice, and for which the economics work because someone, somewhere far away, will sew a t-shirt for less than the price of a coffee. Without that skewed arrangement, the entire fast fashion business model collapses. The garment in your hand is only cheap to you because it has been expensive to someone else , in ways that the price tag does not show. Modern Large Language Models occupy a similar position in the economy, with one important difference, which is that there is no human being in the sweatshop, only a stack of GPUs trained on a corpus of work that other human beings produced over the course of decades. The labor that has been compressed is historical and the model is a kind of compressed copy of the work of millions of programmers, writers, illustrators, and musicians, served back at near-zero marginal cost. Well, at least in theory, and only if the hyperscalers find a way to lower the cost per token, but that’s a different topic. However, the result is the same. A class of goods can suddenly be produced for an order of magnitude less than before. And, just like with TEMU , those goods turn out to be just barely good enough . The most direct manifestation of this so far is what is being called vibe coding . The term refers to the practice of describing what you want in natural language to an LLM , accepting whatever it produces, iterating over it with more refined descriptions of the basic idea and eventually shipping the result into production. Whether the developer actually understands what was generated is increasingly considered an implementation detail . And while the output is technically software, the question is what kind of software it is. A 2025 Veracode report found that approximately 45% of AI-generated code samples failed security tests and contained critical vulnerabilities from the OWASP Top 10 , and a multi-language, multi-model academic study that evaluated outputs from Claude , Gemini , Codestral , GPT-4o and Llama-3 across Python, Java, C++ and C, found that a substantial fraction of generated snippets were either non-compliant with basic secure coding standards or actively triggered classified weaknesses (buffer overflows, hard-coded credentials, SQL injection, cryptographic misuse, path traversal, you name it). Even more concerning is a peer-reviewed 2025 paper from IEEE-ISTAS that documents a 37.6% increase in critical vulnerabilities after just five iterative prompts, suggesting that the more you let the model refine its own code, the worse the security posture gets. When these issues compound over time, the result is a higher total cost than traditional development. However, this doesn’t matter when you don’t think long term , but fast fashion instead. Also, none of this is to say that an experienced engineer cannot use these tools well, because they certainly can. The issue is what happens when the same tools are used by someone who does not know what good looks like in the first place, and there is nobody downstream of them who does either. The output passes the basic test of it runs and looks plausible , ships into production, and accumulates the kind of architectural and security debt that surfaces only when something goes very wrong . Note: There are credible voices in the industry, particularly from the AI tooling vendors themselves, who argue that AI-assisted development raises a floor more than it lowers a ceiling. In this view, the median piece of software has always been mediocre, written under deadline pressure by tired humans, copied from Stack Overflow without much thought, and held together by duct tape. If an LLM produces output of roughly comparable quality in a fraction of the time, the argument goes, nothing got worse. We are simply removing a bottleneck. I find this argument partially persuasive, and partially convenient for the people making it. It is true that a lot of software was already not great, but it is equally true that there is a difference between bad code written by a human who at least understood what they were doing , and bad code written by a system that does not understand anything . The first kind can be questioned and corrected, but the second kind tends to compound, because the person shipping it cannot answer why it does what it does. At least for now. Software is not the only place where this is playing out. The book industry is arguably further along, with estimates suggesting that somewhere between ten thousand and forty thousand AI-generated books are uploaded to Amazon ’s Kindle Direct Publishing platform every month, many without any disclosure that a model was involved. In June 2023, the Kindle Top 100 bestseller list was found to contain only 19 books written by humans . Amazon has since introduced limits and disclosure requirements , but enforcement is patchy and authors continue to push back against what looks like a slow flood. Categories that have been hit particularly hard include travel guides (generated guides to cities the author has never visited, with restaurant recommendations that don’t exist), nutrition and health (generated diet advice with citations to studies that don’t exist), and public-domain rewrites (generated adaptations of older books, relying on the recognizability of titles that the actual authors never agreed to). Travel guides in particular have produced a small genre of stories where readers arrive at addresses that turn out to be empty lots, or follow walking directions through neighborhoods that no human would ever recommend. Note: The defense, again, is that the bottom of the book market was always full of filler, that print-on-demand has been around for a long time, and ghost-written business books and assembly-line genre fiction predate generative AI by decades. However, the new thing is the scale at which low-effort content can now be produced, and the speed at which it can drown out the rest of the catalogue. Authors are competing for shelf space against entities that can ship a hundred new titles in a weekend. A 2025 analysis of 65,000 English-language articles published since January 2020 found that a little over half of all new articles on the internet are now AI-generated , and it’s not only the written word that’s being churned out by machines . YouTube has its own version of the problem, where, according to a Guardian analysis, nearly 10% of the world’s fastest-growing channels feature nothing but AI-generated content , and on Shorts specifically more than one in five videos served to a new user is low-quality AI-generated material . Not even the highly creative and (up until recently) human process of making music is immune to this TEMU-fication . Spotify has been removing ghost artist tracks for years, but the practice scaled up dramatically when generative tools made it trivial to produce convincing lo-fi background music in arbitrary volume. The platform has reportedly removed 75 million spammy tracks in a single year , and high-profile acts like the AI-generated band The Velvet Sundown amassed over a million streams before being unmasked. There has been at least one criminal case, involving over $8 million in fraudulent royalties , built entirely on AI-generated music and bot streams. However, that is no reason to applaud Spotify , as the company appears to fight the AI spam only when it’s someone else trying to make money off of it. However, there is a sliver of hope, as engagement with AI-generated articles reportedly dropped by around 40% in 2024, and human-generated content seemingly still gets roughly 5.4× more traffic than AI-generated material in some studies. About 38% of consumers openly express skepticism about AI-created content, and people do still seem to be voting with their attention. Whether that vote is powerful enough to shift incentives at the platform level is a different question, and personally I’m not particularly optimistic, especially given that the platforms profit either way. Let’s take Netflix as an example. From my understanding, the WGA ’s 2023 deal explicitly prevents studios from treating AI-generated material as source material, or from using AI to write or rewrite scripts, and Netflix was seemingly bound by that agreement until at least May 2026. Netflix ’s own Generative AI Production Guidelines also seem to reflect this, stating that AI is permitted in ideation , but that its use should not replace or materially impact work that would otherwise be done by union-represented writers, actors, or crew members, without proper approvals . While that sounds reassuring on the surface, it is, in my view, a delay and not a limit. The same company has publicly committed to going all-in on AI in its production pipeline , has signed deals with VFX automation providers that explicitly put a chunk of the global VFX workforce at risk, and has already used generative AI in at least one of its programs ( El Eternauta ). The trajectory seems to be “use AI everywhere it is contractually allowed right now, expand into the rest the second the contracts permit it, and spin the result as dEmOcRaTiZaTiOn Of CrEaTiViTy” . So here is my specific (and quite possibly wrong) prediction: Within the next five to ten years, Netflix will offer a basic subscription tier whose catalogue consists predominantly of AI-generated or AI-assisted content. We are talking generated procedural shows where each episode is remixed from a small set of templates, generated kids’ content that is vaguely educational and impossible to remember an hour after watching, and generated dramas that recycle plots from existing IP and vibe the rest. For this, the viewer pays the lowest monthly price, while the platform pays nearly nothing in production cost and keeps an enormous margin. The only “upside” for consumers will be the lack of ad breaks, as targeted advertising will quite possibly be injected in real-time into the show you’re watching, seamlessly blending into the storyline without you noticing it, but ultimately still triggering your ape brain to crave a refreshing soda or a sweet treat . Their premium tier, meanwhile, will become the human-made tier. Series with credited human writers, films with credited human directors, and performances by humans whose likeness has not been digitally replicated. The marketing will not call it human-made , because that would be admitting that the cheap tier isn’t , but the price difference will make it obvious. You will pay extra for the same thing Netflix has been selling you all along, except now it is positioned as a luxury. Clearly, I cannot prove that this is what will happen. Netflix ’s own guidelines, as written, prohibit it, and the WGA deal forced a delay. But once the contractual block has lifted, the financial logic is hard to argue with. A streaming service that can produce good enough content for fractional cost will eventually try to. And, mind you, Netflix is just one example. The same logic applies to every other content-distribution business with a subscription model and a margin. If you want to know what the human side of this two-tier world looks like, I think the best existing model is the handicrafts and handmade goods market . By 2025, that market was estimated at roughly USD 987 billion globally, with projections reaching over USD 1 trillion by 2035 . There is data suggesting that U.S. consumers already spend almost a fifth of their money on handmade goods rather than on mass-produced equivalents, and over half of handicraft buyers globally indicate a preference for products that are eco-certified or made from natural materials, going in the exact opposite direction of what TEMU has been doing. What this market shows is that industrialization does not erase the artisans, but pushes them into a different segment. People did not stop buying handmade chairs when factories started making chairs cheaply. While the masses opted for the cheaper, mass-produced items, a small but sustained minority of buyers continued to seek out the human-made version, and over time were willing to pay a premium for it. If the hypothesis holds, software engineering, writing, acting, illustration, composition and the other content-producing professions will undergo something similar. The bulk of the market will migrate to the cheap, mass-produced, generated tier, while a smaller market will continue to value, and to pay for, work that is verifiably the product of a thinking, breathing, opinionated human being. We are already seeing the first signs of this in agencies that explicitly advertise human-only content (at a premium), and in licensing companies flagging tracks as human-composed to distinguish them from AI library music. I think that the interesting question is not whether this segmentation will happen, but what proportion of the market ends up in each tier, and how robust the upper tier turns out to be. There is a darker version of this analogy. Roughly 57-60% of the daily caloric intake of the average adult in the United States and the United Kingdom now comes from ultra-processed foods . Across 22 European countries the share ranges from 14% to 44% , depending mostly on how protected the local food culture has remained. These foods are cheap, abundant, available everywhere, and nutritionally inferior to the alternatives in ways that have been studied at length . People know this, but they eat them anyway, often because the alternatives are slower, more expensive, harder to find, or require skills that have not been taught. I suspect that AI-generated content is on the same path. The cheap tier will not be a marginal phenomenon serving a marginal audience, but it will be the default , the cornerstone of how most people consume software, entertainment, news, and information, because it is what the platforms will serve them and what their monthly subscription covers. Some will care enough to seek out the alternative, but most will not, just as most people, knowing what they know about ultra-processed food, do not change their grocery habits. Probably the strongest counter-argument to all of this is that LLMs are still early, that the quality issues are transient, and that within a few model generations the gap between AI-generated and human-generated work will narrow to the point where the distinction stops mattering or might not even be possible anymore. If that is true, the two-tier picture collapses, because there is no longer a quality difference to justify the upper tier, only a marketing difference. The handmade analogy breaks because, unlike a hand-built chair, a generated novel is functionally identical to a written novel once you can no longer tell them apart. However, I am doubtful that this is going to be the case. There are tasks where I have watched the gap narrow faster than I expected, but there are also tasks where the gap has stayed stubbornly fixed and the failures have just gotten more sophisticated. My instinct is that for narrow, well-bounded technical work, the gap will close further. For long-form work that depends on a coherent worldview, lived experience, and, most importantly, emotions, I doubt it will, because the model has none of those. The second counter-argument is that the consumer backlash will be stronger than I am giving it credit for. The 40% drop in engagement with AI-generated articles is not nothing, and platform incentives may shift if users start to penalize AI-flooded feeds. Apple and others have started experimenting with content provenance and disclosure schemes that, if widely adopted, could stop the worst of the flooding. So it is possible that I am underestimating the immune response . The third counter-argument is, that the cheap tier might not be sustainable at all, because AI-generated content trained on AI-generated content degrades model quality , and the broader ecosystem ends up poisoning its own training data. If that turns out to be the dominant dynamic, the cheap tier could collapse before it becomes entrenched. I think all three of these arguments are valid and have a certain weight to them, but none of them are strong enough, in my view, to make me confident that the TEMU-fication will not happen. They might modulate how it happens, but they probably do not stop it. Initially, I went looking for an optimistic ending for this write-up, to say that software engineering is not going away , and writers are not going away , and actors are not going away . And while all of that is, I think, true, none of it should be confused with things will look the same . What I expect, and what I am to some degree already seeing, is that the people producing software, books, music, scripts, and other human-made work will not disappear , but they will get pushed into a narrower, more specialized, more “luxury” -coded part of the market, pretty much the same way hand-bound notebooks, independent record stores, and small bakeries that mill their own flour did. There will still be a livelihood in it, at times a very good one, but it will look vastly different, and there will probably be fewer people making a living in these fields. My assumption is that they will be more visible inside their niche, but less visible outside it, and they will make their case in part on the basis of provenance , where something was made by a human who knew what they were doing, and you can tell. Meanwhile, the bulk of what most people interact with will, I suspect, be generated. Some of it will be fine, and some of it will be ultra-processed , in the same sense that a frozen lasagna is ultra-processed. It will be functional, calorically adequate food , but it will not be what your Italian grandmother was making. People will nevertheless eat it because it is there, it is cheap, it is convenient, and because the alternatives have been priced out of their daily life. There is no “inevitability” to it, because none of this is really decided yet. There are still choices, made by platforms, by regulators, by consumers, and by the people doing the actual work, that will shape which tier ends up being how big and how durable. The handmade market exists because enough people kept buying handmade goods to make it viable. The human-made tier of software and digital goods will exist because enough people keep buying it, or it won’t exist at all. If you are someone who writes code, or stories, or music, or scripts, by hand, with intent, and with a point of view, I do not think the LLM is going to kill your job . I do think, however, that it is going to change the shape of the market you operate in, push you toward the upper tier (whether you wanted to be there or not) and ask you to make a more deliberate case for why your work is worth the difference in price. For the rest of us, the more interesting question is which tier we are choosing to consume from, and whether we are choosing it on purpose, or just because it was what the algorithm served us by default. I have my suspicions about the answer, but I would love to be wrong.

0 views
./techtipsy 1 months ago

I guess I'm a cactus farmer now

When I was a teenager, I got gifted a cactus. I didn’t know much about it and had never really had any plants of my own. More than a decade later I looked it up, and it seems I received a cactus of “Mamillaria” family, which is somewhat amusing once you learn what that name roughly translates to. Estonia isn’t known for being an environment where cacti thrive. We have a nice spring, short and sweet summer, nice and chill autumn, and the car-rusting winter season. And yet mine did, eventually. The cactus started growing in length. It continued until it couldn’t continue vertically, and then it started slanting sideways, eventually falling over with its pot. That was annoying to deal with. One day I woke up to my phone alarm ringing, reached out to it only to find that the cactus had fallen over it and I had just found where the cactus was. The cactus also survived a lot of abuse. At one point I put a bit of sawdust from the cage of pet mice that I had (adorable but difficult pets by the way!), and it actually liked it a lot and started growing even faster. Once it had grown horizontally for at least 20cm, it decided that now is a good time to fork it and to grow in more than one direction. I left the cactus alone, occasionally repotting it when necessary, and keeping it from falling over all the time. Then an acquaintance mentioned that hey, you can actually pop those forks off from the cactus and grow them separately. That’s when things got out of control. Once I popped one off, new ones would start growing from the host cactus nearby. Not long after, I had 37 cacti. I started giving them away because annoyingly these things are actually very cheap in gardening shops and I ran out of physical window sill space to put them on. These cacti also bloom in a really nice way, usually once a year near April or May in Estonia, but sometimes again in July-August. All the warnings about over-watering cacti are also very relevant and I suggest looking up the correct behaviour. With age, it has become easier to handle cacti with my bare hands. Sometimes the needles stick in, that is a bit annoying. A few years ago, I had gifted a bunch away and kept some for myself. Everything was going well, until a succulent that I brought in as an IKEA store rescue started having some white things appear on it, and not much time later it got to my cacti. Eventually, most of my cacti shriveled away and the few that were in a different room were not doing great either. The original cacti that spawned this decade-long adventure was now dead. One of the cacti that I gave away had started growing a lot. Ridiculously well. And just like its parent, it spawned baby cacti on it, about a few dozen of them at a time. Things got quite unwieldy and we agreed to a trade: I get the big boi and I will give a smaller one in return. I am once again in possession of a large cactus that loves to grow horizontally, and this one spawns babies like crazy. The trick is to try to wiggle them off, then leave them to sit for a few days to a few weeks away from direct sunlight until they start showing signs of roots, pot them into soil, and grow them within natural light, but not direct sunlight to avoid them drying out. If you’ve done everything correctly and have avoided overwatering them, you will have a bunch of new cacti! I’ve also learned that these plastic growing thingies are great for the first potting as it’s much more space efficient compared to giving each an individual pot. Is that the right way to grow cacti? Probably not, I’ve been winging it this whole time and only have a vague understanding of what to do. Not cacti advice. I guess I’m kind of a cactus farmer now. Send help. Or at the very least clay pots and cacti soil, I’m running out.

0 views
Ivan Sagalaev 1 months ago

Categorization with NLP

Since launching my categorization tool Shoppy I've had some fun analyzing collected data which resulted in considerable complication of the prediction model. And now I feel the urge to write a deeper dive into its inner workings. I'm not sure how useful it would be for anyone who isn't a part of the Grocery Categorization industry, but hopefully some NLP tricks could be at least interesting to any general practitioner. Please note that I'm by no means an NLP expert! Part of the reason for writing this kind of posts is to try and nerd-snipe someone who knows more into sharing their expertise. Oh, and it's not a short one, this! Settle down :-) Just to remind everyone, what I'm trying to solve for my slowly developing shopping list app is suggesting grocery categories for products. So that it knows that "Milk" is dairy, "Apples" is produce and so on. This categorization helps with grouping and sorting, and it also looks nicer with colors and appropriate icons. The usual approach to solving such problems is Machine Learning, and specifically — classification . That doesn't work for me though, as I couldn't come up with any means of collecting enough data myself, and hiring a consultancy is way above the budget of a tiny personal project. So instead I'm manually crafting a clever algorithm with explicitly handled edge cases. The first step is turning free-form input into something more formal and predictable: a set of lexemes. Step by step, it looks like this: This gives me a normalized, stable input key independent of basic morphological variants: A note on stemming. I'm using the original Porter algorithm . It's widely supported, but is also the most simplistic. I don't really care about the correctness of the result from the point of view of the English language. As long as the algorithm used to produce the data is the same as the one used for checking against it, I'm good. The straightforward design for a database is just a mapping from a whole key to a category: . But that would require listing all likely real-world products: all sorts of apples, peppers, beans, etc. Which doesn't work for me since, as I mentioned, I don't have a firehose of data to fill it up. But you'll notice that mostly all such product are defined by one word: an apple is an apple and is produce, regardless of the sort. So let's reify this in the form of a CSV file: These one-word keys are called unigrams . To match a search query, we can simply look up every separate unigram from it in the database. It works for surprisingly many cases, but it breaks on things like "apple juice", because despite having "apple" in it, it's a beverage, not produce. This is fixed by making the order of rows in the database significant, so that goes before . Then, if several of the unigrams in a search query match, the earliest one wins. This might smell to you like something prone to potentially irresolvable ordering cycles, but I actually found that I only really need two groups of significance: Things like "juice", "milk" and "oil" are usually derived from something, as in "apple juice", "oat milk" and "olive oil". Keeping those derivations above raw ingredients ensures those tings are categorized correctly. And don't take the word "raw" too literally. There are things like "ketchup" in there! But since nobody puts "tomato ketchup" on their shopping list, it is considered "raw" in my domain. The next problem is combinations of words. "Spaghetti squash" is not a pasta, but a kind of squash. And "apple sauce" is neither produce, nor a condiment, but a snack! In both of these examples no single unigram is enough to correctly identify the product. This means I need to consider two-word terms called bigrams : All bigrams come before unigrams, as their purpose is to serve as more specific disambiguations of conflicting unigrams. But this ordering is less significant than the groups (Derivations and Raw), so each of those gets their own set of bigrams. At search query time, I produce bigrams as all possible combinations of two from the set of unigrams. Then they're added to unigrams as search terms: The lookup process stays exactly the same: just check all terms one by one. By the way, I need this code to also work in the Kotlin code of my Android app, and since Kotlin doesn't have in the standard library, I wrote a trivial recursive implementation by hand: Felt like solving a coding interview problem :-) A few words about "pepper"… When combined with another word, it most often belongs in produce: "bell pepper", "chili pepper", "serrano pepper", etc. And since there are many of such, I want to classify the unigram "pepper" as produce to cover them all. However the single word "pepper" usually means ground black pepper, which is a spice! I tried to complicate the model at first, supporting wildcards like "* pepper". But I didn't find any other exception that would use it, so I de-complicated the model back and hard-coded a dumb `if` substituting "pepper" with "black pepper" before any lookups :-) Practicality beats purity! Human languages tend to merge words commonly used together. So something straight and forward becomes straight-forward and then just straightforward. I realized I need to care about it here after I collected examples like "redbull" and "lipbalm". Both should properly be spelled as two separate words, but people usually don't consult a dictionary before doing their groceries, so… Funny factoid: "breadcrumbs" and "seaweed" are spelled as single words. This can't be solved by spellchecking because my database doesn't contain original spelling: it has the bigram "bull red" and the unigram "balm", and they both are too far away from those search queries. Instead, I started thinking about splitting words into syllables. To my everlasting surprise, nltk turned out to have several of those, from which I picked SyllableTokenizer . It is pretty simple, but it does work for "redbull" and "lipbalm" as you'd expect. At first, I tried to switch the entire database to contain only syllables instead of full stemmed words, but it didn't work out. There are common syllables like "bar" and "can" that are also full words in their own right. So something like "barbecue sauce" would be split into and will suddenly become a snack because I have a record saying that any kind of "bar" is a snack. So instead I converged on a two-step lookup. First I split the search query into regular words and look them up as I did before. Then, if I didn't get a match, I split the words further into syllables and do a second lookup. The syllabilization algorithm was another thing I had to port to Kotlin, because I couldn't find anything like that in the entire Java ecosystem. Drop me a note if you want that code for some reason. Spell checking proved to be necessary nonetheless. Do you know if it's "fusilli" or "fussili"? Does "camamber(t?) has a "t" at the end? Or is it "haloumi" or "halloumi"? (Answers: 1) the former, 2) it does and 3) both spellings are correct.) Unfortunately, a regular spell checker against English wouldn't work very well: all kinds of international foods and deliberately misspelled brand names kill the idea. Fortunately, I can spell check against my own database, which naturally represents the entire language that my app knows! And the standard approach to spell checking is to employ some "edit distance" algorithm which shows how far apart is the spelling of two arbitrary words. I went with Damerau-Levenshtein (standard Levenshtein plus transpositions). I also had to come up with empirical numbers for maximum allowed distance depending on the word length. This was done completely unscientifically, and I expect to have to tweak it further. But for now it works! The most astute reader might have already spotted a problem: what was a single hash-table lookup before the introduction of edit distance calculation has now turned into a linear search with each test being way more expensive than a straightforward string comparison. So I had to to some low-hanging fruit plucking in terms of optimizations: But I have to say that the main thing working in my favor here is that the data is just small ! I want to highlight two particularly weird wins made possible by the combined effect of syllabilization and spell checking. And I just want to state for the record that the existence of such a thing as "mayochup" made me die a little inside. Is the trouble of mixing mayo and ketchup by yourself in your kitchen so unfathomably hard that you need a brand to produce it for you? Geez… Initially I thought of putting the collected data up on Kaggle , but since it's now heavily dependent on a custom algorithm, I'm just going to leave it as is in the Shoppy repo: Lowercase the input string Normalize it into the [NFD Unicode form][] — the one that separates accents from their characters, which lets me get rid of the former (not everyone bothers typing "crème fraîche" properly). Split the string into "words", ignoring everything else like punctuation and whitespace. In my case words are defined as alpha-numeric characters and an apostrophe (because I want things like "7'up" to be one word). For each word, get rid of apostrophes and stem them. Sort the resulting list of word stems. Assume people don't usually misspell the first letter of a word. Don't bother comparing words that differ in length by more than the maximum allowed distance. Don't bother comparing words with different n-grammness (like bigrams and unigrams). If a search term matches exactly, don't bother two check the rest of them to find a closer match. A single term "may o" works for: "mayo", "mayonnaise", "mayones" and even "mayochup". Syllabilizing these words produces "may" and some variant of "o"/"on"/"oc", which is then smoothed over by spell checking. One of the previously unknown to me items in the feedback was submitted as "separilla", which turned out to be a misspelling of " sarsaparilla ". So I picked out three syllables conveying the meaning: "sar", "pa" and "ril". And, thanks to spell checking, they're enough to cover both the correct spelling and a few incorrect ones. lib.py : all the functions terms.csv : terms database (in a weird format of CSV-with-blank-lines-and-comments) kinds.csv : hierarchy of defined categories (I didn't mention it, but you'll get the idea)

0 views
Farid Zakaria 1 months ago

Nix finally has a source-bootstrapped OpenJDK

One of the earliest requested issues we had opened on GuixPkgs was to add more packages, specifically OpenJDK. “Specifically I would like to see openjdk translated. openjdk is not bootstrapped from source code in nixpkgs” [ issue#3 ] Ever since I learned about the stage0 bootstrap chain and how Guix announced full-source bootstrap for all packages in 2023, I was in awe. They provided a package graph of more than 22,000 nodes rooted in a 357-byte program, including the JDK. 1 GuixPkgs can now build OpenJDK 25 🎉 Does it really work? Let’s take it for a spin. lives in the output, since Guix splits the package. is written in Java. is C++, but the class library it needs is Java, and the compiler that compiles the class library is Java, and it runs on a JVM that needs a class library… and so on. This is a bootstrapping problem. Every distribution resolves this the same way in practice: download a JDK and use it to build your JDK. Debian documents the pain , and the Bootstrappable project has a whole page on it. They are fun reads, I highly recommend them. What makes JDK special is that there is no actively maintained JDK that can be built from source without a JDK. The authors had to go back quite a few years to find one, and bring it back to life. Nixpkgs does the same. is built by : a 135 MB prebuilt-tarball. Nixpkgs makes it pretty easy to audit in the meta.sourceProvenance of the package. What does the source provenance of GuixPkgs’ look like? It’s a little whacky but the overall build chain in Guix is the following: Nineteen complete JDK builds, from a C++ program. We can use to emit the entire closure as a dot file to visualize the differences. I restyled the nodes and edges: dots instead of labelled boxes, and the red to highlight the derivations that exist only because of the JDK bootstrap. The upper panel’s entire Java story is those two red dots and the one edge between them: → . The lower panel’s is the 33-node constellation. Why is the Nix graph still so big if I claimed it was from a binary distribution? It turns out that a JDK closure is mostly not Java. It is a large C++ program that wants X11, cups, fontconfig, freetype, alsa and zlib, sitting on a C toolchain that both sides bootstrap from source. That sub-graph is the same in both and it swamps everything. I was curious to zoom-in and see the source-bootstrap portion of the derivation graph. Since that shared sub-graph is drowning the signal, I chose to filter it out. Which derivations exist in this closure only because of how the JDK is bootstrapped? 🤔 That is a reachability query. We delete the Java-provenance nodes from the graph, see what is still reachable from the root, and whatever is left are the derivations that exist only because of the JDK bootstrap. This lines up with our intuition. Nixpkgs only needs two derivations to bootstrap the JDK, while GuixPkgs needs 876 . Note Interestingly, the bootstrap build for JDK has in its closure: IcedTea 8 wants GTK 2, which wants its own Mesa, which wants 😱 What started off as a fun art project , started during TacoSprint 2026 , has now found relevance to those interested in bootstraping and reproducible builds. Nixpkgs has made a lot of progress on this front as well, however it is still not fully bootstrapped and relies on plenty of prebuilt binaries.  ↩ jikes 1.22 (C++) compiles GNU Classpath 0.93 , a free reimplementation of the Java class library. Classpath is enough to bring up JamVM 1.5.1 , a JVM written in C. Now something can run Java. Now we can finally use Java tools: Ant and ecj , the Eclipse Compiler for Java. Then it doubles back and rebuilds all of that with the newer versions to get more modern JDK features. That stack finally compiles IcedTea 2.6.13 (OpenJDK 7) → IcedTea 3.19.0 (OpenJDK 8) → OpenJDK 9 . After which it is one rung at a time: 9 builds 10, 10 builds 11, …, 24 builds 25. Nixpkgs has made a lot of progress on this front as well, however it is still not fully bootstrapped and relies on plenty of prebuilt binaries.  ↩

0 views
Lalit Maganti 1 months ago

How I Find Problems to Solve as a Staff Engineer

Note: this post was revised after publishing for increased clarity, based on reader feedback . “How do you find problems worth working on?” a senior engineer I mentor asked me recently. He’s trying to make the jump to staff engineer and realized that the role isn’t just about doing the work he’s assigned. He also needs to get involved in figuring out what his team and org should be building. Someone else had suggested blocking out time in his calendar to think about the bigger picture. He’d tried that, but hadn’t found it productive, so he asked if I had any alternatives. I told him I rarely find good problems by staring at a blank page and trying to “think strategically.” Instead, I act like a sponge. I listen to the stream of day-to-day noise, absorb the problems people are having and let them sit in the back of my mind. Over time, some fade away while connections begin to appear between others that initially seemed unrelated. Eventually, I start to see what’s really slowing people down and what my team or I can do about it. I’ve worked with many engineers who’ve never really tried this. They wait for managers or leads to identify opportunities, then demonstrate their value by solving the hardest assigned problems. That can absolutely lead to promotion. But the projects that have made the biggest impression in my career were the ones where I found and solved an important problem my leaders did not yet realize existed. One caveat: my experience comes mainly from working on infrastructure and developer tools at large companies, on teams where engineers have a lot of bottom-up autonomy to influence their roadmaps. In a more top-down environment, there may simply be less room to work this way. People love talking about the problems they are facing: in meetings, chat threads, presentations and email. They explain why their work is hard, complain about what slows them down and describe what they wish they could do. When something overlaps with my area, I start pulling on the thread. I might ask, “If X existed, would it solve your problem?” or point them at an existing feature in a product I own and ask how much of their use case it covers. Users often ask for a particular solution instead of explaining their root issue. Rather than taking the request at face value, I keep digging until I understand what they are trying to accomplish and why existing products do not work for them. As a natural introvert, this sort of ambient listening works particularly well for me. I don’t need to fill my calendar with speculative meetings just to find ideas; there is already an enormous amount of useful information flowing around me during a normal week. When a problem seems worth exploring, though, I become more active; I need to see how it affects the team’s day-to-day work. I’ll sit with them as they walk me through their workflows and the bugs they’re investigating. When I can, I’ll try working through some of those bugs myself. Seeing the problem firsthand makes it easier to separate what the team actually needs from the solution they asked for. I also seek out people who see more of the organization than I do: those who own critical systems, work across several teams or have particularly deep insight into the work downstream of my team. I’ll arrange a 1:1 or coffee chat and ask about interesting problems they’ve come across. They may have already seen the same issue in several places and started connecting the dots, giving me a head start on patterns I might otherwise have taken much longer to notice. Several times, I’ve been burned by moving too fast. I became excited by a request from a vocal team, built the feature and watched them barely use it. Their priorities had changed, or the request had come from a one-off investigation that no longer mattered. How eager a team was in that moment wasn’t the same as how important the feature was relative to everything else my product needed to support. By hyperfocusing on their request, I lost sight of the bigger picture. That taught me to let potential problems pile up. Listening the way I do leaves me with far more of them than I could possibly solve, and not all deserve action. Most don’t need to turn into projects the first time I hear about them; waiting can be a superpower. Waiting means the same problem might pop up independently in different teams, making it a higher priority to solve. Or problems that look different on the surface might turn out to have the same shape, so I can address several use cases in one shot. Or, as I’ve learned painfully, the requesting team didn’t even care that much in the first place. Instead, I make a mental note and revisit the problem if it comes up again. Other engineers I know write this sort of thing down more systematically. The mechanism is a personal choice: everyone has to figure out what works for them. What matters is keeping unresolved problems around long enough for more evidence to accumulate. Waiting helps me collect evidence, but that alone doesn’t tell me what to build. I still need to work out whether the problems I’ve retained are genuinely related and what, if anything, could address them together. Perfetto, the performance debugging tool I work on, is a good example. It displays recordings of system activity on a timeline made up of rows called “tracks.” Over a couple of years, teams kept asking for small, specific additions to the UI. One wanted a command to keep their preferred tracks pinned to the top of the screen; the next team wanted the same, but for a completely different set of tracks. Others wanted Perfetto to open already zoomed in on a particular part of a recording, or to show a custom aggregation tuned to what they cared about. A few had stopped waiting for us and built elaborate workarounds with bookmarklets. 1 By the time enough of these had piled up, my head was the usual tangle: the requests themselves, the constraints on each and a handful of half-formed solutions. I’ve learned not to force a solution by just sitting at a desk and thinking. Instead, my best untangling happens on long, aimless walks around London, where connections come more easily when I’m not trying to force them. What I eventually realized was that none of these teams really wanted the specific feature they’d asked for. Each wanted to personalize Perfetto for their own workflow without imposing their choices on everyone else. The underlying need wasn’t any one feature but rather the ability to extend the UI. When a connection like that finally clicks, it’s one of the best feelings in the job: several awkward requests collapse into a single idea, and possibilities open up that none of them hinted at on their own. That feeling, though, is exactly when I have to be careful, because a common shape is only a hypothesis and elegance is not evidence. When it happened with extending the UI it turned out to be real, but I’ve been fooled before. In another recent case I was convinced that building a transparent caching system for querying Perfetto traces would solve issues with sharing large traces and repeated queries. It was only as I wrote the RFC and built a prototype that I realized the elegance was a lie: the two problems wanted genuinely different solutions. I reluctantly split the design in two, both halves of which have since shipped. 2 You’d think this would be the moment I start building, but it usually isn’t. How far I go depends on how sure I am that the idea works and that people actually want it. If something is useful and low-risk enough, I act straight away: I send the change and let my manager know. When I’m unsure whether an idea will work or how much effort it will take, I build a throwaway prototype instead; it exposes the failure points and gives me something concrete for others to react to. And when an idea is big but I’m convinced by it, I commit to the full effort: weeks or months of work and the hard yards of building support across other engineers and teams. Through all of it, I’m not only trying to convince other people; I’m also trying to convince myself. Sometimes the honest answer is to stop: if people don’t see the value I do, or we hit a major technical wall, I’d rather drop the idea now than build something no one uses or that becomes a maintenance nightmare. And sometimes it holds up but the timing is wrong, so I park it, ready to spring into action the day it becomes an org priority. When an idea does hold up, I don’t necessarily need to be the person who builds it. I might implement it, someone else on my team might, or it might change what the org focuses on. Finding and shaping the right problem can have an impact even when I don’t own the implementation. The Perfetto extensions idea was worth that full effort. We were already building plugins to modularize the UI, but they weren’t enough: teams had to open source all their plugin code, which wasn’t an option for many internal use cases. So before building anything new, I took the problem and my proposal to my manager, teammates and the client teams. I ended up writing two RFCs, having several 1:1s and giving a couple of talks, refining it as the feedback came in. In the end, I designed and implemented macros as “lightweight extensions”: a way to automate actions in the UI without writing a plugin. Extension servers took the idea further by letting teams share their macros. Instead of implementing every requested feature ourselves, we gave teams ways to adapt Perfetto to their own needs. Dozens of teams inside Google now use macros and extension servers, and several other companies use extension servers internally too. The more often I go through this process, the easier it becomes. When I show genuine interest in someone’s problem, ask useful questions or help solve it, they remember. They start coming to me earlier and bring me into conversations with other people facing related issues. That gives me a wider view of what is happening across the organization, making it easier to spot patterns and build things people actually need. Solving one of those problems brings me into more conversations, and the loop continues. Those successes build the kind of trust that comes from long-term stewardship . Early on, I had to turn many of these ideas into something real myself to prove that my judgment was sound. Over time, my manager and org gave more weight to my assessment of what mattered. That allowed me to influence the roadmap without needing to own every project. This differs from the idea that becoming a staff engineer means replacing technical work with meetings and coordination. For me, conversations are inputs into what I build, not the end result. That is what I wanted my mentee to understand: finding problems worth solving isn’t separate from the rest of the job. It comes from staying engaged with people’s work long enough to see what no single request can show you. These workarounds used bookmarklets to run JavaScript against Perfetto’s internal UI APIs.  ↩︎ The original proposal was to use a transparent cache for repeated queries and faster reopening of large traces. As I worked through it, I realized repeated queries were better served by keeping sessions warm in memory, whereas reopening was better served by explicitly exporting a trace into a format designed to load quickly. A transparent disk cache could also retain multi-gigabyte files without the user realizing and would need a new system to manage their lifetime. The proposal was ultimately replaced by warm sessions and streaming table export .  ↩︎ These workarounds used bookmarklets to run JavaScript against Perfetto’s internal UI APIs.  ↩︎ The original proposal was to use a transparent cache for repeated queries and faster reopening of large traces. As I worked through it, I realized repeated queries were better served by keeping sessions warm in memory, whereas reopening was better served by explicitly exporting a trace into a format designed to load quickly. A transparent disk cache could also retain multi-gigabyte files without the user realizing and would need a new system to manage their lifetime. The proposal was ultimately replaced by warm sessions and streaming table export .  ↩︎

0 views

Premium: The Hater’s Guide To Oracle (Part 2)

Good morning premium subscribers! As ever, please ping me at [email protected] if you have any questions. Oracle has one of the strongest mythologies in the tech industry. Ask a regular person and they’ll tell you that it’s “incredibly profitable” and “growing fast,” that it’s “unstoppable,” and that Larry Ellison has the mandate of heaven with regard to the continual sales of software and hardware related to databases and AI. And those people are completely and utterly wrong. The original title of this article was “Is Oracle Dying?” because I assume, when I took a deeper look, that there’d be some sort of debate , some sort of bull case for a decades-old quasi-hyperscaler run by one of the more nakedly-evil CEOs in the history of tech. I assumed — incorrectly, I might add — that Oracle as a business was doing fine other than the ridiculous commitments it made to support the whims of Sam Altman and OpenAI via deals that I believed (and still believe) will kill Oracle . Except it turns out that Oracle has already been on a death spiral for the best part of a decade (if not longer) and has only survived this long by screwing its customers, taking on masses of debt, and — most importantly — more than $85 billion in acquisitions over the last 23 years. Pretty much every major product line outside of databases is a hodge-podge of other people’s innovation stapled together with a legendary contempt for the customer . These acquisitions (and continual price increases ) are the only thing keeping the reaper from Oracle’s door other than margin-destroying GPUs . And that’s why Oracle’s revenue looks like this : After April 2009’s $5.7 billion acquisition of Sun Microsystems , Oracle’s revenues barely kept pace with inflation until December 2021’s $28.3 billion acquisition of Cerner allowed it to create Oracle Health , adding about $6 billion in annual revenue that had 40% lower margins (about 21.7%) than Oracle’s other businesses , though Oracle immediately started closing offices and brutal layoffs to try and bring them up. And as I mentioned above, Oracle’s other plan was to sink a little over $99 billion in capital expenditures since the middle of calendar year 2020 into AI GPUs.  Anyway, let’s see what that’s done to margins - OH MY GOD ! Oracle is a decades-long mission to keep reapplying lipstick to a pig. Billions of dollars of acquisitions have, for the most part, only succeeded in keeping the company’s revenue growth from going negative, and as noted by forensic accountant Howard M. Schilit , this is one of the most well-documented cases of accounting shenanigans being used to cover up that a business is in decline. Today’s newsletter is a sequel to the Hater’s Guide To Oracle , where I told the sordid tale of how Larry Ellison grew a massive, lucrative business out of a database business that one reporter once told me was a “ law firm with a database company attached ,” an Enterprise Resource Planning (ERP) product that competes with SAP to create the most-annoying way to run a large company, and a business built around licensing Java that exists mostly to email people and say “you need to pay us for Java or we’ll sue you.”  Then, as I’ve mentioned, there’s Oracle’s cloud infrastructure business, a decade-old also-ran that was meant to compete with Microsoft Azure and Amazon Web Services, but only managed to catch up following the advent of AI GPUs and a movement where all it took to party was buying billions of GPUs and saying “gosh darn, we love AI.” I originally started drafting this as a much tamer piece where I’d ask whether Oracle was dying, but as my editor and I started digging into the research, it became obvious that not only is Oracle dying , it’s been dying for years , kept alive through decades of acquisitions and a desperate and dangerous commitment to generative AI. And AI, I believe, will be what eventually kills Oracle dead.  In the past, all Oracle had to do to survive was buy somebody else’s company and replace its flagging revenues with theirs, turning the screws on their customers and laying off as many people as necessary to balance the books. While chaotic and decaying, Oracle’s empire has kept above water by never overextending itself, always keeping a positive free cashflow , and generally avoiding buying into industry hype cycles outside of whatever SaaS vehicle might potentially plug the gap in its earnings.  Yet with AI, Oracle broke its long-standing trend of letting someone else figure out the innovation, choosing instead to build its own cloud infrastructure, spending more in capex in its last fiscal year ( $55.6 billion ) than it did in the previous nine years combined ($50.4 billion), tripling its debt from $56.91 billion in FY2017 to $167.4 billion in FY2026, a year that ended with its free cashflow sitting at negative $23.69 billion.   For comparison, Oracle has had positive free cashflow every single year since 2001, including the Great Financial Crisis and COVID. Oracle has doomed itself with its commitment to the AI bubble. It has committed to building 7.1GW of data center capacity for one company — OpenAI — as part of a $300 billion, five-year-long contract that requires it to build an impossible amount of capacity in an impossible period of time for a client that could never afford the $70 billion or more in annual costs to make any of it worth it.  Today I am going to talk trash on what I consider to be one of the single-worst companies in the tech industry that’s survived only through financial engineering and never, ever overextending itself.  With revenue plateauing and customers in revolt, Oracle’s future already looked murky, but with the power of AI — and $95 billion in FY2027 capex — it’s becoming increasingly clear that this may be Larry Ellison’s last dance with Silicon Valley. This is the Hater’s Guide To Oracle Part 2, or AIpoaclypse Now.

0 views
Martin Fowler 1 months ago

The Archaeologist’s Copilot

When people think of legacy modernization, most folks aren't imagining the target environment will be Java 8. But this was the challenge facing Nik Malykhin when he needed to run a Java 1.5 codebase on today's hardware. His early use of LLMs gave plausible answers that did not hold up in the codebase. Progress came when he grounded the process in evidence, using AI to support analysis, validation in a stable Docker environment, and gradual refactoring protected by tests. The main takeaway is practical: AI was most useful when constrained by evidence, clear roles, and a step-by-step modernization strategy.

0 views
Max Bernstein 1 months ago

Travel notes: PLDI Boulder

I had another excellent PLDI this past June. It was my fourth 1 . I continued to meet new people and learn new things! Overall: I got to meet a lot of new people, which was exciting. I had some good chats about research. I asked a question at a talk! I got to show Aaron and Jacob PLDI and see them enjoy it. I missed hanging out with CF Bolz-Tereick and Chris Fallin, the usual suspects at conferences I attend. I’m looking forward to next year. This post is more about the conference than the town of Boulder (unlike the last PLDI post about Seoul) because I didn’t do much Boulder exploring. I got in late on Sunday. Then I had to take a long car ride from Denver airport to Boulder. I don’t think I had ever flown into Denver with intent to go to Boulder before so it was a bit of a surprise. Jacob offered to have a late dinner with me so we had a tasty meal at Gaia Masala and Burger. Shout out to Harry, our server. Monday was a workshop day. I signed up for EGRAPHS and mostly stayed in that workshop. People kept throwing around the term “Knuth-Bendix”, as they have been for several years, and only in one of these workshop talks did someone explain it in a way that made any sense at all. It seems kind of like equality saturation but for the rewrite rules themselves—no actual expression graphs involved. I DMed Phil this sketchy explanation during a talk to get his response and I got to watch him cock his head and think about it in real time. At lunch I met Qiantan Hong and we got to talking about Common Lisp and its object system, CLOS. Seems like a combination of ahead-of-time compilation and multiple dispatch is really tricky. I had dinner with Aaron at Postino and then wandered into a bunch of people staying at the conference hotel chatting in the lobby. Ben Titzer said “fix my subtyping bug”, which I interpreted as him saying hi. I ended up just planting myself at the table as a bunch of interesting people cycled through: Jared Roesch, Mae Milano, Hila Peleg, Russel Arbore. It was a late evening. Back to the workshops! But late because of aforementioned late evening. I saw Vadym’s talk about Remora. I only understood about 40% of it but it was good to catch up. I hadn’t seen him since leaving Northeastern. Around a break time I joined a little cluster of people talking about e-graphs and I guessed asked enough basic questions that Pavel convinced Max Willsey to run a “BYOEG” (build your own e-graph) tutorial. The structure was as follows: Max would instruct Jacob as to what kind of thing to build next but not be prescriptive about exactly how to build it and not look at Jacob’s screen. The rest of us would sit around a table and try to follow along as best we could. I hear Pavel has a blog post about this experience coming soon… I saw Slava Pestov walking around and introduced myself because we keep liking one others’ bad jokes on Mastodon. We ended up getting dinner with Aaron and Jacob that night at Leaf. We learned a lot about monoids, Knuth-Bendix (!), Factor, and Swift. Slava volunteered to do a similar follow-along “BYOKBC” (build your own Knuth-Bendix completion) tutorial the next day. First day of the conference! I was walking into the hotel in the morning and I had made it about three feet onto the property when Alexa VanHattum, who was going the opposite direction, convinced me to instead get coffee elsewhere. We had a nice catch up and I got to hear about what teaching is like these days. Lunch was fun. I got to do another round of “ambush person whose research I admire” and plopped down with Ben and Christian Wimmer. I’d spent a lot of time struggling with Christian’s papers on linear scan, then convinced him to chat about register allocation with us on a video call a couple of months ago. We continued some of that at lunch but then I (kind of accidentally kind of on purpose) got Ben started talking about Sea of Nodes and how it is and is not different between Java and (for example stand-in for dynamic languages) JavaScript. Apparently he is thinking about a similar thing that he is calling Sea of Variables. We talked about inlining challenges and how to infuse profiles with call context, which can be a challenge. I feel more inspired to get type-based alias analysis working in ZJIT. I tracked down Christian later in the courtyard and got to hear about what he’s working on these days. I know very little about ML compilers and ML hardware and things like that so hearing about the challenges was neat. Yannis Smaragdakis joined our little standing table chat and we got to learn about Datalog. Because I had previously written about linear scan register allocation and about liveness analysis with Datalog, I goaded him into pairing with me on writing a full linear scan implementation in Datalog. This ended up taking the rest of the evening and several beers and then a lot of the next day! And after the first bit of code I did not manage to contribute very much at all. I met Hannah Gommerstadt and we got to chatting about bikes and formal methods (separately). Slava walked by and I got to introduce them. Then Jacob too. I continued pairing with Yannis but remained really lost. The only thing I think I contributed was some familiarity with the core algorithm, which he had only really seen in passing before. Eventually he got it fully working, but it needed some deep trickery. More on this soon in its own post. I saw a talk about versioned e-graphs and that got me wondering if their implementation can be used as a persistent e-graph or even just persistent union-find. Sometimes you want to do backtracking, or have undo-redo in your compiler. Then I went to a talk about streaming byte-pair encoding (BPE). BPE is hard to do streaming because it definitionally requires looking over the whole input string. They did some neat trickery to find boundaries in the string that demarcate regions that don’t interfere with one another and thereby tokenize on-the-fly. I didn’t understand it fully but I asked my only question of the conference, which was if this could also be used to implement BPE in parallel. Seems the answer is “maybe” so I should probably reach out and ask further. Slava started showing me and Jacob and Aaron how to implement Knuth-Bendix completion for strings. I had a lot of tiny little bugs which slowed progress. Such is life. The banquet and awards ceremony started so we called it a night and went off to eat dinner. They had good lentils. I ran into Thalia Archibald and John Regehr and we talked about (really, they talked about and I tried to learn something) what it might mean to either port Alive2 to another compiler than LLVM, or build “Alive3”, or “Mini-Alive” for some other IR. John suggested fuzzing the hell out of the thing first, then doing something more formal later… especially if it’s a dynamic language IR where a lot of the opcodes end up being “function call that can do anything”. I had a nice chat with Steven Holtzen and Zach Tatlock about research and grad student life. I got some good advice. I meant to talk to Zach about this thing we keep occasionally chatting about that I call “the big e-graph in the sky”. I talked a bit about it to Max Willsey and he had some good probing questions about what would be slow, challenging, or somehow undefined given my problem statement. I continued struggling to implement Knuth-Bendix with significant assistance from Slava and I think eventually got something working. I had a really nasty bug due to string slicing semantics 2 in Ruby. Aaron went off to learn about deep immutability in Python and then got to chatting with the authors of the paper. We got to compare notes about language and language implementation challenges. It’s been a long time since I was in Python-land. Aaron and Slava and I got enchiladas for dinner. I had no reading material for the flight so I went downtown, intending to buy one book, but got too many books. They barely fit into my bag for the flight home. I started reading Anathem for the second time. It holds up. It’s a damn good book. Actually, it might be my fifth. I just remembered that I attended PLMW in 2020 and also watched a few online talks at wild hours from my living room.  ↩ The semantics in Ruby are probably globally reasonable but did not fit the thing I was trying to do: if we have two strings and , we want to find the index at which they start to overlap, . Then we want to grab the bit of that is to the left of . I had initially written that as . However, if and overlap at the start of , is 0. This generates the range , which means we’ll slice until the end of . Not what we want. Instead, the fix in the commit shows how I had to add a special slice function called that handles the 0 case.  ↩ Actually, it might be my fifth. I just remembered that I attended PLMW in 2020 and also watched a few online talks at wild hours from my living room.  ↩ The semantics in Ruby are probably globally reasonable but did not fit the thing I was trying to do: if we have two strings and , we want to find the index at which they start to overlap, . Then we want to grab the bit of that is to the left of . I had initially written that as . However, if and overlap at the start of , is 0. This generates the range , which means we’ll slice until the end of . Not what we want. Instead, the fix in the commit shows how I had to add a special slice function called that handles the 0 case.  ↩

0 views
Jason Scheirer 1 months ago

A Revival of Sorts: Getting my iPod Classic 6G Working Again

I’ve been very happy with my Y1 MP3 player over the past 9 or so months. I take it with me everywhere! It’s a companion on my commute, it’s a focus tool in my open office, it’s a way to have a single-purpose device that doesn’t have the distractions of my glass Everything Rectangle and, as the phone ages, a way to mitigate its now-horrible battery life by using a different device with a different battery. As God confounded the language and scattered the people building the tower of Babel, I have confounded the functionality and scattered the responsibilities of the apps on my iPhone. My wife brought up a point that is completely fair: why am I using this $60 piece of crap when she, through great sacrifice, bought me a top-of-the-line iPod Classic 160GB for the same purpose? Sure, that was in 2012, but it was expensive . It’s still worth $350+ today, right? So what the hell, I dug it out of my Closet of Cables and Mystery. Plugged it in. Battery charged. It booted. My music was still in it, last addition to the library wa 2014. Fantastic! I bought a protective case, some new 30-pin USB cables because the ones I had remaining were all frayed and kind of scary, and I got ready to swap the Y1 with the iPod for a while as an experiment. Then my first hurdle: I wanted to add some songs to it. I know Rhythmbox , my player of choice 1 , has an iPod plugin on its list of installed plugins. I plug the iPod in, it shows up! Hooray! I try to drag music onto it: no dice. Checking I see some very threatening notices that HFS+ with journaling is not supported by Linux at all . So I know on Mac it’s a simple command line call to turn journaling off on a volume so it’s probably a trivial process, but I have no working personal Apple desktop machines. Have no fear: I found a chunk of unvetted C that directly alters the raw filesystem to do it for me on Linux! Boom! We’re in business! Back to Rhythmbox. Drag the music I want over to the iPod. It copies! Bingo! Only: no bingo! I disconnect the iPod and it says ’no music.’ The music is on the device, but the iPod’s music database got clobbered. Well crap. So now I know Gtkpod is purpose built for this. Apparently the iPod Rhythmbox plugin isn’t any good on these models, so let’s try that. No dice. It repeatedly hangs, crashes, and when it does work it still fails to correctly update the database. Still ’no music.' Maybe this is all because it’s still HFS+ and not FAT? It seems like most tools assume you’ve liberated your iPod and you’re using it in Windows mode, not Mac mode. So I attempt to wipe the drive, but can’t for the life of me figure out how to do it correctly with Gtkpod or just plain old partitioning tools. Looks like I need to restore the hardware from iTunes for this route. What about Rockbox ? I use it on my Y1. The annoying thing is that I have to manually update the database on the actual device, whereas the typical iTunes stock experience is one that updates the database iteratively as a matter of course of adding music. But the trade-off is no more struggling with Gtkpod and friends, which is higher friction than the drag-and-drop experience of putting music on my Y1 anyway. And I saw this totally cool skin on Reddit I want to try ! I already have the Rockbox utility on my machine from installing it onto my Y1. It sees my iPod but dies on an SSL handshake talking to rockbox.org while downloading resources. I don’t remember this happening last time I ran this. I downloaded and ran the utility on another Linux machine and got the same result. I gave up about 45 minutes into building the tool myself from source. Now I need a Windows machine to use iTunes in Windows to reformat the iPod. I have a debloated Win11 VM in Gnome Boxes, I fire that up and go in to iTunes, I plug in the iPod, then I go to set up USB forwarding so the VM can do its magic and – “USB Forwarding is Not Supported in the Flatpak version of Boxes.” So I uninstall the Flatpak and migrate my disk images from to somewhere less Flatpak-specific and install the dnf version of Gnome Boxes. I migrate the machine over, set up forwarding, everything seems to be working. Only USB forwarding forgets the device when it disconnects and I have to reconnect multiple times. It also doesn’t see the device when it’s in that raw flash mode, so it can’t forward to install the iPod firmware. This is a dead end. Okay, so I have one Windows machine in my house: my kid’s 2013 Intel Macbook with Boot Camp and a debloated copy of Win10 we solely use to play Minecraft Java together with. Only ever since I set up a local server with GeyserMC and Floodgate we’ve been playing mixed me-on-Java/him-on-iPad-or-Switch-Bedrock so the laptop is mostly neglected. So I install iTunes and wipe the iPod. Takes awhile, because I have to install a cascading series of drivers, but it eventually works. The firmware was the latest for the Classic, released 2009. Then I remember that 18 year old bit of early enshittification of iTunes: the iPod can’t simply be its own library you add/remove items from. I was falling out of love with Apple about that long ago , and I had forgotten how low and slow we’ve been dealing with the world of You Will Own Nothing enshittification that’s been inflicted on us. No wonder we’re so complicit, we’re pushing a quarter century of Everything Rental now. So to do iTunes proper I’d need enough storage on this laptop to hold the music in my library on it, be logged in, and sync a selection of it to the iPod. I remember this now: they made life harder and worse on purpose. And now we have Spotify, where we never had freedom or affordances at all. I remember thinking what an incredible act of charity it was that Spotify let your have an offline playlist on your device. I would have expected offline first as a matter of course in prior hardware/software cycles. Rhythmbox and Gtkpod still don’t sync correctly. Same database issues, so nothing I’d done with wiping the iPod had fixed the fundamental first issue. So I install the Rockbox utility on the Windows machine. I have to install some additional Windows components to get it to load, but it works. I flash the iPod. It doesn’t boot. I flash it again. It boots. Hell yes. And I have my cool theme. So I drag music over. 16000 tracks to start, takes 2 hours to copy. HDDs are slow . Afterward I have to manually update my database from Rockbox, which takes hours . I fall asleep as it runs. I can hear the physical spinning platters. It’s a very strange experience having a device with a real life magnetic disc hard drive again. The future we occupy today is strange in the UX of the iPod and its software feels modern enough but small aspects like an HDD feel anachronistic. The Rockbox experience is a lot nicer on the hardware it was designed for than the crappy Rockbox-in-emulation on an Android device that has absolutely no business whatsoever claiming it can run Android. It is responsive, it doesn’t crash, all the plugins work, etc. Next rabbit hole is investigating battery/storage upgrades. There are cheap and expensive options, I need to go through them. As is my wont, I do not need bluetooth on anything I own, but a modern USB-C connector might be nice? Do I want to go the SD card route or a proper SSD? That is for another time. Anyway, no normal person would inflict this experience on themselves willingly, and would likely give up at some point close to the beginning. It is a reminder that much like if you stay very quiet near a playing iPod you can hear the whir and rattle of the HDD. If you stand very quietly near me you can hear the fluttering and tapping of dozens of moths smashing their bodies against the inside of my skull in the space where a brain should be. I am not aware of any other MP3 player that can handle large music libraries this well and still have a presentable UI. TUIs usually suck, “new” apps are all super slow because of Wirth’s Law.  ↩︎ I am not aware of any other MP3 player that can handle large music libraries this well and still have a presentable UI. TUIs usually suck, “new” apps are all super slow because of Wirth’s Law.  ↩︎

0 views
Jack Vanlightly 2 months ago

Benchmarking Hardwood 1.0 on a Threadripper 9980X

Hardwood is a minimal-dependency Java library for reading Parquet files. It currently has row-reader and columnar-reader APIs, with Parquet writing planned for the future. Gunnar Morling, Hardwood’s author, published some initial benchmarks in the v1.0 announcement, comparing Hardwood’s row and column readers against Parquet Java . Those benchmarks measured read speed against already-downloaded Parquet files.  Gunnar’s benchmarks ran on an m7i.2xlarge, with 8 vCPUs / 4 physical cores. Each test used three variants: Hardwood with decoder threads = , which equals 8 Hardwood pinned to one CPU thread with taskset Parquet Java, single-threaded I was curious how the same benchmarks would look on my Threadripper 9980X: 64 cores / 128 threads, with 256 GB ECC DDR5. I modified Gunnar’s benchmark code to also test Hardwood with fixed decoder-thread counts: 1, 4, and 8. That gives the following Threadripper variants: Hardwood, unpinned, decoder threads = 128 (available processors) Hardwood, unpinned, decoder threads = 8 Hardwood, unpinned, decoder threads = 4 Hardwood, unpinned, decoder threads = 1 Hardwood pinned to one CPU thread (taskset) Parquet Java, single-threaded One important detail: decoder threads = 1 is not the same as the pinned 1-core test. With decoder threads = 1, the main thread can run on another core. The pinned test constrains the whole process to one logical CPU which is the closest we can get for like-for-like comparison to single-threaded Parquet Java. This benchmark reads all columns of the dataset 48M row dataset. m7i.2xlarge Fig 1: m7i.2xlarge, Hardwood (all cores) 16.5M/s, Hardwood pinned 1-core 3.9M/s, Parquet Java (single-threaded) 3.3M/s Threadripper 9980X Fig 2: Threadripper, Hardwood (all cores) 43.4M/s, Hardwood dt=8 48.4M/s, Hardwood dt=4 44.9M/s, Hardwood dt=1 15.5.9M/s, Hardwood pinned 1-core 11.0M/s, Parquet Java (single-threaded) 5.8M/s A few things stand out: The Threadripper is much faster in the single-core cases than the m7i.2xlarge. Hardwood pinned to one core reaches 11.0M rows/s (with some runs reaching over 12M), versus 3.9M rows/s on the m7i.2xlarge. Generally about 3x faster. Hardwood’s single-core result on the Threadripper is also much stronger relative to Parquet Java. On the m7i.2xlarge, Hardwood 1-core is only modestly ahead of Parquet Java: 3.9M rows/s versus 3.3M rows/s. On the Threadripper, Hardwood 1-core is almost 2x faster: 11.0M rows/s versus 5.8M rows/s. More decoder threads help, but only up to a point. The best result here is 8 decoder threads, at 48.4M rows/s. Four decoder threads are close behind at 44.9M rows/s. The default availableProcessors() setting, which gives 128 decoder threads on this machine, is slower than both, which is not surprising. This benchmark reads all rows of the dataset 48M row dataset. It has two variants: Indexed (positional) columns, i.e. r.getLong(3) Named-columns, i.e. r.getLong("passenger_count") m7i.2xlarge Fig 3: m7i.2xlarge, Indexed-columns, Hardwood (all cores) 14.9M/s, Hardwood 1-core 4.4M/s, Parquet Java (single-threaded) 1.4M/s. Named-columns, Hardwood (all cores) 2.8M/s, Hardwood 1-core 1.9M/s, Parquet Java (single-threaded) 1.4M/s Threadripper 9980X Fig 4: Threadripper, indexed (positional) columns, Hardwood (all cores) 33.4M/s, Hardwood dt=8 36.1M/s, Hardwood dt=4 34.9M/s, Hardwood dt=1 14.4M/s, Hardwood pinned 1-core 10.8M/s, Parquet Java (single-threaded) 3M/s. Named columns, Hardwood (all cores) 5.9M/s, Hardwood dt=8 5.8M/s, Hardwood dt=4 5.9M/s, Hardwood dt=1 5.7M/s, Hardwood pinned 1-core 4.3M/s, Parquet Java (single-threaded) 2.6M/s The indexed-column row reader shows the same basic pattern as the columnar full scan. Hardwood is much faster than Parquet Java even in the pinned 1-core case: 10.8M rows/s versus 3.0M rows/s. The best multi-threaded result is again with 8 decoder threads, at 36.1M rows/s, with 4 decoder threads close behind. The named-column reader is different. Hardwood is still ahead of Parquet Java, but it does not meaningfully scale with decoder threads. The unpinned Hardwood results are all around 5.7M to 5.9M rows/s, regardless of whether the benchmark uses 1, 4, 8, or 128 decoder threads. If you want high throughput, use the indexed-column approach. This test generates data with 4 columns and 50M rows where event_time is perfectly ordered. The filter is event_time < threshold, and therefore the file is therefore clustered by the predicate column, relying on Parquet row-group/page/column statistics. The file contains no bloom filters as Hardwood does not support those yet). There are two variants: selective: event_time < 2,500,000 (about 5% pass) matchAll:  event_time < 50,000,000  (100% pass) The test measures the time for the filtered scan to complete. m7i.2xlarge Fig 5: Selective (5%), Hardwood (all cores) 12.9 ms, Hardwood pinned 1-core 53.8 ms, Parquet Java (single-threaded) 173 ms. Match-all (100%), Hardwood (all cores) 222 ms, Hardwood pinned 1-core 983 ms, Parquet Java (single-threaded) 3157 ms Threadripper Fig 6: Selective (5%), Hardwood (all cores) 10.5 ms, Hardwood dt=8 5.1 ms, Hardwood dt=4 7.2 ms, Hardwood dt=1 24.1 ms, Hardwood pinned 1-core 32.0 ms, Parquet Java (single-threaded) 97.9 ms. Match-all (100%), Hardwood (all cores) 95.0 ms, Hardwood dt=8 80.4 ms, Hardwood dt=4 122 ms, Hardwood dt=1 425 ms, Hardwood pinned 1-core 537 ms, Parquet Java (single-threaded) 1777 ms. The relative shape is similar to the m7i.2xlarge results, but the Threadripper is much faster. In the single-core comparison, Hardwood is about 3x faster than Parquet Java in both cases: 32.0 ms versus 97.9 ms for the selective scan, and 537 ms versus 1777 ms for the match-all scan. With multiple decoder threads, Hardwood is much faster again. The best Threadripper result is 8 decoder threads: 5.1 ms for the selective scan and 80.4 ms for the match-all scan. I hacked on Gunnar’s benchmark code to add some more test cases. Fig 7: Threadripper. Hardwood (all cores) 192M/s, Hardwood dt=8 215M/s, Hardwood dt=4 119M/s, Hardwood dt=1 30.9M/s, Hardwood pinned 1-core 26.8M/s, Parquet Java (single-threaded) 13M/s This is one of the clearest decoder thread scaling results. Hardwood 1-core is about 2x faster than Parquet Java, and 8 decoder threads reach 215M rows/s (14.8x faster than Parquet Java). Unlike the full-scan benchmarks, there is a large gap between 4 and 8 decoder threads here. Fig 8: Threadripper. Hardwood (all cores) 118M/s, Hardwood dt=8 120M/s, Hardwood dt=4 119M/s, Hardwood dt=1 116M/s, Hardwood pinned 1-core 50.1M/s, Parquet Java (single-threaded) 87.1M/s. The string column seems to change the performance profile. This case behaves differently, with Parquet Java winning compared to the pinned 1-logical-core Hardwood test. More than one decoder thread does not help: the unpinned Hardwood results are all between 116M and 120M rows/s. I haven’t profiled this so I can’t explain the result. In this test, we use the predicate , which matches 500324 rows (1%) of the deterministically generated 50M row dataset. This time the files are not clustered by the predicate but the total number of matching rows is 5x smaller than the filter test from earlier. Fig 9: Threadripper. Hardwood (all cores) 141 ms, Hardwood dt=8 135 ms, Hardwood dt=4 131 ms, Hardwood dt=1 129 ms, Hardwood pinned 1-core 291 ms, Parquet Java (single-threaded) 2522 ms. Hardwood is far ahead of Parquet Java here. Even the pinned 1-core Hardwood result is about 8.7x faster than Parquet Java. I ran the benchmark with the flag, which verifies that each test returns the same data, and it passed, so the result looks legit. Decoder threads do not help much in this test. The unpinned Hardwood results are all between 129 ms and 141 ms. That suggests this benchmark is limited by something other than parallel decoding. The Threadripper 9980X is a workstation, not a server. It has a higher clock speed but lower memory bandwidth that its EPYC server counterparts. I imagine you’d see lower performance numbers on the EPYCs for these tests, but the EPYCs would easily beat the Threadripper on the amount of parallel Hardwood workloads due to the 12-memory lanes compared to the Threadripper’s 4 lanes. Thinking about memory bandwidth, I decided to see how Hardwood scales across instances, where each benchmark process was pinned to 4 physical cores and given 4 decoder threads. Fig 10. Threadripper. 1 process (4 physical cores) 26.1M/s, 2 processes (8 physical cores) 47.5M/s, 4 processes (16 physical cores) 79.2M/s, 8 processes (24 physical cores) 81.2M/s, 12 processes (48 physical cores) 79.6M/s, 16 processes (64 physical cores) 75.1M/s. We reached close to this workstation’s memory bandwidth limit at 4 processes on 16 physical cores, and after that there was little benefit or even reduced throughput as efficiency dropped. Fig 11. The memory bandwidth topped out in the 4th test (8 processes, 32 physical cores) The Instructions Per Cycle (IPC) dropped further and further, signalling the reduced efficiency. Fig 12. The IPC drops as we add more and more parallel benchmark instances. And, we became increasingly memory bound. Fig 13. AMD uProf’s top-down estimate of how much CPU pipeline capacity is lost because the backend is waiting on the memory subsystem The EPYC 9575F single socket has 614 GB/s (theoretical) and the dual-socket up to 1.2 TB/s (theoretical) bandwidth, compared to just 205 GB/s theoretical for my workstation (though the max actual I’ve measured is 170 GB/s). So the EPYC would have blown the socks off my workstation. I’m including this as a reminder that benchmarks don’t usually measure things like memory bandwidth saturation under high parallel load. On my Threadipper 9980X, Hardwood’s single-core performance looks strong against Parquet Java across most of these benchmarks. In the full columnar scan, pinned 1-core Hardwood is almost 2x faster than Parquet Java. This contrasted to the m7i.2xlarge where Hardwood only saw a modest single-core advantage over Parquet Java for this specific test. Thus a reminder that your mileage may vary. In the positional row-reader scan, Hardwood was about 3.6x faster than Parquet Java, and in the filtered scans, about 3x faster. The custom predicate benchmark shows an even larger gap.  Hardwood’s multi-threaded performance is also strong up to a certain decoder-thread count (which is workload-hardware-dependent). On this Threadripper, 4 or 8 decoder threads were usually enough. The default value gives a ridiculous 128 decoder threads which was unsurprisingly less efficient than 8. The main exceptions to decoder thread scaling were the named-column row reader, the string column subset, and the custom predicate benchmark. Those cases showed little or no benefit from increasing decoder threads, even when Hardwood still beat Parquet Java overall. I initially wondered if the strong single-thread performance compared to the m7i.2xlarge was the Threadripper’s strong AVX-512 support, but after profiling it with AMDuProfPcm, it turned out that this was not the case. I also tested out enabling the Vector API, but it made no difference to the performance. If any performance engineers out there want a fun project, then my feeling is that Hardwood still leaves a lot on the table for optimizing. It could be a fun project. I finish by saying this benchmarking was for fun on a workstation. So these results are not generalizable but they do correspond to the m7i.2xlarge results (just better). They are mostly useful as a directional look at how Hardwood behaves on a high-core-count workstation. You need to benchmark your own use case, on your chosen hardware. Hardwood with decoder threads = , which equals 8 Hardwood pinned to one CPU thread with taskset Parquet Java, single-threaded Hardwood, unpinned, decoder threads = 128 (available processors) Hardwood, unpinned, decoder threads = 8 Hardwood, unpinned, decoder threads = 4 Hardwood, unpinned, decoder threads = 1 Hardwood pinned to one CPU thread (taskset) Parquet Java, single-threaded The Threadripper is much faster in the single-core cases than the m7i.2xlarge. Hardwood pinned to one core reaches 11.0M rows/s (with some runs reaching over 12M), versus 3.9M rows/s on the m7i.2xlarge. Generally about 3x faster. Hardwood’s single-core result on the Threadripper is also much stronger relative to Parquet Java. On the m7i.2xlarge, Hardwood 1-core is only modestly ahead of Parquet Java: 3.9M rows/s versus 3.3M rows/s. On the Threadripper, Hardwood 1-core is almost 2x faster: 11.0M rows/s versus 5.8M rows/s. More decoder threads help, but only up to a point. The best result here is 8 decoder threads, at 48.4M rows/s. Four decoder threads are close behind at 44.9M rows/s. The default availableProcessors() setting, which gives 128 decoder threads on this machine, is slower than both, which is not surprising. Indexed (positional) columns, i.e. r.getLong(3) Named-columns, i.e. r.getLong("passenger_count") selective: event_time < 2,500,000 (about 5% pass) matchAll:  event_time < 50,000,000  (100% pass)

0 views
Brain Baking 2 months ago

Postcard Teas: A Few Impressions

For almost ten years now, we’ve sworn by Mariage Frères when it comes to shopping for high quality loose tea leaves. The nearest shop, however, is in Lille, which is almost three hours away. Their webshop is crude and doesn’t allow for a taste session before buying hence we did buy our fair amount of misses. Yet we remained faithful: the few times that we diverged from the brand ended up in a disappointment. And then I saw someone claiming that London-based Postcard Teas is “even better than Mariage Frères”. My initial reaction to that was “impossible”. I secretly made a note in my journal regardless. When our stock started to dwindle, I dug up that note and said to myself: what the heck, let’s do something crazy and order elsewhere. Postcard Teas is a small shop in London that sells specialty teas by importing directly from the growers. Their unique selling point is hinted in the name: these growers only have a few acres in which they aim to grow the best quality possible. The result is only a few kilograms of yield each year, yet the average price remains acceptable. Each bag of tea you order comes wich a lovely postcard and piece of art depicting a work from the country of origin. I have no idea where Mariage Frères’s tea comes from and love the fact that with Postcard Teas, this knowledge is accessible—even evident. Besides the location and yield, the back of the postcard even contains the grower’s name and a tidbit of bio. Watch China Minutes’ visit to the small shop to breathe in the atmosphere. Meanwhile, I’ll go prepare myself a cup of their summer Darjeeling. Still interested? Great! Yet people outside UK should be warned as shipping comes with a hefty taxation at the border I didn’t mentally prepare for… Just take that into account when you’re browsing their webshop—and don’t forget to compare prices with your usual supplier per , not the deceptive . With that being said, here are some impressions of the teas we tried out: Golden Darjeeling A lovely dark red tea that goes down very well without being too strong. I usually buy first flush/spring Darjeeling and kind of wish I did here as well as that’s usually milder, but this summer Darjeeling is excellent, even if you accidentally let it steep for too long. Contrary to its spring variant, it also handles heat very well, so I usually set it with boiling water. A pure Darjeeling is usually my go-to in the morning or even right after lunch. This black tea is less black than the cheap green powdered teas bought in supermarkets. 4 out of 5 Blounts—Great. Gianfranco’s Earl Grey The first thing that came to mind after opening the bag is: I hope the strong scent does not reflect in the taste. And luckily, it doesn’t. Mariage Frères’ Roi Des Earl Grey is more purgent, up to the point that they might have overdone it. Gianfranco’s bergamots in Calabria pair very well with Kerala’s small Darjeeling tea farms. The structure and colour of the tea is very similar to the previous one, the Golden Darjeerling. This is because Postcard Teas blends both flavours in their shop in London giving them the advantage of carefully choosing both ingredients. Since I love a good Darjeerling, it’s impossible to resist. I do still prefer Mariage Frères’ more daring lavender Early Grey. 4 out of 5 Blounts—Great. New Assam Chai This is the first tea from Postcard Teas that I like less the more I drink it. The culprit? The particular blend of spices: way too much green cardamon. Cardamon is a spice with a minty freshness that easily overpowers everything else, as it indeed does here. Also, the Assam is cut in finer pieces than I’d wish making this brew very dark and strong. I recognise the need for a strong tea to counterbalance the just as strong spices here but for me it was just a bit too much. Adding lemon and honey helps but only up to a point. I know you’re supposed to drop a few splashes of milk in it but I’m not British nor Indian so I don’t. 2 out of 5 Blounts—Mediocre. This is a traditional curled green tea from Japan called a “kamairicha” tea: instead of steaming the tea to stop the oxidation, kamairicha is roasted in a dry pan. Contrary to most Japanese teas such as Sencha, the typical bitter taste is gone because of this process. Mr Ogasa’s farm in Gokase is only 14 acres big. I’m not a huge Japanese tea expert but I do like this one. I do find it difficult to properly prepare: at more than the tea oxidises and still comes off as a bit too bitter. It’s more evenly balanced than the Senchas I have tried before, but that does mean it can come across as bland. I enjoy this tea the most when I am not doing anything else besides drinking tea. 3 out of 5 Blounts—Good. Miyazaki Oolong This complimentary little bag of Oolong tea leaves from Mr. Takuya Yokoyama tastes like a sweet Sencha instead of a typical Oolong tea. It’s one of the greenest ones with virtually no astringency, as described by Postcard Teas themselves. This is exceptional tea of which only was madein 2025. This is interesting because Oolong is usually made in China, not Japan. The problem is that this tea is very delicate: if you’re working or watching or playing something, you might gulp this down without blinking and afterwards think “what did I just drink?” I think these delicate teas are an acquired taste and require a mindful, peaceful moment of tea but nothing else. But why should I buy this Oolong when I already have the Guri Green? I usually prefer my Oolongs to be a bit more oxidised. I hope I’m not getting slammed for this. Oolong teas have a huge variety in roasting/oxidation/etc and this one ranges in the “barely Oolong at all” category. What I also learned is that for Oolong teas the first steep is usually a “wash” to get to the more flavourful second steeps. Perhaps I should try that for Miyazaki’s tea. 3 out of 5 Blounts—Good. Jasmine Green As mentioned on the postcard: “a delightful Vietnamese tea made with spring-picked green tea from Mr. Than’s tea co-op in the mountain village of Ban Lien in Lao Cai province”. Delightful is indeed the correct word here: this must be one of the best Jasmine teas I have ever tasted. It’s very delicate, never bitter, and after you’ve had a cup, you want to make another. What else can I say? It accepts but you better wait a few more minutes until it cooled down to at least and not let it steep for too long. Of course, our pantry now doesn’t stock the three Mariage Frères jasmine teas we tried, so I can’t directly compare them. They’re all great and completely different from the supermarket-bought Jasmine crap. 5 out of 5 Blounts—Amazing. Related topics: / tea / By Wouter Groeneveld on 29 June 2026.  Reply via email .

1 views
Stone Tools 2 months ago

Visual Basic on the PC w/Windows 3.1

If I dig deep into my own heart, really self-reflect, I find I simply don't possess whatever people like Bill Gates and Elon Musk do. I think most of us are content to know we've touched a life or two, helped make someone's existence a bit more pleasant, and can feel gratitude toward the universe for those small miracles. Others seem to know no limit in their acquisition of influence, power, and wealth. For them, it isn't simply enough to guide an industry, they must be the industry. In this zero-sum game, there is no upper limit to their cravings Before Musk became the first (I'm choking on the word) trillionaire , Gates was the world's richest person for a couple of decades. Like Musk, he crossed a specific monetary milestone back in 1999 as the "first person with a net worth exceeding $100 billion," about $200B in 2026 money. How he earned it and what he did with it has been the subject of any number of documentaries , books , movies , interviews , depositions , and damning rumors . I think the media can agree on at least one point relevant to our discussion today: Bill Gates was hellbent on owning the entire personal computing landscape. He said as much, out loud, on stage, to industry professionals, in front of the press. Jacqui Morby recounted the story on The Computer Chronicles . "Gary (Kildall) got up (at the Rosen Forum panel discussion) and talked about what his plans were for CP/M and where the company was going, and then made a comment, 'Well, this is a very large market, and there's room for lots of companies.' Bill Gates interrupted and said, 'No, there'll only be one company.'" He didn't seem particularly interested in creating innovative things, so much as he wanted to make sure that the innovations of others had a Microsoft response. While working with Apple to develop software for the original Macintosh, Andy Hertzfeld recalled a story of Gates digging in for system details that didn't really have anything to do with the business applications being built by Microsoft. Shortly thereafter, Windows 1.0 released, much to Steve Jobs's frustration . Jobs wouldn't be the last to feel screwed over by Microsoft "taking" ideas . Another tactic employed by Gates was absorption, the tried and true fast-track to acquiring toys one lacks. Consider the story of Alan Cooper . Coincidentally the idea for a visual application builder "popped into his head" just as HyperCard debuted, in 1987, triggered by Microsoft's announced adoption of DLLs, dynamic link libraries, which provided easy access to core operating system functions to whomever wanted to tap into them. Cooper saw this as a unique foundation upon which to build a kind of "construction set" for the DOS visual shell of your corporate dreams. Don't like the default Windows shell? Build your own! Microsoft engineer Gabe Newell was super impressed with Cooper's demo of the construction set, then called Tripod, and arranged for a demonstration for Gates. From the excellent article, "Something Pretty Right" by Ryan Lucas. "Why can't we do stuff like this?" is very revealing phrasing, IMHO as an armchair psychologist. Give that line to 1,000 actors and you'll get 1,000 unique performances balancing the tension between frustration and longing. As a Very Rich Guy™, there was nothing Gates wanted that he couldn't have. Like someone who pays others to level up their RPG character , US$1M and a contract later, Tripod (renamed Ruby) was his. While Cooper insists that HyperCard had no influence on the creation of Tripod , Gates most certainly was thinking about it. In his article "The 25th Birthday of BASIC" for BYTE Magazine , October 1989 ( Visual Basic would debut in 1991). Ruby was reformulated into something with but a passing resemblance to Tripod . Its bespoke scripting language was replaced with a variant of BASIC, and the goal of the program was no longer to build shells on top of the Microsoft DLLs, but to build applications for Microsoft's own shell, Windows 3.0. Visual Basic was born, arguably a more profound product than Cooper's original vision. Credit where it's due, Gates saw potential that Cooper himself couldn't see. A while back, I dug into Apple's HyperCard . Visual Basic gives us an interesting opportunity to look at a similar first-party, visual programming solution from Microsoft's perspective. Like HyperCard , Visual Basic had its own dedicated magazine , and inspired legions of developers long after Microsoft ceased support in 2008. As recently as 2023 , Microsoft has had to issue official statements on their support plans for "classic" Visual Basic, which keeps a huge number of bespoke, legacy applications alive, something HyperCard cannot claim. The Microsoft vs. Apple wars of the day almost necessitated taking sides, but in truth each has something it could learn from the other. Visual Basic 3.0 was the last pure 16-bit application in the line, and was the first version to include robust database capabilities. The true potential of the product was unlocked. This particular OS/application combination is much more in keeping with the spirit of this blog, I feel. There's a lot to learn. When I studied HyperCard , I noted the 1,000 page book that awaited me. Visual Basic ships with 3,000 pages, to say nothing of the wealth of 3rd party publications; an industry unto itself. As a man who recently took another annual step toward that great Blue Screen in the sky, every tick of the second hand gently rattles my bones. For large projects like this I have to consider how quickly I can get up to speed. Well, given the temperament of training books of the day, I suppose the proper first consideration is, "How dumb am I?" I refer to myself as a "big dummy" in blog posts, and I stand by that assertation, but I don't like it when others call me dumb. I can handle more complex material, but like I said, I don't have a lot of time. How quickly can I learn Visual Basic ? That seems unabsorbably fast . Maybe if I didn't sleep? I think I'd forget everything by Monday. Also by Tuesday. "Proglaming" sounds like fun, but a week is still too fast for my pace. Getting closer. Perfect. Slow enough for an old man to follow; fast enough to finish with time to spare before involuntary admission into a retirement home. If I weren't 40 years too late, I'd throw my own hat into the publishing ring and combine "I'm a big dummy" with "I want to learn this quickly." It's been a long time since I last touched Windows 3.1. It's funny, my memory of it doesn't match my hands-on experience today. I recall it being far uglier, though it still suffers from absurdly large title bars which don't provide much in the way of information or utility. I dig that (VGA mode) powder blue , though. It's handsome if perhaps uninspired, the result of a collaboration between Microsoft and IBM for OS/2's Presentation Manager (which predates Windows 2.0). Their "Joint Development Agreement" gave pretty broad latitude to both companies to use, without licensing fees, code shared between the two companies. I'm not even tangentially familiar with law, but it does read, in part: That gave Windows 2 and 3 a nice glow-up after the flop of Windows 1.0. Initially, even Microsoft had trouble getting their own developers to build Windows applications. I imagine it must have been a huge relief for Gates to have a tool that not only made it easy to build Windows applications, but that could even be an enjoyable experience. Jumping into Visual Basic , the first impression is, "I can do this." It looks approachable. I can't explain what every button in the toolbar does, but some of the basic stuff is as easy to identify as in HyperCard . Adding a control, like a text field, is a double-click away. The "Properties" panel makes intuitive sense, for tweaking the characteristics of a selected control, something HyperCard lacks. Appending code to a control is as simple as double-clicking its instance in the window. "Properties" is context aware, only showing what can be tweaked on the selected object. For the large part, the industry abandoned this contextual approach. I wonder why? PageMaker was leaning that way with its control panel, and InDesign promptly threw that away in favor of persistent controls for things that aren't even in the current document context. Why do we need text kerning tools on screen when there's not even a text box in the current document, in Affinity for example ? Tools like Figma , Apple's Pages seem to have kept the contextual flame alive, which is nice to see. "Pros want every tool on-screen at all times," a UX consultant once said with a straight face, I guess. The toolbar could stand to be better organized and starts gesturing in the direction of that meme image about Microsoft's love of buttons . They certainly did lean heavily on this UI metaphor crutch, as a catch-all way of cramming in as many features as possible. It's confusing at times (why a "picture box" and also "images?"), but with this version of the program, on this operating system, things haven't gotten completely out of hand yet. We're getting up to speed on the controls and how to interface with them today. Let's consider some nice things about Visual Basic's approach. I am rapidly growing to appreciate the keyboard shortcuts for UI elements, like buttons and sliders. Visual Basic makes it super simple to add a keyboard hook to an on-screen control. Simply label a button with in the confusingly named "caption" property and the following character will become the keyboard shortcut, via . So, an "Exit" button with the "caption" will read and will function identically to a mouse click on that button. When I say "identically" I do mean identically. The button's built-in method will be triggered, the same as if a mouse had done it. We don't have to worry about bifurcating control logic between keyboard and mouse for such interactions. We're then treated to an amuse bouche of off-kilter things to come. Checkboxes and radio buttons both have an on/off state, where any number of checkboxes can be on/off, but only one radio button in a set can be on. When programming with these controls, checkboxes return a value of or to represent unchecked or checked. Radio buttons return a or boolean on each of the options. For now, we'll file this under "Things That Make Me Give a Skeptical Sideways Glance." After spending a couple of days with it, the built-in text editor is driving me crazy, a "feature" Visual Basic shares with HyperCard ; neither is good. I can excuse a lack of autocomplete, a tool that would debut with Visual Basic 5 , as "Something Yet to be Invented." I cannot excuse the lack of indentation assistance and word-wraps, both already common features in word processors of the day. Microsoft has given us a smidge more than the absolute bare-minimum for a text editor. Keeping code tidy and readable requires significant, diligent effort on my part; it's not coming easily to me. I appreciate the auto-capitalization (though Basic is case-insensitive) and coloring on language keywords, but syntax checking and formatting a line of text the instant I've repositioned the cursor is annoying. Unfinished lines throw up modal dialogs warning me of interpreter troubles, triggered as easily as moving the cursor up or down for a moment. It's unwieldy to sketch out a code block to fill in the details later with those constant interruptions. It would be nice to be able to trigger the parser on-demand. We're learning about the mouse and how to handle mouse events. From a programmatic standpoint, this is pretty basic stuff. One of the nice things about the code editor is the pulldown in the top toolbar surfaces all possible functions for a selected UI element. We don't have to try to remember the exact name and spelling of a function; just pick the one you want to edit and get started. A setting that is theoretically interesting is the default unit of measurement for elements. Until now, I'd never heard of "twips": a "twentieth of a point". Where a point is 72/inch, there are 1,440 twips/inch. Windows used this as a device-independent standardized unit of measure. For on-screen, a conversion to pixels was used, and for print a conversion to printer resolution was used. Any form you design in Visual Basic can be trivially sent to the printer with a simple Basic call, and it will print at the resolution of the printer, not your screen. The coolest trick, though, is "edit and continue." Because the program is being constantly interpreted, not compiled, we can run the program, pause it, modify the code, and continue live execution. This is super handy for iterating solutions to annoying bugs. The Microsoft-faithful have really never known a world without this. The Apple-faithful have had this tantalizing fruit dangled before them a couple of times now, never quite delivering on the promise. I like it. In building out WIMP applications , we need to fill out the "M" part of that acronym. Today we learn how to build menus using the "Menu Design Window." The tool is competent, if a bit inelegant. Initially, it is easy to bang out a rough outline of an application's menu structure without taking one's hands off the keyboard; mouse-free is always a welcome option. Type a menu item, hit , type the next, hit , and the next, etc. Then, apply structure to the menu with the on-screen arrow tools for indentation/reordering elements. Alas, we cannot indent at the time of menu item entry, that hierarchy must be set in a separate step later. One disappointing absence is any kind of relationship between menu elements. Moving a menu item with "submenu" items will not move those submenu elements with it. No "outliner" style editing, ala ThinkTank , here. We also cannot multi-select items to edit them as a group, something we can do with form controls. Slow, patient, one-at-a-time editing of menu items is all we get. To be fair, menus can be programmatically generated, which may honestly be a better option in many ways. That pulls us away from the "Visual" in Visual Basic , though, don't it? The design window also forces its vertical editing into a horizontal view, another "Things That Make Me Give a Skeptical Sideways Glance." The example in the screenshot shows a 3-level menu, and I'm nowhere close to filling that horizontal space. It's wasted screen real estate, made more aggravating by the fact that the menu design window cannot be resized . As I think many in the industry have internalized by now, an editor view should place its primary content front and center, with refining elements playing a supporting role. The menu item properties would be much better served filling the right-hand side of the window, giving the menu itself vertical breathing room on the left. It's one of those things that probably gets better over the years, but is conspicuously half-baked for version 3 of the product. "It's OK, but I expected better by version 3," will be a running theme going forward. Now that I've been at this for a week, the angle of approach to visual programming HyperCard and Visual Basic each take has come into sharper focus. Initially, their superficial similarities led me to expect more direct parity between the two. Both provide a visual toolkit for designing interfaces. Both use a more simplistic language than the core language for each platform. Neither is truly "object oriented" (if that's important to you). Both were killed despite amassing a large, passionate following. Even a simple inspection of their toolbars highlights the philosophical difference between the two approaches. Most of the HyperCard toolbox is devoted to drawing pictures, with the controls reduced to buttons and text fields. It is constantly surprising to me how much mileage is squeezed out of such a restricted set of UI controls. Microsoft, on the other hand, offers a toolbar button for each and every thing you might want to add to an application. They take inverted approaches. Where I might add a generic button in HyperCard , then attach a script which invokes the system file browser, Visual Basic gives me a pre-built file browser control to drag into my app. I prefer Visual Basic's approach of "drag out a rectangle to define a control," especially for buttons and text fields; it feels more modern in its UX. HyperCard makes us add controls strictly by pulldown menu, then we have to drag the corners of the button, with no visual indicators, into the new size. Surprisingly awkward. Visual Basic also earns points in offering a grid to snap elements to position, making it much easier than HyperCard to align and scale elements precisely with one another. Gotta do a lot of eyeballin' on the HyperCard side of things; its grid only works in paint mode. Consequently, laying out something like a calculator is much faster and easier in Visual Basic , at the expense (?) of looking exactly like any other Windows program ever made. (Although the demo calculator doesn't look anything like the actual Windows calculator?) Don't get me wrong, conformance to corporate homogeneity may be exactly what you need at times and Visual Basic can generate something "professional looking" in a jiffy. It is, perhaps, devoid of character, but it also creates something a Windows user can look at and trust. Breaking free of those somewhat rigid constraints requires considered effort in Visual Basic , whereas HyperCard practically begs us to go hog wild. We're firmly in "learning Basic" land here; the application itself doesn't have a whole lot else to it. The panel for exporting our .exe files is about as barebones as one could imagine. There's a color palette, but I'm not entirely clear why; colors for controls can be set in the Properties palette via its own popup color palette. I should also give a shout out to the built-in Help system. Though I wish it were context aware, there's an absurd amount of information available right there in Windows without having to crack open the 10 pound manual. HyperCard has Balloon Help, which is nice and cute, but also anemic; we only get as much explanation as fits in a couple of sentences. Visual Basic's help system gives lengthy, detailed explanations of topics with code samples, is searchable, is bookmarkable (!), has tutorials for understanding the principles of the program, and more. It's quite good! The last week of my training book gets intense with discussions on make files, database connectivity, MDI (multiple document interface), DDE (dynamic data exchange), interfacing with DLLs, and so on. We've only been building throw-away toy applications so far, and I honestly don't feel the book has mentally equipped me for these hairier discussions. It's a pretty significant cognitive leap from the simplicity I feel the product promised. The long and the short of it is, I'm learning enough Basic to squeak by and get a sense of its tempo and grammar, but as a first-time user I find it more overwhelming than HyperTalk. HyperCard and Visual Basic each come with a 600+ page language reference guide. Microsoft also throws in three more manuals, another 2,400+ pages, for good measure. Its language guide would expand to 1,000+ pages in Visual Basic 4. Brevity is the very soul of cowards, I guess was their stance. Though their language reference guides are similar length, Microsoft's is a far more dense, dry tome. Apple spends the first 150 pages talking about "What even is programming?" and the last 150 pages getting into topics outside the scope of HyperTalk; a slim 300 pages to describe the language. Let's examine some concrete examples. Here's how to make the system thrice on the click of a button in HyperCard : Here's how to (ostensibly) do that in Visual Basic 3: Full disclosure: this didn't work, even though it is the example given in the "Programmer's Guide." Something is coalescing the three beeps into one. DOSBox-X issue? Because scripts are kind of "embedded" into their respective HyperCard objects, we don't have to disambiguate subroutines with prefixes; any given script is scoped precisely to its associated GUI object. It's the La Croix of object orientation; just a whiff of a hint of that flavor. HyperCard's approach lends itself better to casual tinkering around, but Visual Basic has an edge in surfacing all functions of our application in the code editor. In HyperCard we have to remember which object contains which code block, or hunt through all objects individually, searching for the code we want. Visual Basic's approach requires unique names for all subroutines. This makes it fairly trivial to trigger events across objects. If we want a button to click another button by proxy, we would have to do something like this in HyperTalk: Sometimes I wish HyperTalk would allow dot-syntax for object specifier chains. In Visual Basic, we simply call the uniquely-named function directly: Where HyperTalk takes a gentle, English-like approach to its language, Visual Basic isn't afraid to be far more "programmery." HyperTalk developers can certainly get into their own weeds trying to figure out the precise incantation to sidestep the interpreter and achieve specific goals. Conversely, Visual Basic developers could quickly find themselves in a world of memory management, DLLs, batch files, and make files. Both developers feel some pain, but one is kind of orthogonal to the other. Your preference may depend on which breed of demon you enjoy slaying. As clearly evidenced by the Voyager series of software and MYST , highly professional software was possible with HyperCard . That said, the upper boundary for Visual Basic feels much higher. As a simple example, with the keyword we can reach in and directly call the Windows Kernel (or any existing) DLL; this of course being the killer feature that triggered Alan Cooper to develop the program in the first place. That's impossible to do out-of-the-box with HyperCard ; it cannot access the Macintosh Toolbox so deftly. Likewise with database data, Visual Basic gives us flexibility in what kind of data to bring in, like dBASE or FoxPro . There may be specialized stacks or XCMDs (plugins) to HyperCard that can assist with these tasks, but nothing native to the program. However, HyperCard provides its own built-in database free of charge, requiring no special effort on the developer's part to leverage it. Building something like an address book is simply a matter of adding some text fields to a card. Those will function like fields in a database by default, and actions like saving/loading user data will happen transparently. Adding search, or something similar, takes a few extra steps, but is conceptually simple through a HyperTalk command like Visual Basic provides a "Data Manager" module, which allows us to create simple Access databases for use as the backbone of the application. This is all explained in detail in the supplemental 300+ page "Visual Basic 3.0 Professional Features, Book 2." Once the database is built, interfacing with its records is straightforward using the "Data Control" tool. When the database is linked in properly, controls like images and text fields can be wired up directly to their corresponding fields in the database schema, called "bound controls." The database widget itself provides buttons to step through records and corresponding data will auto-populate the bound layout elements. If "browsing" is the extent of your database needs, you're in good shape. I imagine most will want to do more than that, perhaps adding fields, or doing search queries. You'll want to steel yourself, because it gets gnarly real quick. I'll just say that the book is 300+ pages for a reason, with talk about complex subjects like Dynasets, Snapshots, Tables, the JET engine, SQL queries, and more. It's far more capable than HyperCard , as we can work with multiple databases in our VB application, access remote databases, and more. That power is paired with an equivalent learning curve, one which is thrust upon any developer who needs even a tiny bit more than the drag-and-drop controls provide. Overall, it would be fair to call the IDE "competent." It contains the tools we need, arranged by palette, and makes certain actions (like adding a button) as easy as a double-click. What's not to like? I think what frustrates me about these tools is how they feel like somewhat careless design solutions to their respective problems. Look at the "Properties" palette, for example. This looks, to my eyes, like a developer was told, "The properties for a selected object should be available for editing." The developer iterated them as a literal list, adding some basic editing niceties, like making a color chooser available when a color property is edited. What I find in practice is that the vast majority of the properties go untouched, especially for something like a Form object, and the ones I actually need require scrolling through a long list to find and edit. Later properties in the list, even those which are common to all controls, shift around in position depending on how many properties a given control has. I'm constantly having to read through that list, scanning for the "Name" property, which is where we set the programmatic name for the control. It's arguably the most important property , and it's playing peek-a-boo. When I make a new form (a "form" is a window; I don't know why they call it a "form") I have a few things I need to set right off the bat: the size, the title, and the programming reference name. After that, sometimes I want to set the background color. We'll ignore the fact that property names don't make sense; naming conventions had perhaps not yet been firmly established in an era when the terms UI and UX had not yet become common vernacular. From a pure, "What is the user most likely to need?" point of view, this simple alphabetical list is the laziest solution to the design challenge. Fair point, HyperCard's lack of any properties palette was more lazy, but this is version 3 of this product. I frankly (perhaps unfairly) expect more considered effort from a first-party solution. My frustration extends to the main toolbox as well. It's just a bunch of buttons with no organizational structure applied. Tooltips, similar to what we understand today, were introduced with Macintosh System 7 as "Balloon Help" the same year VB3 released, so I can't fault Microsoft for "failing to implement" them in this release. Still, icon-only is a lazy way to handle it, when the goal is to shove as many icons into the toolbar as possible. Asymetrix Toolbook 3 , a similar visual IDE for Windows development, provides more robust, logically arranged tools for the job. Here's the text editor and object properties panels. Note in particular a few things: Visual Basic itself contains a similar contextual help in other parts of the application, like its "Crystal Reports" tool, making its absence in the main app even more frustrating. This kind of haphazard application of tools and controls feels sloppy, which reminds me of something I wanted to talk about. While going through the official manuals for Visual Basic , something kept bothering me. I couldn't put my finger on it at first, but once I saw it, my eyes were forever cursed . This is a small grievance, "petty" some would say, "a colossal waste of mental resources" others may scoff. But what's a tech blog without a certain level of pedantry? I'm not above pedantry. Here we see the Visual Basic 3 manual is laid out in Helvetica and Times. Man, I'm already bored. Anyway, beyond the utterly pedestrian font choices (in fairness, they did have to lay out 3,000+ pages of this stuff), something seems "off" about it. In particular, that Helvetica looks malformed, with sloppy kerning and unbalanced strokes. Let's take a closer look. Helvetica Neue doesn't match, and Arial (my original suspect) is ruled out by the end caps on the capital "C". Helvetica Condensed is also not right. Wait, I see what's happening. It's the same issue I have with the user interface, manifested in the manual. This isn't Helvetica Condensed, it's Helvetica physically squashed into a fake condensed version. The richest man in the world couldn't afford to buy a proper condensed font for his company? "Or is this indicative of a deeper issue?" he asked, slipping back into his pop-psychology armchair. It smacks of "good enough," never striving for "great." That kind of sums up my feelings toward Windows and Windows applications of this period. The stuff worked, and had obvious success, but never seemed to be borne of thoughtful consideration. Did that inattention to detail come from cost-cutting measures, or perhaps some kind of cultural blindness? Were the deficiencies seen and ignored, or simply not seen at all? And that reminds me of something else I wanted to talk about. In the PBS documentary series, Triumph of the Nerds , Steve Jobs famously said of Microsoft, "They have no taste." I genuinely think Bill Gates could not understand the meaning of Jobs's accusation. Or rather, he couldn't fathom why "taste" should enter into his calculus whatsoever. Having no taste didn't stop him from becoming the richest man in the world. What does "taste" have to do with stockholder value? When Apple teased with a new release of OS X, "Redmond, start your photocopiers," I think Gates was thinking, "Of course we will. Thanks for the free R&D." He bristled at being publicly chastised for copying , but my read on that is he really wanted to say, "So what if we copy Apple? Why shouldn't we? Look at our success and tell me it hasn't been a good strategy." What Jobs saw as creative bankruptcy, Gates saw as business efficiency. Being asked to frame his success on Jobs's terms ruffled Gates's feathers. Jobs said, and I agree, that innovation means saying "no" to 1000 things before saying "yes." "Process" is that very action. "Process" is the pruning of the possibility space. It's the self-awareness to distinguish "good enough" from "great." It's when you step away from your work, give it the critical stink eye, and apply taste . That's an impossible task if one has no taste to begin with. So what's a tasteless corporation to do? While Microsoft may have not cared too much about process, they had manufacturing down cold. Put in PenPoint OS, out pops Windows for Pen Computing. Put in OS X 10.3, out pops Windows Vista. Put in Java, out pops J++. Put in a Dreamcast, out pops an Xbox. Even today, similar "factory production" charges are levied against them. I'm not suggesting they "stole" ideas so much as they simply seemed content to let others do the hard work of saying "no" 1,000 times. While they may have shortcut the creative process, they still had to learn how to manufacture products. In so doing, they accidentally picked up a little taste along the way, which would lead to pretty good stuff from time to time. It's been part of the fabric of the industry for decades, and now the torch of manufacturing tasteless product from the creative work of others has been passed on to generative AI. To scale , no less. The ramifications weigh heavily on my mind, especially when someone publicly calls for the absorption of my work into the generative AI apparatus. I'm both flattered and appalled. On average, how many times do you think I rewrite the introductions to these posts? How many thousands of words have I thrown away to reach something approaching what I wanted to actually say? I tend to rewrite intros 3 or 4 times, and I mean that truly; each rewrite is radically different from the others. In this post alone, I have thrown away some 5,000 words. Some might think those 5,000 words are the cost of the process, but that's not right. They are the process. The unpublished words are the important ones. Those are the words that got me to these words. Knowing that, throw any creative work into the generative wood chipper and it should be obvious why what comes out cannot live up to the original. It's lacking the 1,000 nos. I'm disappointed in the ending of this book. Day 21 comes and goes without even a hint of acknowledgement that we've made it through the gauntlet. At the end of it all, we also haven't built anything of value. Every chapter created little baby programs to illustrate specific concepts; no project built upon a previous project except for a few shallow, superficial glow-ups. Contrast that with HyperCard , where we had a full-fledged address book, with database, search, custom art, and save/load. With Visual Basic , I never felt that same spark I did with HyperCard . Visual Basic seems great for when you have a strong idea of what you want to build. However, its lack of drawing tools and "don't worry about it, I've got you covered" database curtail creative exploration far more than I would have predicted at the beginning of my studies. Not having to worry about those details opens up a wider world of "lemme try something real quick" experimentation and iteration. In an ideal product, I'd combine the prototyping strengths of HyperCard with the professional-strength of Visual Basic . Then, later we could swap out the default database with Access, or export the placeholder drawings as image assets for a professional artist to clean up in another revision. I cannot personally find a place for Visual Basic in my heart, but I can absolutely understand why it took off. It filled a major gap in the programming landscape, helping amateurs and pro-ams build tools for themselves, and even throwing a lifeline to a generation of COBOL engineers needing to transition ASAP. Like Apple with HyperCard , that gap was re-opened by the discontinuation of the product, abandoning a whole fleet of developers and, perhaps just as importantly, potential developers. I suppose nothing lasts forever, but these companies are worth multi (choking on the word again) TRILLIONS of US dollars. At valuations like that, with the fealty they demand from us, I consider it a moral imperative for them to provide excellent tools which empower the widest possible breadth of users' skill levels. Not providing such tools is a choice . Considered from another angle, I'll leave you with this open question. What software do Apple and Microsoft provide today that will be spoken of, with the same reverence as HyperCard and Visual Basic, 25 years from now? Ways to improve the experience, notable deficiencies, workarounds, and notes about incorporating the software into modern workflows (if possible). With Visual Basic 3, 2, 1, and DOS 1.0, the applications you build are 16-bit only and are therefore relegated to running only in virtual environments on 64-bit Windows. If this fits your modus operandi, you're in good shape. If you're hoping to keep it old-school, but still want the option of running your creation on modern hardware, then you'll want to get Visual Basic 6 up and running in Windows 2000? XP? I tried it in Windows 98SE and it wouldn't launch. VB6 builds 32-bit applications as standalone, compiled executables, can connect to the Internet, and produces builds which run on Windows 10/11. Note that Windows 11 promises to run applications built with VB6 , but does not promise to run VB6 itself. However, I gave it a shot and though there were issues with the install, and the IDE acts a little weird, and it complains on launch about missing OLE files, it did run and I was able to build an executable on Windows 11. For funsies, here's Gates and Jobs demonstrating their respective visual programming environments. Gates giving a subdued demo of the just-announced Visual Basic 1.0 . His voice cracking at 0:33 is adorable . Jobs had just returned to Apple after they bought NeXT, and here he's showing the technology Apple has bet its future on. We know it today as Xcode , but it started life as Interface Builder . The line he drew between components in the demo was called a "binding," something that has conceptually resurfaced in SwiftUI. DOSBox-X 2026.01.02, Windows x64 build. CPU set to Pentium DOS reports as v6.22 Host system folder mounted as drive C:\ holds Windows Windows 3.1, basic installation 1024 x 768, 32K colors under DOS reports total RAM, but Free only reports . Good enough for today, but 16-bit Windows should be able to register 4MB, not just 2MB. A few extra applications for comparative/convenience reasons: Toolbook, Actor, ObjectVision, Acrobat Distiller Visual Basic 3.0 Reports 386 Enhanced Mode enabled Reports free RAM In lieu of tooltips, at the bottom of the current window we have a contextual description of the current tool, much like Bank Street Writer and Lotus 1-2-3 . The text editor includes indent/outdent tools, can set our editing font of choice, waits to check syntax until we ask it to, and even includes a simple "build a function" utility to wire up common tasks to common UI events. The properties panel is laid out hierarchically, keeping the most-needed stuff front and center, while demoting less-used options to secondary emphasis. DOSBox-X ran everything smoothly and without issue. I did not install Windows on top of real DOS, though. I relied on DOSBox-X's implementation. This may account for a couple of strange issues, outlined below. I experienced one crash in Visual Basic 3 , when accessing the Help system. Issuing a looped command resulted in only a single system beep. My guess is that something in the emulated environment is suppressing this. I could never get databases to connect, even the ones that ship with Visual Basic , let alone any personal data carried over from previous database explorations. It may be the result of DOSBox-X using an emulated version of . Strangely, I saw it work once and then it stopped working as suddenly as it started and never worked again. An installation of Windows on a proper installation of MS-DOS might fix this problem.

0 views
Neil Madden 2 months ago

Java’s SSLContext protocol name is a footgun

This should be old news, but I keep seeing the same mistake crop up, so I thought I’d blog about it and spread awareness. In Java, if you want to configure TLS you generally start with an . And you get an instance of this class by calling the static method , specifying the version of the protocol you want to support. But typically a TLS connection supports other versions of the protocol, so what exactly does specifying “TLSv1.3” here mean? Probably not what you think it means… The Java Security Standard Algorithm Names document doesn’t say much useful: “Supports […] TLS version 1.3; may support other SSL/TLS versions.” Well, which other versions? Later ones or earlier ones? That seems kind of important. It’s even vaguer if you don’t specify a version – the generic “TLS” identifier is specified as (my emphasis): OK, but what about the JSSE chapter in the Java Security Developer’s Guide . That seems at first glance to be more precise: Like other JCA provider-based engine classes,   objects are created using the   factory methods of the   class. These static methods each return an instance that implements  at least  the requested secure socket protocol. The returned instance may implement other protocols, too. For example,   may return an instance that implements TLSv1, TLSv1.1, and TLSv1.2. OK, that sounds promising! So, if I specify a version then that is taken as a minimum version and I may also get more recent versions? Great, sign me up! Except, that is the exact fucking opposite of what the default SunJSSE provider does! When you specify “TLSv1.1” (for example), what the default provider does is treat that as a maximum TLS version . So the resulting supports all versions of TLS up to (and including) 1.1, but nothing later. So if you have old code that requests version 1.1 and you try to connect to a modern server that only supports 1.2 and 1.3, then you’ll get a connection failure. And in modern Java, this will fail earlier because TLS 1.1 is disabled by default. If you specify “TLSv1.2” then you’ll just silently get a downgraded protocol for no good reason at all, when you probably thought you were being good and specifying a sensible minimum version. It’s not just the default provider that does this, lots of others have followed the lead, including e.g., the Conscrypt/BoringSSL provider used by Android. I suspect this behaviour exists because of a fear of breaking poorly-written software that baulks at unknown versions and doesn’t handle downgrades properly. But the problem is that many developers think it is best practice to specify a version when creating an SSLContext, and some security scanners even tell you to do so . In the best case, you then get code that is secure until the next major hole is discovered in TLS and v1.4 gets released. In the worst case you’ve silently implemented a self-inflicted protocol downgrade attack. I wonder how many Java apps were (and maybe still are) only supporting TLS 1.1/1.0 despite the underlying JDK supporting 1.2 or even 1.3? I should stop here and mention a subtlety: this behaviour applies to client TLS connections only. Server-side SSL contexts completely ignore the protocol you specify here for the most part and go ahead and support everything . So what should you do instead? Well there’s really no good answer here. Probably the best thing to do is to use the generic “TLS” identifier, which gets you an unspecified version of TLS, but which all providers I’ve looked at so far interpret as “sensible modern protocol versions”, i.e., TLS 1.3 and 1.2 (with 1.1 and earlier supported but disabled by default). There’s no guarantee at all of that behaviour, but there’s also no guarantee when you specify a version, so pick your poison. (I’ve raised a bug for this, as it finally pissed me off enough, but my guess is they’ll either ignore it or fix the guide to be as vague as the standard names descriptions).

0 views
Blog System/5 2 months ago

Is anyone still using Emacs?

In a recent discussion at the orange site sparked by the Emacs 31 Is Around the Corner: The Changes I’m Already Daily Driving article, people were asking themselves “Is anyone still using Emacs?” and then providing their own perspective. For me, the answer is a resounding yes… but the interesting part is that I’m not still using Emacs: I’m actually using Emacs again . And instead of burying my answer to the opening question in a long discussion thread, I thought I’d explain my journey with and without Emacs for the last… almost 30 years. At the end, I’ll unveil the specific feature that I feel gives me superpowers and that keeps me hooked. Show your support for this publication by subscribing. It’s free if you want it to be! I got into Linux around 1997 via Caldera OpenLinux 1.1. Before then, I had extensively played with Borland Turbo C++ and Visual Basic as a kid so I was heavily accustomed to those fancy IDEs that we lost . As I got into Linux and found myself in an alien world, I had to buy a couple of introductory books. Yes, books, the printed kind—because that’s how we had to learn new stuff before. Both books talked about Vim and Emacs and presented them as the advanced choices. I found this strange because the IDEs I had used before seemed more complete, but I, being a Windows renegade for some reason, charged ahead. I learned the basics of both editors and went through their tutorials at different times. The two old books I used to learn Linux back in the day, open to show their Vim and Emacs introductions. Since then and until roughly 2015, I flip-flopped between Vim and Emacs. At times I used one, and at times I used the other. I favored Emacs for long-running coding sessions but Vim excelled at my pkgsrc work where I had to edit tens of different files in quick succession. Even though Vim and Emacs worked well for me, I missed something. Language integration was poor so I was tempted by the more modern editors that everyone was touting, and especially so as I moved to macOS. I tried a bunch, like the now-defunct Atom and Brackets , but they all felt brittle and overwhelming: they had too many features, too many settings. And then, VSCode arrived in 2015. As I took it for a test drive, it “felt right” from the get-go. It looked modern, was relatively small, and its plain and simple settings editor—read: just a JSON file because there were no settings panels yet!—made me feel like I was in control. I could understand this modern editor and easily tune it to my needs. Soon after, I started learning Go and then Rust, and VSCode’s integration with their corresponding LSPs made that process so much easier: code auto-completion and real-time error highlighting sped up my learning significantly. I stuck with VSCode for these languages and slowly phased Emacs out. I was sold. During that time period, I was also working on Bazel—a Java project—at Google and the natural choice for it was IntelliJ. I had tried to use Emacs for Java development at some point, but IntelliJ was (and still is) so good that it was the only realistic choice. My usage of VSCode with its Vim plugin continued through my short stint at Microsoft, where I was working on a C++ codebase and had to connect to remote Windows boxes. Most people used RDP to work on the remote machine “directly”, but I couldn’t stand that workflow: I very much preferred running VSCode on my desktop and using SSH to connect to the remote machine, which is something that VSCode does very well. And then… I moved to Snowflake in 2022 where development used to happen inside an ancient Linux VM and where my day-to-day job was to write shell scripts and Bazel build files: neither VSCode nor IntelliJ were going to save me here, and as I mentioned earlier, I hate the feeling of working within the constraints of a “remote” graphical environment. So my instinct was to go back to SSH and connect to the local VM with it. As I did that, I needed an editor for long work sessions, and the old and trusty Emacs was there waiting for me. But this time around, I didn’t have the patience to set it up. You see: I had accumulated hundreds of lines in my file over the years without understanding much about them, and I wanted to throw it all away and start over… but it all felt like too much work. Maybe destiny brought Doom Emacs my way at the right time. Stock Doom Emacs screenshot from the project's website. You see, Doom Emacs is an Emacs “distribution” where someone has gone through the pain (or joy, I won’t judge) of configuring Emacs from the ground up. More specifically, Doom Emacs offers sane defaults, predefined language integrations, and an experience that welcomes ex-Vimers. It doesn’t claim to be an IDE… but it feels like one to me. Once I set it up, I experienced déjà-vu: Emacs felt right just like VSCode did in 2015. All of a sudden, lots of Emacs features became discoverable via interactive popup menus accessible behind space-based shortcuts that don’t destroy your wrists, and coexisting with the same Vim-style key bindings that I had grown so used to. But what’s more: the configuration felt simple and understandable, spread across just three trivial files: to specify global settings like the theme or the fonts to use, to select which Doom-specific modules need to be enabled, and to install non-Doom packages. The defaults for these files are reasonable, with plenty of comments to configure the few details you might want to tune. With this new setup, I have had the best Emacs experience ever. Thanks to the advances in LSPs (for which we have to thank VSCode) and modern features like tree-sitter, Emacs now feels like an IDE: I get proper language integration for most languages I have to deal with. And the absolute killer feature for me is that I get the exact same development environment no matter what machine I need to work on. It doesn’t matter if it is a MacBook or a Linux laptop, or if I’m connecting to a Linux cloud workstation or even my own FreeBSD server: all I need is a shell, tmux, and Emacs, and I am equally productive. This, to me, is really valuable because I tend to work on a variety of machines and muscle memory pays off. If you research Doom Emacs online, you will find people “complaining” that “it does too much”. And that’s true: it does, which is why I find it so useful. But I often wonder if I could cut things down because someday I’d like to learn more about Emacs. This is especially true now that I see many modern third-party modules “graduating” and becoming part of the stock package. For those reasons, I’ve recently been tempted to try the Bedrock or Emacs Solo distributions. However… the activation energy required to make the switch is pretty damn high. And if I decided to go that route, well, I’d still question myself for not truly going all the way to “raw” Emacs. And before closing, a related thought: I can’t quite comprehend how Emacs becomes transformative for people due to its Elisp backing. Sure, I could implement more logic and workflows within Emacs, but I already do “everything” with ease in the shell via scripts—and scripts feel more Unix-y because “Unix is my IDE”. I actually don’t like how Org mode and Magit are “locked” behind Emacs instead of being standalone applications. I’m surely missing something, but I’m not quite sure what it is… So coming back to the question that opened the article: yes, I still use Emacs, and it has become even more important to me than it was in the past due to my need to work on disparate remote machines all the time. Now the questions for you are: do you “still” use it too? What distribution, if any? How does Emacs transform your workflows? Now that you have made it this far, consider subscribing and/or sharing this article in your favorite social platform for further discussion! The two old books I used to learn Linux back in the day, open to show their Vim and Emacs introductions. Since then and until roughly 2015, I flip-flopped between Vim and Emacs. At times I used one, and at times I used the other. I favored Emacs for long-running coding sessions but Vim excelled at my pkgsrc work where I had to edit tens of different files in quick succession. The switch to VSCode and IntelliJ Even though Vim and Emacs worked well for me, I missed something. Language integration was poor so I was tempted by the more modern editors that everyone was touting, and especially so as I moved to macOS. I tried a bunch, like the now-defunct Atom and Brackets , but they all felt brittle and overwhelming: they had too many features, too many settings. And then, VSCode arrived in 2015. As I took it for a test drive, it “felt right” from the get-go. It looked modern, was relatively small, and its plain and simple settings editor—read: just a JSON file because there were no settings panels yet!—made me feel like I was in control. I could understand this modern editor and easily tune it to my needs. Soon after, I started learning Go and then Rust, and VSCode’s integration with their corresponding LSPs made that process so much easier: code auto-completion and real-time error highlighting sped up my learning significantly. I stuck with VSCode for these languages and slowly phased Emacs out. I was sold. During that time period, I was also working on Bazel—a Java project—at Google and the natural choice for it was IntelliJ. I had tried to use Emacs for Java development at some point, but IntelliJ was (and still is) so good that it was the only realistic choice. My usage of VSCode with its Vim plugin continued through my short stint at Microsoft, where I was working on a C++ codebase and had to connect to remote Windows boxes. Most people used RDP to work on the remote machine “directly”, but I couldn’t stand that workflow: I very much preferred running VSCode on my desktop and using SSH to connect to the remote machine, which is something that VSCode does very well. Back to (Doom) Emacs And then… I moved to Snowflake in 2022 where development used to happen inside an ancient Linux VM and where my day-to-day job was to write shell scripts and Bazel build files: neither VSCode nor IntelliJ were going to save me here, and as I mentioned earlier, I hate the feeling of working within the constraints of a “remote” graphical environment. So my instinct was to go back to SSH and connect to the local VM with it. As I did that, I needed an editor for long work sessions, and the old and trusty Emacs was there waiting for me. But this time around, I didn’t have the patience to set it up. You see: I had accumulated hundreds of lines in my file over the years without understanding much about them, and I wanted to throw it all away and start over… but it all felt like too much work. Maybe destiny brought Doom Emacs my way at the right time. Stock Doom Emacs screenshot from the project's website. You see, Doom Emacs is an Emacs “distribution” where someone has gone through the pain (or joy, I won’t judge) of configuring Emacs from the ground up. More specifically, Doom Emacs offers sane defaults, predefined language integrations, and an experience that welcomes ex-Vimers. It doesn’t claim to be an IDE… but it feels like one to me. Once I set it up, I experienced déjà-vu: Emacs felt right just like VSCode did in 2015. All of a sudden, lots of Emacs features became discoverable via interactive popup menus accessible behind space-based shortcuts that don’t destroy your wrists, and coexisting with the same Vim-style key bindings that I had grown so used to. But what’s more: the configuration felt simple and understandable, spread across just three trivial files: to specify global settings like the theme or the fonts to use, to select which Doom-specific modules need to be enabled, and to install non-Doom packages.

1 views
David Bushell 2 months ago

RSS Club #008: Duck duck, swan?

This is an RSS-only post, thank you for subscribing :) If you’re only here for web and tech talk you can skip this one! I rescued an animal today! Probably… The UK has its fair share of canals. I like canals. They cut through urban life offering an escape back to nature and are teaming with wildlife. Canal towpaths are perfect for running. They’re easy underfoot — until late spring when the goslings hatch and then I’m doing a ballet to avoid trouble. Canada goose are the most visible bird living here all year round. They gather in groups and are rather docile around humans until the little yellow fluffballs arrive and then it’s mayhem. Mute swans are a less common sight on the routes I run. This year a pair chose to nest in a safe but visible spot which was wonderful to witness. Swan nests are huge mounds of dirt, twigs, and coke bottles, apparently. This morning I found dad-swan charging back and forth across the water. He stopped to peer into an overflow trench around 2–3 feet deep aside the canal. As I ran closer I saw a young bird has fallen in. It was older than a fluffball but still covered in muddied down. Larger than a duck, for scale. It was still too young to fly out of its predicament. At first I thought it was one of the cygnets. Mum-swan was in the nest with the others not far away. I don’t speak bird but dad-swan seemed more aggressive than concerned. As I got closer he paddled a short distance away to observe. I went down on my stomach and slowly reach under the guard rails wondering how painful a finger-pecking would be. I kept my ears open for a charge attack. The young bird didn’t flinch. It allowed me to reach under its belly and lift it up. Before I could place it safely on the ground it attempted a Loony Tunes escape by running in the air. This unbalanced and forced me to tip it sideways, thankfully onto the stones just below water level and not back into the trench. It then frantically hopped not into the water, but up onto the towpath and quickly waddled behind me into the grass. As I got back to my feet dad-swan returned to investigate and looked satisfied the young bird was gone before returning to his nest. It took me a minute to find the young bird now resting deep in the brambles. It was only then did I realise this might not be a swan but a goose. It was large enough to have outgrown the distinct yellow colouring. I left it where it was hiding. My presence would only cause further distress. It was not physically injured otherwise I might have called the RSPCA who can rescue wildlife ( RSPB don’t; common misconception). I don’t go running with my phone anyway so I returned later to check and take photos. The young bird had vanished from its hiding spot. I’m almost certain it was a goose now after seeing this family not far from the scene. It’s funny, despite being so common I’ve never once seen an actual goose nest. I’ve no idea where they hide them. Thanks for reading! Follow me on Mastodon and Bluesky . Subscribe to my Blog and Notes or Combined feeds.

0 views