Latest Posts (20 found)
Unsung Today

“I found an innocent loop that had no test in it.”

(It’s Halloween Week on Unsung, where we’re sharing scary, spooky, and creepy stories.) Speaking of fireside stories, this is a classic 1983 post called “The story of Mel, a real programmer”: […] I first met Mel when I went to work for Royal McBee Computer Corp., a now-defunct subsidiary of the typewriter company. The firm manufactured the LGP-30, a small, cheap (by the standards of the day) drum-memory computer, and had just started to manufacture the RPC-4000, a much-improved, bigger, better, faster — drum-memory computer. Cores cost too much, and weren’t here to stay, anyway. (That’s why you haven’t heard of the company, or the computer.) […] I don’t super love this presentation/​style, but it’s an interesting “close to the metal” programming anecdote (similar in some ways to Duff’s device ), and doubly interesting to try to go through the jargon and actually understand it all. Not particularly relevant to the story itself, but this is LGP-30, launched in 1956: = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/i-found-an-innocent-loop-that-had-no-test-in-it/1.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/i-found-an-innocent-loop-that-had-no-test-in-it/1.1600w.avif" type="image/avif"> I’ve known LGP-30 for a while – some of the photos people put in the Wikipedia entry are actually ones I took at the Computer History Museum – but I saw a running one only earlier this year at a computer museum in Stuttgart. I particularly loved the industrial, lit up buttons… = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/i-found-an-innocent-loop-that-had-no-test-in-it/2.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/i-found-an-innocent-loop-that-had-no-test-in-it/2.1600w.avif" type="image/avif"> …and the “display” which was just an oscilloscope contorted to fit within a specific binary template: As far as I know, RGP-4000 – the 1960’s successor – didn’t feel much different, so perhaps seeing these might actually enhance the spooky story.

0 views

Fixing my LocalBytes Tasmota smart bulb which had lost its Tasmota template

I have used a LocalBytes smart bulb in our porch light for ages . It is controlled by Home Assistant, so that it comes on a few minutes before sunset, and goes off later in the evening. For some reason, it reverted to Tasmota’s default settings - without the LocalBytes bulb functionality - a while ago. Now that the clocks have changed, and it is darker earlier, fixing it moved up my priority list. Fortunately, it was pretty straightforward, thanks to the LocalBytes documentation . The key bit is the Tasmota template for the bulb: With this, and the Tasmota instructions for importing templates , I have a working bulb again.

0 views
Unsung Today

The Swiss Cheese model, pt. 4: The iPhone jump scare

(It’s Halloween Week on Unsung, where we’re sharing scary, spooky, and creepy stories.) The first time I encountered this bug, years ago, I jumped up my seat. I was listening to the music on my iPhone and suddenly, for no particular reason, the music got extremely loud. I frantically held the volume down button, calmed down the sound, and didn’t think much about it. But it happened again a few days later. And then again. And again. I slowly started paying more attention to this, trying to figure out what might have been causing it. I started piecing together that it wasn’t wholly random: I noticed whenever it happened, it seemed to be only when I started or resumed the music – but not every single time I did so. Then, I also noticed it only happened whenever I was putting the phone away. Eventually, I figured out the whole sequence of events. In 2019, Apple changed the design of the volume indicator so that it went closer to the physical volume buttons. Here’s before and after: I praised the contemporary version of this animation earlier on Unsung , but what I didn’t mention then was that it wasn’t just a visual change. No, the new control was also interactive. Once you popped it up on the screen, you could drag the volume up and down with your finger: I never noticed that, nor would I ever want to change the volume this way, particularly since the only way I knew how to invoke it was with my fingers already on hardware volume buttons. A second component? For reasons that are likely intentional, but I don’t personally understand, the iPhone doesn’t cease touch detection in the few hundred milliseconds between clicking the lock button and the screen completely fading away. And then, the last thing: It turns out that as I was putting the phone away into my left pocket, my thumb would graze the screen, near the left edge, always in the same position. That position happened to be the location of the interactive volume indicator. I think you can assemble the whole puzzle now. Occasionally, I would start playing the music, instinctively adjust the volume with one of the buttons, lock the phone, and put it in my pocket. Adjusting the volume would make the control appear, and the control was ready to accept touch input, too. Locking the phone would not immediately shut down touch detection. As the phone was sliding down my pocket, the volume controls still on the screen, the screen about to fade out but still responding to touches, my thumb would graze the volume control – and while the thumb was stationary, the phone would be moving down into my pocket quickly, so it felt like an upward slide. Every single one of those instances, it was me who accidentally turned the volume all the way up. This all needed to happen quickly, almost as one gesture – but that is exactly how it goes if you’re doing it the same way for years . As a matter of fact, here’s a video of how it happens, and it’s so quick you cannot see anything: (Unsung doesn’t support audio at the moment, but you can watch the same video on YouTube to hear the sudden volume increase.) The funny thing is that I was likely doing this for years, but it never mattered, as my accidental swipe up didn’t target anything on the screen. It was only with the introduction of the interactive volume control that the Swiss Cheese model was complete: Here’s the tricky part: It’s been years since I figured out the sequence of events. I know exactly what happens. I know how to avoid it. Yet, those things happen so naturally, that I still occasionally blast music right into my ears and scare myself to death. The previous three times we talked about the Swiss Cheese Model , the examples were more cerebral. This one is visceral and frightening. I hate this bug so, so much. the control was interactive, there was no (accessibility?) option to make it not interactive – which I would want anyway, since I sometimes activate this control in other contexts, too, and never do it intentionally, locking the phone still made the screen accept touch inputs for a moment, as I was sliding the phone into my pocket, my thumb ended up being in the perfect position to change the volume, it just happened that the movement of the phone and the direction of the control resulted in the worst possible scenario – volume going all the way up.

0 views
Unsung Today

“For the first several seconds, there would have been little to suggest that anything was wrong.”

(It’s Halloween Week on Unsung, where we’re sharing scary, spooky, and creepy stories – and in posts like this one, sometimes not particularly enjoyable ones.) In October 2004, a Boeing 747 cargo aircraft crashed on take-off in Halifax, Canada, claiming the lives of the crew of seven. A YouTube channel Mentour Pilot walks us through the complex set of events that led to the crash: = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/for-the-first-several-seconds-there-would-have-been-little-to-suggest-that-anything-was-wrong/yt1-play.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/for-the-first-several-seconds-there-would-have-been-little-to-suggest-that-anything-was-wrong/yt1-play.1600w.avif" type="image/avif"> The story is presented more thoughtfully than the thumbnail would make you believe, and went into places I did not expect these kinds of videos to go: corporate culture, regional geopolitics, software onboarding. Also, in the end, it would be extremely reductive, but also not fully inaccurate to say: the tragedy wouldn’t have happened if the UI of one piece of software didn’t try to help the user. What resonated most with me was the argument that wasn’t actually stated: that we altogether treat software, and UI design, not seriously and rigorously enough. Boeing asked its pilots to use consumer-quality UI loaded on a regular laptop, and lacked the imagination for what could be the consequences of doing things this way. (Thank you to Ben Zotto for sharing the video with me.)

0 views

Extending BlogShake with Per-Tag Atom Feeds

This is an external post of mine, originally published at Extending BlogShake with Per-Tag Atom Feeds .

0 views
Sean Goedecke Yesterday

Don't anthropomorphize big tech companies

Bryan Cantrill has a great line from a talk where he cautions against anthropomorphizing 1 Oracle CEO Larry Ellison. I think it’s also a mistake to treat any tech company like a person. Here’s the full quote: You need to think of Larry Ellison the way you think of a lawnmower. You don’t anthropomorphize your lawnmower, the lawnmower just mows the lawn, you stick your hand in there and it’ll chop it off, the end. You don’t think ‘oh, the lawnmower hates me’ — lawnmower doesn’t give a shit about you, lawnmower can’t hate you. Don’t anthropomorphize the lawnmower. Don’t fall into that trap about Oracle. Here are some examples of anthropomorphizing companies: All of these are mistakes. Your relationship to the tech company that employs you is not like a relationship with a person. It’s more like your relationship with your car 2 . If you drive poorly or fail to change your oil, your car may stop working or even kill you, but it doesn’t love or hate you. It has no sense of fairness. It’s simply a dangerous, useful machine: useful enough to be worth getting into despite the risks. Anthropomorphizing it makes it harder to avoid those risks 3 . Why is it so hard to avoid this trap? First, humans anthropomorphize everything by default. Second, whenever you’re interacting with the company, you’re interacting with a person: your manager, your colleague, someone in HR, and so on. Third, when you’re in the power of some entity, it’s psychologically comforting to think of it like another human. This is the same principle that caused early humans to attribute lightning to the power of Zeus, because Zeus (unlike electrical imbalances in storm clouds) can be placated with prayer and sacrifice. Relationships with machines are very different from relationships with people. The biggest difference is that machines do not respect effort . If you put a lot of work into doing a favor for a friend, your friend will appreciate it even if the favor itself is not a big deal. Conversely, if you help somebody out at trivial cost to yourself, that help will be relatively unappreciated. But a car doesn’t care how much effort you put into maintaining it. If you spend hours replacing a car’s engine, but the brake line is damaged, the car won’t remain functional out of respect for your effort on the engine: instead, the brakes will fail, killing you. Tech companies work like this. Everyone who’s worked at one for long enough knows that it’s very hard to predict what work will be high-impact. The project you slaved over for months will often get a “nice, now what’s next” reaction, while the tool you built in fifteen minutes (or the question you answered in a Slack channel) will sometimes get wild praise and recognition. This can drive engineers a little mad, because they’re anthropomorphizing their tech company, and a person who behaved like this would be dangerously unstable. But if you treat your company like a machine, this shouldn’t be surprising at all. It’s hard to predict high-impact work, but it’s not impossible. The best way to operate at a big tech company — as I’ve said again and again — is to try and develop this skill, and to spend most of your time looking for the low-effort high-reward wins. It’s hard to do that if you’re secretly imagining your company as a person. A natural objection here is to point out that tech companies (unlike cars) are composed of people who might indeed owe you, hate you, love you, and so on. According to this view, when people say “my company hates me”, they aren’t making the mistake of anthropomorphizing a corporation, they’re using shorthand to say “my boss and skip-level hate me”. I think this is still a dangerously sloppy way to think. In my experience, engineers who think they’re hated are often not. Managers in large tech companies do hold grudges, but (particularly the higher up you get) are tolerably cold-blooded. You don’t get to senior director or VP by needlessly throwing away useful tools. I can recall a few cases 4 where engineers told me they were hated, but I later found out 5 that managers considered them dangerous or unreliable: loose cannons whose usefulness didn’t outweigh their tendency to go off the rails and cause damage. From the perspective of the engineer being forced out of the company, it feels like you’re being run out of town by people who hate you. But if you want to avoid being run out of town, it’s useful to recognize when you’re being treated as a defective tool. It’s also useful to keep in mind the difference between “my manager hates me” and “my company hates me”, because managers are often temporary. You can sometimes just wait out a bad relationship with your manager. You’ll have a slightly uphill battle with the next one — since the handoff won’t be positive — but it’s certainly possible to get back on track. If you can figure out why your manager hates you, you might also be able to fix it (or at least mitigate 6 it). Engineers who’ve learned not to anthropomorphize tech companies sometimes go too far in the other direction, by treating companies as a computer program . This is the wrong kind of machine: tech companies aren’t people, but they aren’t programs either. They operate on incentives and illegible interpersonal relationships, not on consistent logical steps. The classic example of this is the engineer who reads the company handbook cover-to-cover and makes sure they follow all the rules. When they’re looking to be promoted, they take the role guidelines and make sure they’re following them point-by-point. They take what they’re told very literally: when the company says “X is our top priority”, they internalize that. The problem with this is that handbooks, guidelines and internal communications are the legible superstructure of a massive illegible web of dependencies that make up the actual operations of any company. That superstructure will not cover the real criteria for promotions, or the rules that are designed to be broken . The “top priority” of the superstructure can rapidly shift, leaving literal-minded engineers confused and disheartened. It’s understandable to feel ill-treated by a company, or to harbor resentment — particularly since there are actual humans at the company who are involved in that treatment of you. But companies themselves don’t hold grudges, even as they force out people who they’re incentivized to force out 7 . It can even be a mistake to anthropomorphize very senior managers, whose work personas are more identified with the machine the higher up you go 8 . I suppose at the top you have Larry Ellison, as Bryan Cantrill says, who is better modeled as a lawnmower than as a human being. No relationship with an individual person at your company will save you from being careless with the machine: it will chew you up and spit you out. You need to learn how to drive the car . This is scary, because it means that sometimes you will just get punished for no reason . But it’s also good news, because it means that you can get disproportionate rewards for relatively little effort, so long as you make the right decisions. Treating as a human. I like this example better than Cantrill’s lawnmower because cars are more central to our lives, like jobs are, and are also far more dangerous. For instance, if you’re prone to thinking “my car loves me, it wouldn’t spin out when I take this hard turn on gravel”, you are more likely to crash. Though certainly not all. Grudges do exist. Of course, this is my interpretation of the situation, and could be wrong. There are a range of techniques here, ranging from “do all your paperwork defensively” to “find some out-of-the-way project” to “ingratiate yourself with your skip-level”. For instance, people who make their managers look bad, or who publicly point out the inevitable inaccuracies in legible reports, or who campaign against the current big push the company is making. I say “work personas” because these people often have a rich personal life that is largely firewalled off from their work selves. For more on how managers come to identify with the rules of the organization, I recommend Robert Jackall’s Moral Mazes , a book that has informed much of my thinking and that I will some day write a long-form review of. My company really owes me for that hard thing I did I’m fighting the good fight against my company on issue X (e.g. accessibility) and they hate me for it My company is vindictively retaliating against me because of that mistake I made My company and I are partners in the mission to make the world a better place Treating as a human. ↩ I like this example better than Cantrill’s lawnmower because cars are more central to our lives, like jobs are, and are also far more dangerous. ↩ For instance, if you’re prone to thinking “my car loves me, it wouldn’t spin out when I take this hard turn on gravel”, you are more likely to crash. ↩ Though certainly not all. Grudges do exist. ↩ Of course, this is my interpretation of the situation, and could be wrong. ↩ There are a range of techniques here, ranging from “do all your paperwork defensively” to “find some out-of-the-way project” to “ingratiate yourself with your skip-level”. ↩ For instance, people who make their managers look bad, or who publicly point out the inevitable inaccuracies in legible reports, or who campaign against the current big push the company is making. ↩ I say “work personas” because these people often have a rich personal life that is largely firewalled off from their work selves. For more on how managers come to identify with the rules of the organization, I recommend Robert Jackall’s Moral Mazes , a book that has informed much of my thinking and that I will some day write a long-form review of. ↩

0 views
Unsung Yesterday

My first visit to New York Cities

(It’s Halloween Week on Unsung, where we’re sharing scary, spooky, and creepy stories. This is my memory of the first bug that freaked me out, and perhaps the first bug I ever remember witnessing.) N.Y.C. The Big Apple walked so that Grand Theft Auto could drive. = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/my-first-visit-to-new-york-cities/1.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/my-first-visit-to-new-york-cities/1.1600w.avif" type="image/avif"> Well, not really. But the 1984’s awkwardly-titled game for 8-bit computers could be seen as a spiritual predecessor of the first GTA. Your job was to drive and walk around New York City. If you crashed into another car, your car would be towed. If you ran out of fuel, you had to use the subway – or go on foot. If you then fell under another car, you ended up in a hospital. These were the map mechanics. The overarching narrative was as complex as 1984 allowed: You were supposed to dip into various areas and buildings, and complete different minigames hiding within each. The post office hid a clone of Frogger , Central Park was a facsimile of Qix , and the subway offered shades of Mario: There were twelve areas, and I believe you had to visit and finish all of them to win the game. (You can watch this walkthrough to see someone doing it all.) The minigames themselves were hard, and driving in New York appropriately stressful, but what added to the pressure were three clocks ticking along. One was the fuel gauge in your car. The second was cash in your pocket – running out meant automatic game over. The last one? A literal clock, rendered in the upper right corner. You only had one in-game day to accomplish everything. I am not sure how popular NYCTBA was. Its reception felt mixed. Here are two modern reviews from Atarimania , clearly written from two different coasts: The most aggravating game unleashed on the [Atari] platforms that I can think of. Horrid controls and collision detection/​responses, pathetic graphics, annoying sounds. This game is more than merely the sum of its parts: The many little arcade games are embedded into a very dynamic and amusing traffic simulation, which combines the numerous, fairly challenging tasks to a very satisfying whole. Personally, I was fascinated with it, and the GTA connection above came to me not just from the vehicular carnage, or the top-down view used by GTA before it embraced the third dimension. NYCTBA also tried to be cinematic; at the beginning of the game placing you just outside New York City, so you could actually start by driving into it. In your mind’s eye, you could imagine a classic Hollywood shot revealing the city’s skyline – but realized with the limited production value the computer’s eight whole bits provided. And that one decision happened to come with the first bug I ever remember encountering as a little kid. Upon game launch, the only thing you were meant to do was to drive right into the city to begin your adventure. Even moving your joystick left had the car moving right toward the buildings. But the game’s author didn’t predict you could do another thing: Wait. Specifically, wait for over 13 minutes with your car idling, watching your fuel meter go all the way to zero. At that point, the game declares YOU ARE OUT OF GAS in a classic Atari font, followed by a familiar routine – the car disappears, and the player is ejected to continue the journey on foot. Except now, you can walk left . And what is going on to the left is truly frightening. The road ends and you encounter another city… a nightmare city of sorts, with cars changing shapes, driving into the buildings, and crashing into each other. Get run over by one of these mangled cars, and things get worse – you travel forever over a surreal 8-bit nightmare world, with random disconnected streets, to a land in the middle of nowhere. It’s a creepy mirror world, an endless universe filled with broken cities where the rules of time and space don’t apply, landscapes echoing with sounds of car collisions that never seem to cease. (I uploaded the entire experience, with sound, to YouTube .) This freaked me out so much as a kid I remember waiting for many days until trying it for the second time. Many years later, I have an inkling how this bug came to be – the programmer didn’t remap left movement to right for walking, because you were not supposed to be able to walk on the game’s initial road – and I could probably fire off an assisted emulator to understand the rest fully. Was it some overflow somewhere? Some memory corruption? Some version of Pac-Man’s kill screen ? I could probably even learn stuff from it. But I think I want to leave this alone, this strange memory of a rare bug in a forgotten game. In my head, it still feels like a fireside story, something that no one would believe actually happened, a mind-bending anomaly that frightened me some two decades before I myself got to visit The Big Apple. (To play the game yourself, go to Atarimania and click the Download button to get the ATR file, then go to this browser-based Atari 800 emulator , click on the hamburger menu, click on Boot Image, and point to the ATR file. Then, press F4 (start) – or press ⌘K, and search for Start. Then, use arrow keys to move around. Left Shift is your solitary fire button. If you press space to pause, just move the joystick again.)

0 views
Unsung Yesterday

“You hold down Control, Shift, Alt, and click Technical Support. And boom.”

(It’s Halloween Week on Unsung, where we’re sharing scary, spooky, and creepy stories.) Imagine this: It’s 8pm on a rainy Sunday night in the ’90s, and you’re bored, so you boot up the family computer looking for something to do. But you’ve already played Solitaire a bunch, and you still don’t really get how Minesweeper works. You’ve had a laugh in paint, making abstract art. So now what? Keep in mind, you’re not allowed to go onto the internet because firstly, someone in the house is already on phone call and secondly you’ve already been warned about running up the phone bill, but then you remember something a legend that had been circling around the net about a secret portal into a demonic realm. And the way to get there, so they say, was through Excel. I’m just going to leave it at that. For the rest, watch the video :

0 views
Unsung Yesterday

“If you’re still using Popcorn Time, we think it’s time to move on.”

(It’s Halloween Week on Unsung, where we’re sharing scary, spooky, and creepy stories.) From The Verge : Remember Popcorn Time , the app that served as an easy-to-use front end for streaming torrents of movies and TV shows? Development of the MPAA’s least-favorite app of the 2010s has been abandoned for years , but the “Netflix of Piracy” software is still installed on millions of devices that are just waiting to join a bot army. Researchers at Eye Security found the updater for a Popcorn Time fork called “Time4Popcorn” still running on tons of Windows, macOS, and Android devices located around the world, phoning home to now-abandoned web domains over unencrypted HTTP and running an outdated, insecure version of Chrome. […] Eye Security’s Niels Teusink says the updater could have been used to install and run basically any kind of software with full privileges, easily enabling someone to create a botnet just by taking over one of the domains, using a man-in-the-middle attack to target specific devices, or perform surveillance by identifying the machines running the software. One security company took it upon themselves to prevent that from happening, but the way to do it was to… trick the apps into… updating themselves to death . (A more detailed information can be found on Eye Security’s website . It’s not the same story, but it reminded me of another creative way to solve a widespread problem that we talked about.)

0 views

BlackBerry Dev Alpha C

I was digging through a drawer and remembered I had this guy from college, the BlackBerry Dev Alpha C! It's a cool little device that was handed out to developers during the early days of BlackBerry 10. If you created a certain amount of BlackBerry apps, you'd be eligible for a device (and some cash)! I also got a BlackBerry Playbook for free. To be honest, neither were good devices. I remember the tablet was slow and clunky, and this guy had the worst camera you've ever seen (even back then). Unfortunately I can't get past the splash screen, the device refuses to power up without a battery (which is long gone). Still, it's a neat relic and probably decently rare at this point!

0 views
Rik Huijzer Yesterday

Verses that describe the shape of the earth

Proverbs 18:13, "He that answereth a matter before he heareth it, it is folly and shame unto him." All verses from the King James Version. Psalms 93:1, "The LORD reigneth, he is clothed with majesty; the LORD is clothed with strength, wherewith he hath girded himself: the world also is stablished, that it cannot be moved." Psalms 75:3, "The earth and all the inhabitants thereof are dissolved: I bear up the pillars of it. Selah." Psalms 96:10, "Say among the heathen that the LORD reigneth: the world also shall be established that it shall not be moved: he shall judge the people righteously." Psalms 104:5, "Who laid the foundations of the earth, that it should not be removed for ever." Proverbs 8:27, "When he prepared the heavens, I was there: when he set a compass upon the face of the depth:" 1 Samuel 2:8, "He raiseth up the poor out of the dust, and lifteth up the beggar from the dunghill, to set them among princes, and to make them inherit the throne of glory: for the pillars of the earth are the LORD’s, and he hath set the world upon them." Isaiah 11:12, "And he shall set up an ensign for the nations, and shall assemble the outcasts of Israel, and gather together the dispersed of Judah from the four corners of the earth." Isaiah 40:22, "It is he that sitteth upon the circle of the earth, and the inhabitants thereof are as grasshoppers; that stretcheth out the heavens as a curtain, and spreadeth them out as a tent to dwell in:" 1 Chronicles 16:30, "Fear before him, all the earth: the world also shall be stable, that it be not moved." Job 26:7, "He stretcheth out the north over the empty place, and hangeth the earth upon nothing." Job 26:10, "He hath compassed the waters with bounds, until the day and night come to an end." Job 28:24, "For he looketh to the ends of the earth, and seeth under the whole heaven;" Job 37:3, "He directeth it under the whole heaven, and his lightning unto the ends of the earth." Job 37:18, "Hast thou with him spread out the sky, which is strong, and as a molten looking glass?" Revelation 7:1, "And after these things I saw four angels standing on the four corners of the earth, holding the four winds of the earth, that the wind should not blow on the earth, nor on the sea, nor on any tree."

0 views
Kev Quirk Yesterday

2026-10-24 19:47: Hooman. I believe it is dinner time.

Hooman. I believe it is dinner time. Thanks for reading this post via RSS. RSS is ace, and so are you. ❤️ You can reply to this post by email , or leave a comment .

0 views
Evan Hahn Yesterday

List of Vim forks

Here are the Vim forks I know about: Neovim : “a Vim-based text editor engineered for extensibility and usability, to encourage new applications and contributions.” Vim Classic : “a stable, long-term support fork of the ubiquitous text editor, maintained without the assistance of generative AI tools.” EVi : “a hard-fork of Vim v9.1.0 (Jan 2024) before AI was used in the project.” MacVim : “a macOS version of the Vim text editor, providing a graphical user interface for Vim, while tightly integrating with macOS and providing features specific to the platform.” This list doesn’t include Vim-inspired editors like Kakoune , Vim-style plugins for other editors like IdeaVim , or Vim “distributions” like LazyVim . Contact me if you know of other forks! Neovim : “a Vim-based text editor engineered for extensibility and usability, to encourage new applications and contributions.” Vim Classic : “a stable, long-term support fork of the ubiquitous text editor, maintained without the assistance of generative AI tools.” EVi : “a hard-fork of Vim v9.1.0 (Jan 2024) before AI was used in the project.” MacVim : “a macOS version of the Vim text editor, providing a graphical user interface for Vim, while tightly integrating with macOS and providing features specific to the platform.”

0 views
Farid Zakaria Yesterday

Dafuq is a deterministic VM?

The life of a system call inside Rewind VM. Since I started writing about Rewind VM , the question I get most from friends is not “does it work?” but “how does it actually work?” 🤔 D’uh! It is a virtual machine that only gets interrupted when it asks to be . That seems to not have helped much, so let’s go deeper. We will follow a tiny C program through the machine and, at each step, look at where non-determinism would have crept in and what Rewind does about it. Aside: If you have not seen Rewind before, start with a flaky build you only catch once . CPUs must be deterministic. If they were not, we would have a very bad time trusting them to do anything. Give a CPU the same registers, the same memory and the same instructions, and it computes the same result. What does change? Why are machines not deterministic? They are not closed systems. They interact with the outside world and that interaction is not deterministic. A timer interrupt can land at any instruction or a disk read can complete at any time. Aside: Computers can’t actually make random numbers. A deterministic machine can only fake it with pseudo -randomness: run the same algorithm from the same seed and you get the same numbers back. The real stuff has to come from the physical world. My favorite example is how Cloudflare keeps a wall of 100 lava lamps in its San Francisco office with a camera pointed at it, feeding its encryption keys. If we could capture all external influences, we could replay the external world and get the same run again. That is what Rewind does. Every source from the outside world is either removed or replaced with a value derived from a stored seed. Time, for instance, is a clock managed by Rewind. Interrupts arrive only at points we choose. ⏱️ One more outside influence is easy to miss: other CPUs. On two cores, which thread wins a race comes down to cache misses and pipeline stalls, so Rewind provides a single virtual CPU and the threads take turns on it. To control the outside world, Rewind needs moments where the software stops and Rewind can step in. At those moments it writes down what just happened, hands over the current time, or delivers an interrupt. So how do you stop the software at the same place on every run? One answer is to make the hardware do it. CPUs have performance counters that can interrupt after a set number of instructions. Unfortunately, the interrupt may arrive a few instructions late, and the amount varies. 1 Rewind uses a cooperative approach: let the software tell us when something happens! Rewind runs the guest under KVM, the hypervisor built into Linux. The guest’s code runs directly on the real CPU until it touches a device. The CPU then stops the guest and hands control to the program running the VM, Rewind’s monitor ( ). That hand-off is a VM exit . Rewind patches the guest kernel so that whenever it needs something only the outside world has, such as the time, a timer, or “I have nothing to do”, it makes an exit and asks the monitor. Reporting what happened, like a process starting or a line of output, is an exit too. Each exit the guest makes is called a step , and they are numbered starting from boot. If we assume the guest is deterministic up to step N , step N happens at the same instruction on every run. The monitor’s loop, sketched in Python: normal VM a different instruction each run run 1 timer run 2 timer Rewind step N, every run run 1 (recording) step N run 2 (replay) step N an exit the guest made: one step the monitor's interrupt each line is the guest's instructions, left to right This is true on the first run too. Nothing gets recorded, and a replay just runs the same inputs again. Three pieces make this work, and your program doesn’t need to know about any of them. A patched Linux kernel. The guest kernel is the piece that cooperates. A small patch teaches it to take its clock, its timer and its idle loop from the monitor instead of from hardware, and to report what processes do: exec, exit, output, files written. The monitor also hides the CPU features that would leak the outside world in, such as the hardware cycle counter and the random number instruction, so the kernel never uses them. rewind-init. The special PID 1 is a small program that mounts the input image (a Nix closure or a root filesystem), runs the command, reports its exit status and powers the machine off. The image is mapped straight into guest memory, so reading a file is reading memory, and there is no disk to finish a read at a random moment. rewind-vmm. The monitor owns time, decides when interrupts go in, and keeps the trace of everything the kernel reports. Inputs Nix closure or rootfs → erofs Guest VM · 1 vCPU on stock KVM Code runs natively on the real CPU Build / test processes nix build, make check rewind-init PID 1, mounts erofs Patched Linux + Rewind platform Clock, timer, idle and events go through the host VM exit = one step Interrupt only at an exit Host: rewind-vmm Owns time, ordering and memory Virtual clock exits + branch count Event trace processes, files, output Keyframes dedup page store Seeded randomness Same inputs + seed → same run The life of a system call Here is the program. It reads the clock, draws a random number, prints both with the address of a stack variable, sleeps for a second, then reads the clock again. On my laptop every run prints something new. In Rewind, every run prints the same thing, down to the nanosecond, until you change the seed: Same inputs, same run id, same 432 steps. Oh, and the program slept for a second, yet the whole VM finished in 0.39 seconds. 🤓 Normally, reading the time never enters the kernel: a bit of code in every process reads a hardware counter directly. Rewind hides that counter, so the read goes to the kernel, and the kernel asks the monitor. guest host hello user space guest kernel Rewind platform KVM host kernel rewind-vmm host process syscall no hardware counter read the clock an I/O port VM exit made by the guest count a step clock += 5 µs + work leave the time in memory timer due? interrupt run the guest again resume the guest return the time …400.001006990 Rewind hides the CPU’s random number instruction, so the only thing available to the guest kernel is the seed Rewind passes at boot. The kernel’s random number generator is seeded with that value, so every run of the same program with the same seed gets the same random numbers. Most system calls never leave the guest. The few that need the outside world make an exit, and the monitor writes down what happened. The guest kernel also reports when it is idle, so the monitor can jump its clock to the next timer interrupt. 2 Control those few doors and the guest is deterministic: the outside world shows up at the same step on every run. So, dafuq is a deterministic VM? Like I said at the start, it’s a VM that only gets interrupted when it asks to be. Hopefully that makes a bit more sense now. 😅 Intel’s counter is good enough that you can single-step the rest of the way to the exact instruction. I have an AMD laptop. 🙃  ↩ costs no real time. The monitor jumps its clock straight to the deadline and delivers the timer interrupt.  ↩ lock both reach the lock at once the winner depends on the cache Rewind: one vCPU vCPU A B A exits threads switch only at an exit the same order every run A step is a VM exit To control the outside world, Rewind needs moments where the software stops and Rewind can step in. At those moments it writes down what just happened, hands over the current time, or delivers an interrupt. So how do you stop the software at the same place on every run? One answer is to make the hardware do it. CPUs have performance counters that can interrupt after a set number of instructions. Unfortunately, the interrupt may arrive a few instructions late, and the amount varies. 1 Rewind uses a cooperative approach: let the software tell us when something happens! Rewind runs the guest under KVM, the hypervisor built into Linux. The guest’s code runs directly on the real CPU until it touches a device. The CPU then stops the guest and hands control to the program running the VM, Rewind’s monitor ( ). That hand-off is a VM exit . Rewind patches the guest kernel so that whenever it needs something only the outside world has, such as the time, a timer, or “I have nothing to do”, it makes an exit and asks the monitor. Reporting what happened, like a process starting or a line of output, is an exit too. Each exit the guest makes is called a step , and they are numbered starting from boot. If we assume the guest is deterministic up to step N , step N happens at the same instruction on every run. The monitor’s loop, sketched in Python: normal VM a different instruction each run run 1 timer run 2 timer Rewind step N, every run run 1 (recording) step N run 2 (replay) step N an exit the guest made: one step the monitor's interrupt each line is the guest's instructions, left to right This is true on the first run too. Nothing gets recorded, and a replay just runs the same inputs again. Three pieces make this work, and your program doesn’t need to know about any of them. A patched Linux kernel. The guest kernel is the piece that cooperates. A small patch teaches it to take its clock, its timer and its idle loop from the monitor instead of from hardware, and to report what processes do: exec, exit, output, files written. The monitor also hides the CPU features that would leak the outside world in, such as the hardware cycle counter and the random number instruction, so the kernel never uses them. rewind-init. The special PID 1 is a small program that mounts the input image (a Nix closure or a root filesystem), runs the command, reports its exit status and powers the machine off. The image is mapped straight into guest memory, so reading a file is reading memory, and there is no disk to finish a read at a random moment. rewind-vmm. The monitor owns time, decides when interrupts go in, and keeps the trace of everything the kernel reports. Inputs Nix closure or rootfs → erofs Guest VM · 1 vCPU on stock KVM Code runs natively on the real CPU Build / test processes nix build, make check rewind-init PID 1, mounts erofs Patched Linux + Rewind platform Clock, timer, idle and events go through the host VM exit = one step Interrupt only at an exit Host: rewind-vmm Owns time, ordering and memory Virtual clock exits + branch count Event trace processes, files, output Keyframes dedup page store Seeded randomness Same inputs + seed → same run The life of a system call Here is the program. It reads the clock, draws a random number, prints both with the address of a stack variable, sleeps for a second, then reads the clock again. On my laptop every run prints something new. In Rewind, every run prints the same thing, down to the nanosecond, until you change the seed: Same inputs, same run id, same 432 steps. Oh, and the program slept for a second, yet the whole VM finished in 0.39 seconds. 🤓 Normally, reading the time never enters the kernel: a bit of code in every process reads a hardware counter directly. Rewind hides that counter, so the read goes to the kernel, and the kernel asks the monitor. guest host hello user space guest kernel Rewind platform KVM host kernel rewind-vmm host process syscall no hardware counter read the clock an I/O port VM exit made by the guest count a step clock += 5 µs + work leave the time in memory timer due? interrupt run the guest again resume the guest return the time …400.001006990 Rewind hides the CPU’s random number instruction, so the only thing available to the guest kernel is the seed Rewind passes at boot. The kernel’s random number generator is seeded with that value, so every run of the same program with the same seed gets the same random numbers. Most system calls never leave the guest. The few that need the outside world make an exit, and the monitor writes down what happened. The guest kernel also reports when it is idle, so the monitor can jump its clock to the next timer interrupt. 2 Control those few doors and the guest is deterministic: the outside world shows up at the same step on every run. So, dafuq is a deterministic VM? Like I said at the start, it’s a VM that only gets interrupted when it asks to be. Hopefully that makes a bit more sense now. 😅 Intel’s counter is good enough that you can single-step the rest of the way to the exact instruction. I have an AMD laptop. 🙃  ↩ costs no real time. The monitor jumps its clock straight to the deadline and delivers the timer interrupt.  ↩

0 views
DHH Yesterday

Seven years from China

I first went to China in 2011, racing for the Lotus/Jetalliance team at Zhuhai in the Intercontinental Le Mans Cup. It was a delightful experience, but also a glimpse of China's gritty side. Zhuhai had an unapologetically industrial look, with a permanent, low-hanging haze. The parade through the city felt like the crowd had been instructed to attend on pain of... consequences. Brutalist aesthetics through and through. From 2013 through 2019, I raced nearly every year at the Shanghai International Circuit in the World Endurance Championship. The skyline got a little brighter with every visit, but the smog remained thick. I still recall a taxi driver straddling lanes all the way to the circuit before blasting straight through a red light, horn blaring. Exciting, but in a very frontier kind of way. Then Covid hit, the WEC stopped going to China, and so did I. For seven years, I watched China's rise from the West. First the electronics, then the services, and finally the cars. Even from afar, it was clear that China was accelerating at a completely different pace from the West. But until last week, when I visited Shenzhen, Hangzhou, and Beijing, it was hard to appreciate just how much the country had changed since my last visit. It might as well have been forty years. There are still pockets of that gritty look, but the main show is now a gleaming, shiny set of cities radiating The Future. From the bullet trains to the lavish airports, from the impressive skylines to Hangzhou's West Lake. Gone is not just the smog, but the sound of diesel engines too. I've never seen any country change so much, so quickly. In Shenzhen, they talk about "Shenzhen Speed" (like three floors of a skyscraper in a day). It looks like the entire country has been running at that clock frequency for the past decade. And they have all the results to show for their efforts. Those efforts are clearly considerable. I repeatedly got replies late into the night and early in the morning during our stay. The pace, and the expectations that came with it, felt almost as jarring compared with America as America's pace does compared with Europe. Everyone was just on it, all the time. I'm sure that has a cost. The 9-9-6 culture (9am to 9pm, six days a week) isn't free. But it's also obvious that all this effort has been buying results. Extraordinary results. Take Xiaomi. I visited their nearly fully automated car factory in Beijing, and it looked like something straight out of a sci-fi movie. I've done a lot of car-factory tours in Europe, and never seen anything like this. The production lines we saw were almost entirely devoid of humans, and the few who were milling about were mostly holding laptops and clipboards, overseeing the robotized lines. The company has gone from launching its first car in 2024 to now delivering over 40,000 cars per month! More than 800,000 cars delivered in under three years. And a halo car in the SU7 Ultra that has over 1,500 hp, lapped the Nürburgring in 7:04, and costs just $75,000 in China. And that's perhaps the biggest material change from the last time I was there: those cars. Long gone are the Soviet-style taxis, replaced by a dizzying array of Chinese EV brands that don't just rival the best of the German giants in style and comfort, but surpass them. We kept getting into one luxury electric SUV after another, from brands I'd never heard of. Sumptuous leather, quiet cabins, and reclining captain's chairs. With Didi, the rides were as little as $5. Then at one hotel, they sent a classic Mercedes MPV. Gas-powered too. It felt like a big step back into the past. What would be a very nice car by most Western standards suddenly seemed cheap by Chinese standards. Crude, even. But what stood out even more than the material differences from Europe were the cultural ones. Everywhere we went, people were not just incredibly hospitable, but genuinely excited to see us and meet. From executives at major tech companies to students at the GOSIM conference, it was just endless great vibes, great interest, and a deeply informed curiosity. Sure, I came as a bit of a niche celebrity. REWORK has sold over 100,000 copies in China, GOSIM was full of programmers with experience using Ruby on Rails, and just about everyone was super excited about the age of agents and Omarchy's role in it. But it's not like those conditions aren't present in the West! And still, this just felt different. Much more optimistic, much more ambitious, much more driven. The contrast is fresh too. Speaking with large American tech companies about Omarchy, then comparing the quality of those conversations, and the pace of the follow-through, with their Chinese counterparts is eye-opening. Whether it was Alibaba, Tencent, Xiaomi, Huawei, or smaller players like Beelink. Very different. And also, yes, tourists are almost guaranteed to get a tinted view of a country through the rose-colored glasses that always accompany a trip as short as a week. China clearly has its own challenges, economically, politically, and culturally. Like any place does. But it's not at all obvious that those challenges are bigger than what we're dealing with in the West. On the contrary, I'd say. I could even imagine plenty of people in stagnating countries like the UK or France who might not be as bothered by one-party rule as they once were, if it meant a booming economy, national giants sprouting everywhere, and a set of safe, clean cities on the ascent. As a man of the West, that's a tough pill to swallow. Especially since, while China may well be under one-party rule, the competitive intensity in many of its markets is far greater than what we can brag about in the West. And nearly all the major companies I visited were led by engineers, not MBAs. Difficult not to be a little envious of at least that! But in terms of the international balance of power and trade, it doesn't actually matter that much what our opinions are. What matters is what is, and that's a formidable country producing remarkable products and services at extremely competitive prices. There's a lot of work ahead of Europe and America to regain a competitive footing in all these industries where China is now accelerating and ascending. There's a version of this where it all plays out wonderfully to everyone's benefit. The world gets better, faster, cheaper products and services. The trade imbalances are corrected as Europe and America catch up, reindustrialize, robotize, and revitalize their economies and political systems. Equal players, equal partners. But there's also a version of this where the West doesn't heed the competitive warnings, and continues to disadvantage itself through eco nonsense, long-term protectionism, and a failure to kickstart moribund growth. I don't think anyone, even China, is interested in seeing where that leads. Either way, we need more people in the West to realize what time it is. It's not enough to just see the consequences when they arrive in the form of shuttered factories or bankrupt businesses. Not only is it then too late, but it's also hard to understand exactly why, because it remains abstract and numerical. That Xiaomi assembly line is very concrete. So is going from your first car in 2024 to delivering over 40,000 a month just a couple of years later. And that's just one obvious example. For Europe in particular, the other is AI. Every single major company I visited had its own models. And China has a whole slew of open-weight labs that are focused just on that too. What does Europe have? Mistral? The easy answer to all of this is just to go dooming. It's so over! We're never going to catch up! The West, and Europe in particular, just can't meet the challenge! Nonsense. Every day is a new chance to make different choices. The world has seen many great empires rise and fall. Some of them more than once. I have complete faith in the possibilities of the West. In the 1980s, France built more than 40 nuclear reactors! Mercedes-Benz once built the W126. Yes, that's the past. But the present also includes America leading the AI revolution, still dominating in software, and being the home of SpaceX and Tesla. Don't you dare black pill on what's possible in the West! But enough about the West. This is an account of China, first and foremost. And I've simply been blown away by the ambition, spirit, and scale of what's going on over there. I had an absolutely wonderful time, and I'm excited to go back soon.

0 views

Continuing at the beginning

After hiking up to the end of the ridge, it was now time to scout the initial part. The starting point is some 25km east of where I parked the car last time around, at the parking spot of the Italian ossuary in the lovely town of Kobarid. The plan is to walk my way up till the beginning of the ridge, reaching Starijski Vrh at the foothills of Mount Stol, the highest point of the entire ridge, at 1673 meters. The reason why I’m doing this hike is that I read conflicting information about the trail, with some people claiming it’s a slippery mess. So I wanted to come check it out before hiking the whole ridge from one side to the other. And I’m happy to report that the trail is nowhere near as bad as some people were claiming. It’s not super well maintained, especially the middle section that’s a bit overgrown, but it’s on par with many of the trails I walk here in my area. So that’s good. There are some steep parts that are just a bit slippery, but it’s honestly nothing crazy and slipping here wouldn’t be the end of the world anyway. The overall scenery is not much different from what I have all around me here where I live, and that makes sense since we’re still in the same overall area. Kobarid is less than an hour away from where I live, and that’s only because there are a few valleys separating us. If this were flat ground, I could probably be there in 15 minutes. I’m looking at the map, and it’s about 12km north of here in a straight line. It took me about two hours to gain the roughly 800 meters of elevation from down to the parking spot to up here at the beginning of the ridge, and that’s even including the 10 minutes I spent walking around the ossuary and my quick detour to explore some trenches on my way up here. This can probably be done in almost half as much time on a normal day, I’d say. We’ll see how long it will take me when I come back to walk the whole thing. The view from up here didn’t disappoint though. This place is exactly how I imagined it when looking at it from afar. Saw some sheep roaming around, enjoying the sunny day, and there’s a lovely view of Mount Krn , with its very distinctive shape. Another thing that seems to never be missing during these walks is paragliders up in the sky. There’s a takeoff location not too far from here and the guys in the sky today were flying so low that I managed to easily exchange greetings with a few of them. The plan for the day was to walk as little as possible on top of the ridge. I only wanted to check the uphill part and then see the scenery from above and check the conditions of the trail. And everything is in perfect condition, so there’s nothing more left to do on my checklist. It’s only a matter of deciding when to come walk this whole thing. I came back down through a trail that runs on the north side of the ridge. The first part comes down through a very steep section inside the forest and then it connects with a service road that gently descends back down to then reconnect with the beginning of the trail I walked on my way up. These areas are primarily pasture lands, and in fact I had to go through more than a couple gates on my way back to the car. Animals here are usually pretty chill; they’re used to people, but one cow in particular was not happy to see me there and so I made sure to quickly move past her. Last thing I need is to be chased by a pissed-off cow. And about an hour and a half later I was out of the trail, and back at the ossuary. This was a very relaxing hike, I have to say, and I should probably explore more of this part of the country. There’s another ridge that runs parallel to the one I want to walk that also looks quite interesting. And it’s a lot shorter, so who knows, maybe I’ll go walk that as well. As usual, the data recorded by my watch is available if you’re interested in that kind of stuff. And if there’s anything you want to know about this or any other walk I did just send me an email or a message. I’m always down to talk hiking and walking around. You love the outdoors and RSS. You're one of the special ones.

0 views
Ahead of AI Yesterday

Building Reasoning Models from the Ground Up: A Full Video Course

I have written a lot about reasoning models here over the last two years on Ahead of AI, including a recent article on mechanisms to control reasoning effort levels in contemporary LLMs . Reasoning models are still as relevant as ever. In fact, almost all current-generation LLMs today are reasoning models or offer a reasoning mode. For those who are interested in how reasoning models work, including inference scaling, reinforcement learning, and distillation, I wrote my recent Build Reasoning Model (From Scratch), which implements these from the ground up in PyTorch, which, in my experience, is still the best way to understand and learn how things really work. Now, in the last few months, I also sat down and recorded accompanying lectures I wanted to share with you. So, below, you will find eight reasoning model-from-scratch coding videos (11 hours of content). We start with a pre-trained LLM and implement methods to evaluate and improve it in Python and PyTorch. So, if you enjoyed my Coding LLMs from the Ground Up course last year, this is essentially the sequel (but you can also watch it without the prior course as a prerequisite). I’m currently taking a short break and will be back soon with more exciting LLM research roundups next month. In the meantime, I hope you enjoy coding along! As a small side note, and since I often get asked how reasoning models relate to today’s AI agents, here’s a short section on this before getting started with the videos. A reasoning model is essentially an upgrade from a conventional LLM, that is, adding more post-training via reinforcement learning with verifiable rewards (and optionally inference scaling techniques) to make it more capable of solving more challenging problems. Classically, reasoning models are developed for math and code, but the capabilities can generalize to other problem domains as well. A reasoning model then functions as the engine inside an agent harness, where the agent harness is a software or tooling framework around the LLM/reasoning model to tackle even more challenging problems (bigger coding tasks, travel booking, and all other sorts of personal assistant tasks). The task of the harness is sophisticated context management and giving the LLM access to tools. Figure 1: An engine analogy for conventional LLMs, reasoning models, and agent harnesses. Figure from my Components of a Coding Agent article. In this analogy depicted in the figure above, the model is the engine, and the agent harness adds the machinery that lets us use it for various things (as an upgrade from a simple chatbot interface). An agent can use either a conventional LLM or a reasoning model, but reasoning models are the more typical choice as they are more capable. (As mentioned above, most current-gen LLMs are reasoning models with variable reasoning effort settings.) So, the bottom line is that reasoning models, as the “engine,” remain an important part of understanding modern LLM agents. The resources accompanying these topics shown above are: The Build a Large Language Model (From Scratch) book and videos The Build A Reasoning Model (From Scratch) book (and videos below) The Components of a Coding Agent article on Substack The videos below focus on developing the reasoning model that can power such a system. The reasoning-from-scratch videos focus on a small pre-trained Qwen3 model, where we implement the reasoning methods ourselves in Python and PyTorch. Starting with a pre-trained model lets us focus on the interesting part: evaluation, inference-time scaling, reinforcement learning, and distillation. Regarding the prerequisites, I recommend being comfortable with Python and PyTorch. I designed the reasoning materials as standalone content, so the previous Build A Large Language Model (From Scratch) content is recommended but not strictly required. Compute-resource-wise, you can work through the early text-generation and evaluation examples on a CPU. For the later experiments, especially training, a GPU makes the work much more practical. The first video covers the setup and some more recommendations; the repository also includes hardware and environment recommendations . Figure 2: An overview of the topics being covered. These videos accompany my book, Build a Reasoning Model (From Scratch) . If you like to read through the explanation in more detail and at your own pace, work through exercises, and so on, I recommend the book as a companion here. For the most in-depth learning experience, my suggestion is to read a chapter, work through its code alongside the corresponding video, and then try the exercises. The book is available from Manning and Amazon , and the print edition is in color! Figure 3: A random excerpt from the Build a Reasoning Model (From Scratch) print book. Thank you to everyone who has picked up a copy or recommended it to a friend. Your support helps me keep creating these materials. As a last, quick side note before getting to the main videos below: For a shorter, more conceptual overview, I also recommend my AI Reasoning Models course on LinkedIn Learning (it should be free for 24h when accessing it via the link there, which shouldn’t then require a LinkedIn Premium subscription). The course is about 90 minutes and explains how reasoning models work, covering the main ideas behind inference-time scaling, reinforcement learning, and distillation. It’s a good optional preview if you’d like the main ideas without working through the full implementations below. To get to the main part of this article, here are the eight main lecture videos. The first video starts with an explanation of the relationship between conventional LLMs, reasoning models, and agents. Next, the main topic of the video is setting up the coding environment. This includes installing the Python and PyTorch dependencies (with uv), among others. I also discuss hardware choices and common setup issues. Topics covered The relationship between conventional LLMs, reasoning models, and agent harnesses On the benefits of learning through code implementations The scope of this video series Installing Python dependencies Choosing hardware and checking CUDA or Apple Silicon support Using a remote GPU with VS Code and troubleshooting the environment Watch video 1 · Setup instructions Before adding reasoning methods in the upcoming videos, we need a working base model. Instead of training it from scratch, since that would be a whole other book (see Build A Large Language Model (From Scratch) ), we start with a pre-trained model. Here, we load the pre-trained weights of a Qwen3 0.6B model. Then, we implement the text-generation process via a streaming text generator and add KV caching and model compilation to make text generation more efficient. These implementations give us the baseline and base components we will evaluate and improve in the following videos. Topics covered Encoding text as tokens and decoding it back into text Downloading Qwen3 weights and loading the model on the selected compute device Going over tensor shapes, logits, and next-token predictions Implementing greedy decoding and streaming text generation Handling end-of-sequence tokens and measuring generation speed Adding a KV cache to reuse attention keys and values Combining KV caching with and comparing hardware performance Watch video 2 · Chapter 2 code In this lecture, we build a math-answer verifier step by step, from answer extraction to normalization and checking mathematical equivalence. We then use it to evaluate the model on MATH-500. We use this verifier to evaluate the base model’s performance on the MATH-500 evaluation dataset. However, the verifier also has a second job later in the series: providing the reward signal for reinforcement learning with verifiable rewards (RLVR). Topics covered Intro to verifiers and how they fit into LLM evaluation methods (other methods are covered in the book’s appendix) Reusing answer verification as a reward signal for RLVR (more on that in chapter 6) The verifier flow, including extracting final answers, normalizing answers and checking mathematical equivalence Building an answer grader and an evaluation loop for MATH-500 Comparing base and reasoning models on MATH-500 Watch video 3 · Chapter 3 code Now we explore ways to improve answers while keeping the model weights fixed. But before implementing specific inference-scaling techniques, we first implement methods to generate diverse answers from the same prompt (this is standard practice in proprietary LLM chatbots, such as ChatGPT). Concretely, we implement temperature scaling, multinomial sampling, and top-p filtering to generate diverse responses. Then we combine multiple responses through self-consistency, using majority voting over their final answers. We also examine the tradeoff between answer accuracy and the extra computation needed to generate more candidates. Topics covered Comparing how additional compute benefits model performance during training and during inference A gentle intro to inference-scaling through chain-of-thought prompts Implementing temperature scaling and multinomial sampling to generate diverse answers Adding top-p filtering to control the quality of the diverse answers Combining candidate answers through self-consistency, one of the most popular inference scaling techniques Evaluating MATH-500 accuracy against the cost of additional inference compute Watch video 4 · Chapter 4 code Building on the previous video, we implement scoring functions to compare multiple answers generated from the same prompt. We can also use these scoring methods to break ties in self-consistency (covered in the previous lecture). The scoring methods covered here include computing token and sequence log probabilities (this will also be relevant in the later reinforcement learning chapters). We then implement a self-refinement loop in which the model critiques an answer and attempts a revision. The evaluation lets us check how useful those additional steps are for our model. Topics covered Using scoring functions for comparing multiple answers and breaking ties Implementing a simple heuristic scorer Computing token probabilities and sequence likelihoods (also relevant for the following lectures) Using log probabilities for improved numerical stability Building a self-refinement loop for generating critiques and revised answers Evaluating self-refinement on MATH-500 Watch video 5 · Chapter 5 code In this lecture, we move on to implementing a training technique called Reinforcement Learning with Verifiable Rewards (RLVR). We first implement a simplified version of Group Relative Policy Optimization (GRPO), generate groups of responses, score them with our verifier, and use their relative rewards for the model weight updates during training. Topics covered Generating reasoning traces and understanding the relationship between answer lengths and problem-solving abilities Discussion and comparison of two important LLM post-training methods: Reinforcement Learning with Human Feedback (RLHF) and Reinforcement Learning with Verifiable Rewards (RLVR) Turning verifier scores into rewards and relative advantages Computing sequence log probabilities and a simplified GRPO loss Assembling the training loop with logging and checkpoint saves Evaluating checkpoints and checking training stability and memory needs Watch video 6 · Chapter 6 code Now that we implemented the training loop in the previous lecture, we can revisit it and analyze the training runs in more detail. Concretely, this video adds more training metrics for inspection and analysis, including entropy and advantage statistics, clipped policy ratios, KL loss term values, and formatted rewards. We then review the training logs and evaluation results to understand what these additions do and whether additions such as clipped policy ratios and KL loss terms are helpful. Topics covered Running longer training experiments and interpreting their logs Tracking advantages and entropy to understand training behavior Implementing clipping in the policy loss and analyzing its effects Adding a KL loss term and discussing reward hacking Understanding limitations of the simplified KL formulation Implementing format rewards and conditioning them on answer correctness Comparing checkpoints on MATH-500 and exploring further GRPO research Watch video 7 · Chapter 7 code Finally, we explore another way to train a small reasoning model, that is, learning from answers generated by a stronger (and bigger) teacher model. We discuss different forms of distillation, prepare reasoning data, and implement the supervised training loop. We finish this lecture and course by evaluating the distilled model on MATH-500 and comparing the results. This rounds out the series with an alternative to developing reasoning capabilities through reinforcement learning. Topics covered Comparing the different combination and approaches to distillation, including soft and hard distillation, and on-policy and off-policy distillation Creating and collecting teacher-generated answers for the MATH dataset (using DeepSeek R1) Preparing the distillation dataset, including tokenization and training/validation/test splits Computing answer-token log probabilities and understanding the cross-entropy loss Implementing supervised training and tracking training and validation losses Evaluating checkpoints on MATH-500 and comparing distillation results Watch video 8 · Chapter 8 code As you watch these lectures, I encourage you to pause occasionally and try a small experiment of your own if something interesting comes to mind. For example, feel free to change the sampling settings, inspect a few failed answers, or try variations of different ideas similar to the suggestions in the exercises. And if you would like the full written walkthrough, figures, and exercises alongside the videos, you can find Build a Reasoning Model (From Scratch) at Manning and Amazon . Thanks for reading and supporting Ahead of AI. I am taking a short break to regenerate, and I look forward to being back with the next round of LLM research articles next month. In the meantime, happy viewing and tinkering! Figure 1: An engine analogy for conventional LLMs, reasoning models, and agent harnesses. Figure from my Components of a Coding Agent article. In this analogy depicted in the figure above, the model is the engine, and the agent harness adds the machinery that lets us use it for various things (as an upgrade from a simple chatbot interface). An agent can use either a conventional LLM or a reasoning model, but reasoning models are the more typical choice as they are more capable. (As mentioned above, most current-gen LLMs are reasoning models with variable reasoning effort settings.) So, the bottom line is that reasoning models, as the “engine,” remain an important part of understanding modern LLM agents. The resources accompanying these topics shown above are: The Build a Large Language Model (From Scratch) book and videos The Build A Reasoning Model (From Scratch) book (and videos below) The Components of a Coding Agent article on Substack Figure 2: An overview of the topics being covered. The companion book These videos accompany my book, Build a Reasoning Model (From Scratch) . If you like to read through the explanation in more detail and at your own pace, work through exercises, and so on, I recommend the book as a companion here. For the most in-depth learning experience, my suggestion is to read a chapter, work through its code alongside the corresponding video, and then try the exercises. The book is available from Manning and Amazon , and the print edition is in color! Figure 3: A random excerpt from the Build a Reasoning Model (From Scratch) print book. Thank you to everyone who has picked up a copy or recommended it to a friend. Your support helps me keep creating these materials. The videos As a last, quick side note before getting to the main videos below: For a shorter, more conceptual overview, I also recommend my AI Reasoning Models course on LinkedIn Learning (it should be free for 24h when accessing it via the link there, which shouldn’t then require a LinkedIn Premium subscription). The course is about 90 minutes and explains how reasoning models work, covering the main ideas behind inference-time scaling, reinforcement learning, and distillation. It’s a good optional preview if you’d like the main ideas without working through the full implementations below. To get to the main part of this article, here are the eight main lecture videos. 1. Motivation and code setup The first video starts with an explanation of the relationship between conventional LLMs, reasoning models, and agents. Next, the main topic of the video is setting up the coding environment. This includes installing the Python and PyTorch dependencies (with uv), among others. I also discuss hardware choices and common setup issues. Topics covered The relationship between conventional LLMs, reasoning models, and agent harnesses On the benefits of learning through code implementations The scope of this video series Installing Python dependencies Choosing hardware and checking CUDA or Apple Silicon support Using a remote GPU with VS Code and troubleshooting the environment Encoding text as tokens and decoding it back into text Downloading Qwen3 weights and loading the model on the selected compute device Going over tensor shapes, logits, and next-token predictions Implementing greedy decoding and streaming text generation Handling end-of-sequence tokens and measuring generation speed Adding a KV cache to reuse attention keys and values Combining KV caching with and comparing hardware performance Intro to verifiers and how they fit into LLM evaluation methods (other methods are covered in the book’s appendix) Reusing answer verification as a reward signal for RLVR (more on that in chapter 6) The verifier flow, including extracting final answers, normalizing answers and checking mathematical equivalence Building an answer grader and an evaluation loop for MATH-500 Comparing base and reasoning models on MATH-500 Comparing how additional compute benefits model performance during training and during inference A gentle intro to inference-scaling through chain-of-thought prompts Implementing temperature scaling and multinomial sampling to generate diverse answers Adding top-p filtering to control the quality of the diverse answers Combining candidate answers through self-consistency, one of the most popular inference scaling techniques Evaluating MATH-500 accuracy against the cost of additional inference compute Using scoring functions for comparing multiple answers and breaking ties Implementing a simple heuristic scorer Computing token probabilities and sequence likelihoods (also relevant for the following lectures) Using log probabilities for improved numerical stability Building a self-refinement loop for generating critiques and revised answers Evaluating self-refinement on MATH-500 Generating reasoning traces and understanding the relationship between answer lengths and problem-solving abilities Discussion and comparison of two important LLM post-training methods: Reinforcement Learning with Human Feedback (RLHF) and Reinforcement Learning with Verifiable Rewards (RLVR) Turning verifier scores into rewards and relative advantages Computing sequence log probabilities and a simplified GRPO loss Assembling the training loop with logging and checkpoint saves Evaluating checkpoints and checking training stability and memory needs Running longer training experiments and interpreting their logs Tracking advantages and entropy to understand training behavior Implementing clipping in the policy loss and analyzing its effects Adding a KL loss term and discussing reward hacking Understanding limitations of the simplified KL formulation Implementing format rewards and conditioning them on answer correctness Comparing checkpoints on MATH-500 and exploring further GRPO research Comparing the different combination and approaches to distillation, including soft and hard distillation, and on-policy and off-policy distillation Creating and collecting teacher-generated answers for the MATH dataset (using DeepSeek R1) Preparing the distillation dataset, including tokenization and training/validation/test splits Computing answer-token log probabilities and understanding the cross-entropy loss Implementing supervised training and tracking training and validation losses Evaluating checkpoints on MATH-500 and comparing distillation results

0 views
DHH Yesterday

Be a tech bro

We should all strive to be our brother’s keeper: to look after each other, inspire each other, and simply enjoy each other’s company. For men this usually requires a shared mission or interest. Mine has always been computers, so I happily consider myself a tech bro. I know the term has been used to denigrate anyone who has done well with computers or who dares to be excited about them in ways that rub some people the wrong way. But who cares? There is no reason to surrender a perfectly pleasant term of endearment just because small-minded people deploy it to discredit opinions they dislike. There is even less reason to be ashamed of an earnest interest in computers, the businesses around them, and the merit-based communities that keep forming because of them. You should beam with pride at being part of that progress. Is the label a free pass to be an asshole or a scammer? Of course not. We can simply call those people assholes and scammers. Their interest in computers, or their skill with them, is not the problem. Neither is their gender. Yes, men dominate computing. There are many reasons for that; none of them justifies collective guilt. The fact that women are a minority does not diminish the many who work in the industry, and it is more than fine to celebrate their achievements with gusto. Turning the percentages into a struggle session is nonsense. So if you are a tech bro committed to advancing technology, celebrating progress, and rejecting managed decline, you are indeed my brother. I will always want to shake your hand and cheer you on.

0 views
マリウス Yesterday

On Honor-Based Value Transfer

Disclaimer: This is an opinion piece, written from the perspective of someone who has grown steadily more uneasy and frankly tired with the amount of surveillance and compliance that modern banking now takes for granted. This is, however, not a naive claim that the system I am about to describe is somehow free of abuse, because it very much is not. It is more of an attempt to ask why we have decided that financial privacy is inherently suspect, and to make the case that a tool used overwhelmingly by criminals only stays that way for as long as honest people refuse to use it. By touching topics like this every once in a while I’m hoping to remind the average reader of a world they might have long forgotten or be even unaware it existed (and still does, at least in some parts of the world), in order to spark awareness for the fact that world isn’t necessarily the way one might think it is, and that there are other realities that might be worth exploring. As usual, summary at the end. There is a particular kind of arrangement that I have always found fascinating, because it should not work nearly as well as it does. Picture a roadside farm stand somewhere in the countryside, with a table of vegetables, a handwritten sign with the prices, and an unguarded tin box for the money. You take what you want, you drop the coins in the box, and you drive off. Nobody checked your bag, nobody filmed you, and nobody asked for your name, let alone your ID. The entire arrangement rests on a single assumption, namely that you will do the honest thing even when there is no one around to make you. That is what we call an honor system , and once you start looking for them, you notice that they are everywhere. Universities run unproctored exams on them, trains across much of Europe let you board without a gate and only occasionally check whether you actually bought a ticket, and in some places you might even find your average convenience store with unguarded self-checkout cashiers, which, while running 24/7 camera surveillance, still trust their customers at first, until there’s hard evidence that someone cannot be trusted going forward. The honor system is, at its core, nothing more than “a way of running a variety of endeavours based on trust, honour, and honesty” , without a whole apparatus of enforcement and surveillance on top of it. We tend to think of these arrangements as quaint, small in scale, and perhaps a little naive, but what I suspect most people do not realize is that one of the oldest and by far the largest of these trust systems moves money across the entire planet, and has been doing so for the better part of a thousand years. When you send money abroad through your bank, you tend to imagine your money physically traveling somewhere, but that couldn’t be further from the truth. What actually happens is a series of ledger entries. Your bank debits your account, a chain of correspondent banks adjust the balances they hold with one another, and a bank on the far end credits the recipient. No suitcase of cash ever crosses a border, and what moved was not money but value , which these days is increasingly nothing more than numbers in databases, that most of the time aren’t even backed by anything of actual value… but that’s a story for another time. Anyway, what allowed the value to move was a chain of institutions that trust each other enough to keep running tabs and settle up later. After all, as most of us have got to experience first hand during the 2008 financial crisis, almost all of finance is at its core based on trust, whether it’s trusting a government and its bonds to continue paying interest, or whether it’s trusting private individuals to not default on their mortgage. A value transfer system is simply the general name for any arrangement that moves value, and the realization at the heart of this post is that you do not actually need banks in order to build one. All you need are, first and foremost, people in two different places who are each able to pay out locally, and, secondly, enough trust between them that each is willing to be owed by the other for a while. An informal value transfer system , or IVTS , is precisely that, namely a network of people operating outside the formal banking system that, in the dry language of the regulators, “receives money for the purpose of making the funds or an equivalent value payable to a third party in another geographic location” . These systems are not some modern invention, either. They predate banks by thousands of years and exist, under one name or another, in nearly every culture that ever had both merchants and distances to bridge. The Chinese called it fei ch’ien (飞钱), which translates rather poetically to “flying money” . In South Asia it was hundi , in Thailand phoe kuan , and in Latin America the closely related Black Market Peso Exchange . And across the Middle East, Afghanistan and South Asia there is the one name that has come to stand in for all of them: The word Hawala comes from Arabic and means both, transfer and trust , and the mechanism of Hawala itself is rather simple. Suppose you are working in, let’s say, Dubai, and you want to send money home to your family in a village in Pakistan. You walk into a shop, the kind of small place that might also sell phone cards or groceries, and you hand the man behind the counter your cash. He is a Hawaladar , a Hawala broker. The two of you agree on a password, some word or number, and you then call your family and pass it along to them. The Hawaladar in Dubai contacts a counterpart of his in the town nearest your village. Your family walks into that shop, says the password, and walks out with the money in local currency, minus a small commission. And the whole thing takes maye a few hours, or sometimes only minutes. What I find truly remarkable in all of this, is not what happened, but instead what did not happen. No money crossed the border, and no promissory note, contract, or anything else legally enforceable changed hands between the two brokers. The Hawaladar in Dubai now simply owes the one in Pakistan, and the two of them will settle that debt later, between themselves, over the course of many transactions, perhaps by netting it against money flowing the other way, perhaps through trade, perhaps with a conventional wire when it happens to be convenient. As the YouTuber Johnny Harris of a recent and pretty good video essay on the subject put it, “the ledger is only trust” . That trust is anything but abstract, as it is enforced, just not by courts or regulators. It is enforced by family, clan, village and ethnic ties, and by the fact that a Hawaladar who cheats another will be cut out of the network entirely, which is an “effective excommunication and the loss of honour” that ends his livelihood and follows his name. This honor system can be scaled up until it moves billions of dollars across continents and it works for exactly the same reason the farm stand’s tin box works, with the difference that the social cost of stealing from this particular box is total and permanent. If you’re thinking that this is some fringe or marginal activity then you might be in for a surprise. In large parts of the world, Hawala is not the alternative to banking so much as it is the banking. For example in Afghanistan it has long been described as the de facto banking system of the entire country, and in Somalia, after the formal banks collapsed, Hawaladars have handled something on the order of $1.6 billion a year in remittances, which is money that keeps families fed and that keeps local economies turning over. The reasons people reach for it instead of a bank, where they even have the choice, are quite obvious once you lay them out. Hawala tends to be considerably faster, settling in hours rather than the days a bank transfer can still take, it is usually a good deal cheaper, with commissions that are a fraction of what banks and the large remittance companies charge, which matters to a migrant worker for whom those few percentage points are the difference between a child eating well or not, and it frequently offers a better exchange rate than the official one. On top of that, it reaches the places banks simply will not go, namely the rural towns, the failed states, and the refugee populations that no compliance department on earth will ever find profitable enough to serve. Last but not least, it requires almost no paperwork and, crucially, no identity documentation whatsoever. That last point is what makes regulators in the West distinctly nervous, and it is also the detail that I want to highlight, because I have come to think we have been indoctrinated educated to read it backwards. Strip away the moralizing for a moment, and what Hawala really amounts to is a parallel financial system. It runs alongside the regulated one, but it only touches it when it has no other option. Otherwise, it lets its users do something that has become almost impossible these days, at least in the western world: To transact freely , in the literal sense of these words. There is no account waiting to be frozen, no form demanding to know the purpose of the payment, no “AI” algorithm flagging the transaction as suspicious, merely because it did not fit the profile some model has built of you, and no permanent record handed over to a government that doesn’t ever ask for your permission to have it. You also don’t need to confirm that your data is still up to date every few months and it certainly won’t require you to hand over utility bills just to be allowed to work with your own money . We have arrived, more or less without noticing, at a point where moving your own money has become a privilege that the financial system grants you on fairly arbitrary and ever-changing conditions, and that it can revoke at any moment. To open an account you must submit to Know Your Customer ( KYC ) checks. To keep it, you must continuously re-verify the data and avoid tripping any tripwires, which are calibrated by people who do not know you and who are under no obligation to explain themselves to you. Banks close accounts with no reason given and no meaningful avenue of appeal. Payment processors decide, entirely on their own initiative, which perfectly legal businesses, professions, public figures, and groups, are too inconvenient to be allowed to receive money at all. All while governments float the idea of central bank digital currencies ( CBDC ) that would, by design, make every transaction you ever make visible to, and potentially programmable by, the state. I have written before about a world that increasingly insists you identify yourself before you are permitted to do much of anything, and about how the slow accretion of regulation has closed doors that used to stand open. Financial surveillance is the same story, told in money rather than purely in paperwork. Long time ago, privacy used to be the assumption and disclosure the exception, but now disclosure is the mandate and privacy is treated as inherently suspect, the sort of thing only a person with something to hide could possibly want. Held up against that backdrop, an honor-based network that lets ordinary people move value to one another without first asking anyone’s permission starts to look less like a grey area, and instead more like a relic of a saner arrangement, in which your money was actually yours. Of course, the same properties that make Hawala a gift to a migrant worker and that can make it a viable alternative to everyone else also make it equally spectacular for criminals. The very network of flying money that began as a way for Tang Dynasty tea merchants to avoid hauling heavy copper coins past bandits has, in its modern incarnation, become one of the most important criminal instruments on the planet. The aforementioned video that inspired parts of this post lays the modern version out in detail, and it shows why the system is so difficult to stop. A wealthy man in China wants to move money out of the country, past the $50,000 annual limit that the state imposes on capital leaving its borders. A Mexican cartel, meanwhile, has a pile of drug cash sitting in Chicago that it needs to get back to Mexico. A Chinese broker in the United States sits in the middle of the two. The wealthy man’s money never actually leaves China, the cartel’s cash never crosses a border, and through a series of so-called mirror transactions the two needs cancel one another out. The brokers square up afterwards, very often by over- or under-invoicing real shipments of goods (e.g. refrigerators, electronics, …) that move through entirely legitimate global trade and leave next to no trail behind them. As the investigators put it, it is sometimes far more useful to follow the value than to follow the money. These money laundering networks have, over the past decade or so, quietly become the preferred launderers for the Mexican cartels , exactly because the combination of capital controls on one side and drug cash on the other makes the mirror trade so efficient. It funds the fentanyl trade, it funds wildlife trafficking, and the totoaba bladders, shark fins and ivory that travel along the very same path as the narcotics. It is the engine behind a great deal of misery, and somewhere between three and five percent of global GDP, on the order of four trillion dollars a year, is estimated to be criminal proceeds sloshing through systems very much like this one. So no, I am not romanticizing any of this. But I do want to be specific about what , exactly, is doing the harm here. The harm is the drug trade, the trafficking, the poaching, but Hawala is just the plumbing those things happen to flow through. Blaming the plumbing for the sewage is convenient, because plumbing is so much easier to regulate than human nature is, but it is also a mistake, and it happens to be the very same error that gets wheeled out to justify surveilling everyone’s perfectly lawful transactions on the off chance of catching the handful of unlawful ones. This brings me to the argument I actually want to make. At present, the user base of these networks skews heavily towards the people who have little choice in the matter, specifically those whom the banks will not serve, and those who cannot afford the scrutiny that banking brings with it. While the first group is almost entirely legitimate, the second one is oftentimes criminal. That imbalance is part of the problem, because a system used overwhelmingly by criminals comes, in the public imagination, to be a criminal system, which in turn justifies cracking down on it, which pushes the legitimate users out first, since they are the ones with the most to lose and the least appetite for any risk at all. What I would actually like to see, is ordinary people, acting on entirely ordinary motives, making up the bulk of the traffic. People who simply believe, quite correctly in my view, that a payment between two consenting adults is nobody’s business but their own. People who would rather not have every coffee, every donation, every gift recorded and assembled into a profile, then sold, leaked, or held in reserve to be used against them later when the political weather has changed. The more such people there are in the network, the less it resembles a criminal artifact and the more it resembles what it has been all along, which is ordinary financial privacy, exercised by ordinary people. Privacy, after all, is something of a herd property. A single individual insisting on cash, or on a private transfer, looks suspicious. An entire society that treats financial privacy as the norm, however, makes it unremarkable, and in doing so turns the surveillance back into the aberration it ought to have remained. After all, the way you protect the migrant worker and the dissident is by standing in the crowd next to them, not by leaving them to be the only two faces in it. Allow me to imagine, for a moment, a world in which financial privacy were once again simply the default. You want to send a friend on the other side of the planet a little money for their birthday. You do not open an app that records the transfer, files it, profiles it, and then waits to see whether it trips some internal rule, but instead, you move the value directly, from you to them. They have it within minutes, and the only two parties who know anything about it are the two parties who ought to. A small business pays an overseas supplier without a week of compliance friction and without a cut being skimmed off by three separate intermediaries along the way, just like a charity gets funds into a disaster zone where the banks are rubble. None of these people are doing anything remotely wrong, and not one of them owes anybody an explanation. That world is not a fantasy and for several hundred million people it is simply the normal way of doing things. The only thing standing between it and the rest of us is the trust problem, which is the single weakness sitting at the heart of every honor system, and the reason Hawala never managed to scale to strangers. You can only join a Hawala network if someone already inside it is willing to vouch for you. You can only place your trust in a Hawaladar if your family has trusted his for a generation or two. The system runs on relationships, and relationships, by their nature, do not scale to a planet full of strangers who share no clan, no village, and no name in common. Especially not with a technocracy trying as hard as it can to disrupt communities and actively make us strangers … but that’s also a topic for another time. As we start to realize, the weak part of Hawala was never in the transfer itself, but it was in the settlement , in that later moment when two brokers have to trust that the debt standing between them will in fact be honored. The entire elaborate apparatus of clan, honor and excommunication exists in order to solve how one could possibly know that the other will pay them back, when there is no court anywhere that they can drag each other in front of. However, it turns out that we have solved the trust issues over a decade ago: Bitcoin, and the cryptocurrencies that followed in its wake, allow strangers to settle transcations instantly, without any intermediary involved. In Hawala the brokers hold what are effectively IOUs against one another, which are basically promises that are only worth anything for as long as the trust behind them holds. On a public blockchain, however, there are no IOUs at all. The coins are not accounting entries waiting to be settled at some later date, but they are the settlement. When value moves, it has actually moved, verified by a network that ideally no single party controls and that no broker can simply default on. The very thing that Hawala needed honor in order to guarantee, cryptocurrency guarantees instead with mathematics and a public ledger that anyone can inspect and that is impractical to forge. Put differently, cryptocurrency is Hawala with the trust problem subtracted out. It keeps nearly everything that made the old system so attractive in the first place, the speed, the low cost, the reach into the places banks ignore, the freedom from any gatekeeper’s permission, and it discards the single thing that kept the old system confined to tight-knit communities, namely the requirement that you have to belong to one. You no longer need a cousin somewhere in the network, and you also no longer need anyone to vouch for you. A complete stranger on the far side of the world can be your counterparty, because you do not need to blindly trust them at all. Vires in numeris It is no accident, then, that the criminal networks worked this out, in exactly the same way they worked out Hawala , and encrypted messaging, and nearly every other tool of financial privacy before the rest of us got to it. Investigators now describe “flying money paired with crypto, paired with encrypted communication” as a combination that is “almost unbreakable from the outside” . The tools that allow the worst people among us to transact without anyone’s permission are the very same tools that could allow the rest of us to do the same, and the more of us who pick them up for honest, unremarkable reasons, the less these networks belong, by default, to the criminals. The honor system that has been moving value across the world for the better part of a thousand years has, for the first time in its long history, been handed a way to extend its trust to people who have never once met. Whether we leave that entirely in the hands of the ghosts, or take some of it back for ourselves, seems to me a question worth thinking about. Honor-based value transfer systems, of which Hawala is the best known, have moved money across the world for the better part of a thousand years using not much more than local payouts, a password, and the trust between brokers who cannot afford to cheat one another. They are fast, cheap, and reach the people and places the formal banking system has written off, which is why they remain indispensable across much of the world. Unfortunately they are also abused on a vast scale, by money laundering networks, by cartels, and by traffickers of nearly every description. My argument, however, is that the system is plumbing , and that the reasons it appeals to criminals (speed, low cost, and above all the freedom to transact without first surrendering your identity and your privacy) are very much the same reasons it ought to appeal to ordinary, honest people in an age when banking has become an instrument of surveillance. A tool used only by criminals stays a criminal tool, and the honest users are always the first to be pushed out when the crackdown comes. Using cryptocurrency, every one of us can benefit from the advantages of the traditional Hawala system all while leaving behind the trust problem that the confined the old system to tight-knit communities. I would like to see a future in which financial freedom and privacy is being taken more seriously by society, and in which modern adaptations of these millennia-old systems are used for unremarkable everyday transactions between regular people.

0 views