Posts in Java (20 found)
Max Bernstein 1 weeks ago

Travel notes: PLDI Boulder

I had another excellent PLDI this past June. It was my fourth 1 . I continued to meet new people and learn new things! Overall: I got to meet a lot of new people, which was exciting. I had some good chats about research. I asked a question at a talk! I got to show Aaron and Jacob PLDI and see them enjoy it. I missed hanging out with CF Bolz-Tereick and Chris Fallin, the usual suspects at conferences I attend. I’m looking forward to next year. This post is more about the conference than the town of Boulder (unlike the last PLDI post about Seoul) because I didn’t do much Boulder exploring. I got in late on Sunday. Then I had to take a long car ride from Denver airport to Boulder. I don’t think I had ever flown into Denver with intent to go to Boulder before so it was a bit of a surprise. Jacob offered to have a late dinner with me so we had a tasty meal at Gaia Masala and Burger. Shout out to Harry, our server. Monday was a workshop day. I signed up for EGRAPHS and mostly stayed in that workshop. People kept throwing around the term “Knuth-Bendix”, as they have been for several years, and only in one of these workshop talks did someone explain it in a way that made any sense at all. It seems kind of like equality saturation but for the rewrite rules themselves—no actual expression graphs involved. I DMed Phil this sketchy explanation during a talk to get his response and I got to watch him cock his head and think about it in real time. At lunch I met Qiantan Hong and we got to talking about Common Lisp and its object system, CLOS. Seems like a combination of ahead-of-time compilation and multiple dispatch is really tricky. I had dinner with Aaron at Postino and then wandered into a bunch of people staying at the conference hotel chatting in the lobby. Ben Titzer said “fix my subtyping bug”, which I interpreted as him saying hi. I ended up just planting myself at the table as a bunch of interesting people cycled through: Jared Roesch, Mae Milano, Hila Peleg, Russel Arbore. It was a late evening. Back to the workshops! But late because of aforementioned late evening. I saw Vadym’s talk about Remora. I only understood about 40% of it but it was good to catch up. I hadn’t seen him since leaving Northeastern. Around a break time I joined a little cluster of people talking about e-graphs and I guessed asked enough basic questions that Pavel convinced Max Willsey to run a “BYOEG” (build your own e-graph) tutorial. The structure was as follows: Max would instruct Jacob as to what kind of thing to build next but not be prescriptive about exactly how to build it and not look at Jacob’s screen. The rest of us would sit around a table and try to follow along as best we could. I hear Pavel has a blog post about this experience coming soon… I saw Slava Pestov walking around and introduced myself because we keep liking one others’ bad jokes on Mastodon. We ended up getting dinner with Aaron and Jacob that night at Leaf. We learned a lot about monoids, Knuth-Bendix (!), Factor, and Swift. Slava volunteered to do a similar follow-along “BYOKBC” (build your own Knuth-Bendix completion) tutorial the next day. First day of the conference! I was walking into the hotel in the morning and I had made it about three feet onto the property when Alexa VanHattum, who was going the opposite direction, convinced me to instead get coffee elsewhere. We had a nice catch up and I got to hear about what teaching is like these days. Lunch was fun. I got to do another round of “ambush person whose research I admire” and plopped down with Ben and Christian Wimmer. I’d spent a lot of time struggling with Christian’s papers on linear scan, then convinced him to chat about register allocation with us on a video call a couple of months ago. We continued some of that at lunch but then I (kind of accidentally kind of on purpose) got Ben started talking about Sea of Nodes and how it is and is not different between Java and (for example stand-in for dynamic languages) JavaScript. Apparently he is thinking about a similar thing that he is calling Sea of Variables. We talked about inlining challenges and how to infuse profiles with call context, which can be a challenge. I feel more inspired to get type-based alias analysis working in ZJIT. I tracked down Christian later in the courtyard and got to hear about what he’s working on these days. I know very little about ML compilers and ML hardware and things like that so hearing about the challenges was neat. Yannis Smaragdakis joined our little standing table chat and we got to learn about Datalog. Because I had previously written about linear scan register allocation and about liveness analysis with Datalog, I goaded him into pairing with me on writing a full linear scan implementation in Datalog. This ended up taking the rest of the evening and several beers and then a lot of the next day! And after the first bit of code I did not manage to contribute very much at all. I met Hannah Gommerstadt and we got to chatting about bikes and formal methods (separately). Slava walked by and I got to introduce them. Then Jacob too. I continued pairing with Yannis but remained really lost. The only thing I think I contributed was some familiarity with the core algorithm, which he had only really seen in passing before. Eventually he got it fully working, but it needed some deep trickery. More on this soon in its own post. I saw a talk about versioned e-graphs and that got me wondering if their implementation can be used as a persistent e-graph or even just persistent union-find. Sometimes you want to do backtracking, or have undo-redo in your compiler. Then I went to a talk about streaming byte-pair encoding (BPE). BPE is hard to do streaming because it definitionally requires looking over the whole input string. They did some neat trickery to find boundaries in the string that demarcate regions that don’t interfere with one another and thereby tokenize on-the-fly. I didn’t understand it fully but I asked my only question of the conference, which was if this could also be used to implement BPE in parallel. Seems the answer is “maybe” so I should probably reach out and ask further. Slava started showing me and Jacob and Aaron how to implement Knuth-Bendix completion for strings. I had a lot of tiny little bugs which slowed progress. Such is life. The banquet and awards ceremony started so we called it a night and went off to eat dinner. They had good lentils. I ran into Thalia Archibald and John Regehr and we talked about (really, they talked about and I tried to learn something) what it might mean to either port Alive2 to another compiler than LLVM, or build “Alive3”, or “Mini-Alive” for some other IR. John suggested fuzzing the hell out of the thing first, then doing something more formal later… especially if it’s a dynamic language IR where a lot of the opcodes end up being “function call that can do anything”. I had a nice chat with Steven Holtzen and Zach Tatlock about research and grad student life. I got some good advice. I meant to talk to Zach about this thing we keep occasionally chatting about that I call “the big e-graph in the sky”. I talked a bit about it to Max Willsey and he had some good probing questions about what would be slow, challenging, or somehow undefined given my problem statement. I continued struggling to implement Knuth-Bendix with significant assistance from Slava and I think eventually got something working. I had a really nasty bug due to string slicing semantics 2 in Ruby. Aaron went off to learn about deep immutability in Python and then got to chatting with the authors of the paper. We got to compare notes about language and language implementation challenges. It’s been a long time since I was in Python-land. Aaron and Slava and I got enchiladas for dinner. I had no reading material for the flight so I went downtown, intending to buy one book, but got too many books. They barely fit into my bag for the flight home. I started reading Anathem for the second time. It holds up. It’s a damn good book. Actually, it might be my fifth. I just remembered that I attended PLMW in 2020 and also watched a few online talks at wild hours from my living room.  ↩ The semantics in Ruby are probably globally reasonable but did not fit the thing I was trying to do: if we have two strings and , we want to find the index at which they start to overlap, . Then we want to grab the bit of that is to the left of . I had initially written that as . However, if and overlap at the start of , is 0. This generates the range , which means we’ll slice until the end of . Not what we want. Instead, the fix in the commit shows how I had to add a special slice function called that handles the 0 case.  ↩ Actually, it might be my fifth. I just remembered that I attended PLMW in 2020 and also watched a few online talks at wild hours from my living room.  ↩ The semantics in Ruby are probably globally reasonable but did not fit the thing I was trying to do: if we have two strings and , we want to find the index at which they start to overlap, . Then we want to grab the bit of that is to the left of . I had initially written that as . However, if and overlap at the start of , is 0. This generates the range , which means we’ll slice until the end of . Not what we want. Instead, the fix in the commit shows how I had to add a special slice function called that handles the 0 case.  ↩

0 views
Jason Scheirer 2 weeks ago

A Revival of Sorts: Getting my iPod Classic 6G Working Again

I’ve been very happy with my Y1 MP3 player over the past 9 or so months. I take it with me everywhere! It’s a companion on my commute, it’s a focus tool in my open office, it’s a way to have a single-purpose device that doesn’t have the distractions of my glass Everything Rectangle and, as the phone ages, a way to mitigate its now-horrible battery life by using a different device with a different battery. As God confounded the language and scattered the people building the tower of Babel, I have confounded the functionality and scattered the responsibilities of the apps on my iPhone. My wife brought up a point that is completely fair: why am I using this $60 piece of crap when she, through great sacrifice, bought me a top-of-the-line iPod Classic 160GB for the same purpose? Sure, that was in 2012, but it was expensive . It’s still worth $350+ today, right? So what the hell, I dug it out of my Closet of Cables and Mystery. Plugged it in. Battery charged. It booted. My music was still in it, last addition to the library wa 2014. Fantastic! I bought a protective case, some new 30-pin USB cables because the ones I had remaining were all frayed and kind of scary, and I got ready to swap the Y1 with the iPod for a while as an experiment. Then my first hurdle: I wanted to add some songs to it. I know Rhythmbox , my player of choice 1 , has an iPod plugin on its list of installed plugins. I plug the iPod in, it shows up! Hooray! I try to drag music onto it: no dice. Checking I see some very threatening notices that HFS+ with journaling is not supported by Linux at all . So I know on Mac it’s a simple command line call to turn journaling off on a volume so it’s probably a trivial process, but I have no working personal Apple desktop machines. Have no fear: I found a chunk of unvetted C that directly alters the raw filesystem to do it for me on Linux! Boom! We’re in business! Back to Rhythmbox. Drag the music I want over to the iPod. It copies! Bingo! Only: no bingo! I disconnect the iPod and it says ’no music.’ The music is on the device, but the iPod’s music database got clobbered. Well crap. So now I know Gtkpod is purpose built for this. Apparently the iPod Rhythmbox plugin isn’t any good on these models, so let’s try that. No dice. It repeatedly hangs, crashes, and when it does work it still fails to correctly update the database. Still ’no music.' Maybe this is all because it’s still HFS+ and not FAT? It seems like most tools assume you’ve liberated your iPod and you’re using it in Windows mode, not Mac mode. So I attempt to wipe the drive, but can’t for the life of me figure out how to do it correctly with Gtkpod or just plain old partitioning tools. Looks like I need to restore the hardware from iTunes for this route. What about Rockbox ? I use it on my Y1. The annoying thing is that I have to manually update the database on the actual device, whereas the typical iTunes stock experience is one that updates the database iteratively as a matter of course of adding music. But the trade-off is no more struggling with Gtkpod and friends, which is higher friction than the drag-and-drop experience of putting music on my Y1 anyway. And I saw this totally cool skin on Reddit I want to try ! I already have the Rockbox utility on my machine from installing it onto my Y1. It sees my iPod but dies on an SSL handshake talking to rockbox.org while downloading resources. I don’t remember this happening last time I ran this. I downloaded and ran the utility on another Linux machine and got the same result. I gave up about 45 minutes into building the tool myself from source. Now I need a Windows machine to use iTunes in Windows to reformat the iPod. I have a debloated Win11 VM in Gnome Boxes, I fire that up and go in to iTunes, I plug in the iPod, then I go to set up USB forwarding so the VM can do its magic and – “USB Forwarding is Not Supported in the Flatpak version of Boxes.” So I uninstall the Flatpak and migrate my disk images from to somewhere less Flatpak-specific and install the dnf version of Gnome Boxes. I migrate the machine over, set up forwarding, everything seems to be working. Only USB forwarding forgets the device when it disconnects and I have to reconnect multiple times. It also doesn’t see the device when it’s in that raw flash mode, so it can’t forward to install the iPod firmware. This is a dead end. Okay, so I have one Windows machine in my house: my kid’s 2013 Intel Macbook with Boot Camp and a debloated copy of Win10 we solely use to play Minecraft Java together with. Only ever since I set up a local server with GeyserMC and Floodgate we’ve been playing mixed me-on-Java/him-on-iPad-or-Switch-Bedrock so the laptop is mostly neglected. So I install iTunes and wipe the iPod. Takes awhile, because I have to install a cascading series of drivers, but it eventually works. The firmware was the latest for the Classic, released 2009. Then I remember that 18 year old bit of early enshittification of iTunes: the iPod can’t simply be its own library you add/remove items from. I was falling out of love with Apple about that long ago , and I had forgotten how low and slow we’ve been dealing with the world of You Will Own Nothing enshittification that’s been inflicted on us. No wonder we’re so complicit, we’re pushing a quarter century of Everything Rental now. So to do iTunes proper I’d need enough storage on this laptop to hold the music in my library on it, be logged in, and sync a selection of it to the iPod. I remember this now: they made life harder and worse on purpose. And now we have Spotify, where we never had freedom or affordances at all. I remember thinking what an incredible act of charity it was that Spotify let your have an offline playlist on your device. I would have expected offline first as a matter of course in prior hardware/software cycles. Rhythmbox and Gtkpod still don’t sync correctly. Same database issues, so nothing I’d done with wiping the iPod had fixed the fundamental first issue. So I install the Rockbox utility on the Windows machine. I have to install some additional Windows components to get it to load, but it works. I flash the iPod. It doesn’t boot. I flash it again. It boots. Hell yes. And I have my cool theme. So I drag music over. 16000 tracks to start, takes 2 hours to copy. HDDs are slow . Afterward I have to manually update my database from Rockbox, which takes hours . I fall asleep as it runs. I can hear the physical spinning platters. It’s a very strange experience having a device with a real life magnetic disc hard drive again. The future we occupy today is strange in the UX of the iPod and its software feels modern enough but small aspects like an HDD feel anachronistic. The Rockbox experience is a lot nicer on the hardware it was designed for than the crappy Rockbox-in-emulation on an Android device that has absolutely no business whatsoever claiming it can run Android. It is responsive, it doesn’t crash, all the plugins work, etc. Next rabbit hole is investigating battery/storage upgrades. There are cheap and expensive options, I need to go through them. As is my wont, I do not need bluetooth on anything I own, but a modern USB-C connector might be nice? Do I want to go the SD card route or a proper SSD? That is for another time. Anyway, no normal person would inflict this experience on themselves willingly, and would likely give up at some point close to the beginning. It is a reminder that much like if you stay very quiet near a playing iPod you can hear the whir and rattle of the HDD. If you stand very quietly near me you can hear the fluttering and tapping of dozens of moths smashing their bodies against the inside of my skull in the space where a brain should be. I am not aware of any other MP3 player that can handle large music libraries this well and still have a presentable UI. TUIs usually suck, “new” apps are all super slow because of Wirth’s Law.  ↩︎ I am not aware of any other MP3 player that can handle large music libraries this well and still have a presentable UI. TUIs usually suck, “new” apps are all super slow because of Wirth’s Law.  ↩︎

0 views
Jack Vanlightly 2 weeks ago

Benchmarking Hardwood 1.0 on a Threadripper 9980X

Hardwood is a minimal-dependency Java library for reading Parquet files. It currently has row-reader and columnar-reader APIs, with Parquet writing planned for the future. Gunnar Morling, Hardwood’s author, published some initial benchmarks in the v1.0 announcement, comparing Hardwood’s row and column readers against Parquet Java . Those benchmarks measured read speed against already-downloaded Parquet files.  Gunnar’s benchmarks ran on an m7i.2xlarge, with 8 vCPUs / 4 physical cores. Each test used three variants: Hardwood with decoder threads = , which equals 8 Hardwood pinned to one CPU thread with taskset Parquet Java, single-threaded I was curious how the same benchmarks would look on my Threadripper 9980X: 64 cores / 128 threads, with 256 GB ECC DDR5. I modified Gunnar’s benchmark code to also test Hardwood with fixed decoder-thread counts: 1, 4, and 8. That gives the following Threadripper variants: Hardwood, unpinned, decoder threads = 128 (available processors) Hardwood, unpinned, decoder threads = 8 Hardwood, unpinned, decoder threads = 4 Hardwood, unpinned, decoder threads = 1 Hardwood pinned to one CPU thread (taskset) Parquet Java, single-threaded One important detail: decoder threads = 1 is not the same as the pinned 1-core test. With decoder threads = 1, the main thread can run on another core. The pinned test constrains the whole process to one logical CPU which is the closest we can get for like-for-like comparison to single-threaded Parquet Java. This benchmark reads all columns of the dataset 48M row dataset. m7i.2xlarge Fig 1: m7i.2xlarge, Hardwood (all cores) 16.5M/s, Hardwood pinned 1-core 3.9M/s, Parquet Java (single-threaded) 3.3M/s Threadripper 9980X Fig 2: Threadripper, Hardwood (all cores) 43.4M/s, Hardwood dt=8 48.4M/s, Hardwood dt=4 44.9M/s, Hardwood dt=1 15.5.9M/s, Hardwood pinned 1-core 11.0M/s, Parquet Java (single-threaded) 5.8M/s A few things stand out: The Threadripper is much faster in the single-core cases than the m7i.2xlarge. Hardwood pinned to one core reaches 11.0M rows/s (with some runs reaching over 12M), versus 3.9M rows/s on the m7i.2xlarge. Generally about 3x faster. Hardwood’s single-core result on the Threadripper is also much stronger relative to Parquet Java. On the m7i.2xlarge, Hardwood 1-core is only modestly ahead of Parquet Java: 3.9M rows/s versus 3.3M rows/s. On the Threadripper, Hardwood 1-core is almost 2x faster: 11.0M rows/s versus 5.8M rows/s. More decoder threads help, but only up to a point. The best result here is 8 decoder threads, at 48.4M rows/s. Four decoder threads are close behind at 44.9M rows/s. The default availableProcessors() setting, which gives 128 decoder threads on this machine, is slower than both, which is not surprising. This benchmark reads all rows of the dataset 48M row dataset. It has two variants: Indexed (positional) columns, i.e. r.getLong(3) Named-columns, i.e. r.getLong("passenger_count") m7i.2xlarge Fig 3: m7i.2xlarge, Indexed-columns, Hardwood (all cores) 14.9M/s, Hardwood 1-core 4.4M/s, Parquet Java (single-threaded) 1.4M/s. Named-columns, Hardwood (all cores) 2.8M/s, Hardwood 1-core 1.9M/s, Parquet Java (single-threaded) 1.4M/s Threadripper 9980X Fig 4: Threadripper, indexed (positional) columns, Hardwood (all cores) 33.4M/s, Hardwood dt=8 36.1M/s, Hardwood dt=4 34.9M/s, Hardwood dt=1 14.4M/s, Hardwood pinned 1-core 10.8M/s, Parquet Java (single-threaded) 3M/s. Named columns, Hardwood (all cores) 5.9M/s, Hardwood dt=8 5.8M/s, Hardwood dt=4 5.9M/s, Hardwood dt=1 5.7M/s, Hardwood pinned 1-core 4.3M/s, Parquet Java (single-threaded) 2.6M/s The indexed-column row reader shows the same basic pattern as the columnar full scan. Hardwood is much faster than Parquet Java even in the pinned 1-core case: 10.8M rows/s versus 3.0M rows/s. The best multi-threaded result is again with 8 decoder threads, at 36.1M rows/s, with 4 decoder threads close behind. The named-column reader is different. Hardwood is still ahead of Parquet Java, but it does not meaningfully scale with decoder threads. The unpinned Hardwood results are all around 5.7M to 5.9M rows/s, regardless of whether the benchmark uses 1, 4, 8, or 128 decoder threads. If you want high throughput, use the indexed-column approach. This test generates data with 4 columns and 50M rows where event_time is perfectly ordered. The filter is event_time < threshold, and therefore the file is therefore clustered by the predicate column, relying on Parquet row-group/page/column statistics. The file contains no bloom filters as Hardwood does not support those yet). There are two variants: selective: event_time < 2,500,000 (about 5% pass) matchAll:  event_time < 50,000,000  (100% pass) The test measures the time for the filtered scan to complete. m7i.2xlarge Fig 5: Selective (5%), Hardwood (all cores) 12.9 ms, Hardwood pinned 1-core 53.8 ms, Parquet Java (single-threaded) 173 ms. Match-all (100%), Hardwood (all cores) 222 ms, Hardwood pinned 1-core 983 ms, Parquet Java (single-threaded) 3157 ms Threadripper Fig 6: Selective (5%), Hardwood (all cores) 10.5 ms, Hardwood dt=8 5.1 ms, Hardwood dt=4 7.2 ms, Hardwood dt=1 24.1 ms, Hardwood pinned 1-core 32.0 ms, Parquet Java (single-threaded) 97.9 ms. Match-all (100%), Hardwood (all cores) 95.0 ms, Hardwood dt=8 80.4 ms, Hardwood dt=4 122 ms, Hardwood dt=1 425 ms, Hardwood pinned 1-core 537 ms, Parquet Java (single-threaded) 1777 ms. The relative shape is similar to the m7i.2xlarge results, but the Threadripper is much faster. In the single-core comparison, Hardwood is about 3x faster than Parquet Java in both cases: 32.0 ms versus 97.9 ms for the selective scan, and 537 ms versus 1777 ms for the match-all scan. With multiple decoder threads, Hardwood is much faster again. The best Threadripper result is 8 decoder threads: 5.1 ms for the selective scan and 80.4 ms for the match-all scan. I hacked on Gunnar’s benchmark code to add some more test cases. Fig 7: Threadripper. Hardwood (all cores) 192M/s, Hardwood dt=8 215M/s, Hardwood dt=4 119M/s, Hardwood dt=1 30.9M/s, Hardwood pinned 1-core 26.8M/s, Parquet Java (single-threaded) 13M/s This is one of the clearest decoder thread scaling results. Hardwood 1-core is about 2x faster than Parquet Java, and 8 decoder threads reach 215M rows/s (14.8x faster than Parquet Java). Unlike the full-scan benchmarks, there is a large gap between 4 and 8 decoder threads here. Fig 8: Threadripper. Hardwood (all cores) 118M/s, Hardwood dt=8 120M/s, Hardwood dt=4 119M/s, Hardwood dt=1 116M/s, Hardwood pinned 1-core 50.1M/s, Parquet Java (single-threaded) 87.1M/s. The string column seems to change the performance profile. This case behaves differently, with Parquet Java winning compared to the pinned 1-logical-core Hardwood test. More than one decoder thread does not help: the unpinned Hardwood results are all between 116M and 120M rows/s. I haven’t profiled this so I can’t explain the result. In this test, we use the predicate , which matches 500324 rows (1%) of the deterministically generated 50M row dataset. This time the files are not clustered by the predicate but the total number of matching rows is 5x smaller than the filter test from earlier. Fig 9: Threadripper. Hardwood (all cores) 141 ms, Hardwood dt=8 135 ms, Hardwood dt=4 131 ms, Hardwood dt=1 129 ms, Hardwood pinned 1-core 291 ms, Parquet Java (single-threaded) 2522 ms. Hardwood is far ahead of Parquet Java here. Even the pinned 1-core Hardwood result is about 8.7x faster than Parquet Java. I ran the benchmark with the flag, which verifies that each test returns the same data, and it passed, so the result looks legit. Decoder threads do not help much in this test. The unpinned Hardwood results are all between 129 ms and 141 ms. That suggests this benchmark is limited by something other than parallel decoding. The Threadripper 9980X is a workstation, not a server. It has a higher clock speed but lower memory bandwidth that its EPYC server counterparts. I imagine you’d see lower performance numbers on the EPYCs for these tests, but the EPYCs would easily beat the Threadripper on the amount of parallel Hardwood workloads due to the 12-memory lanes compared to the Threadripper’s 4 lanes. Thinking about memory bandwidth, I decided to see how Hardwood scales across instances, where each benchmark process was pinned to 4 physical cores and given 4 decoder threads. Fig 10. Threadripper. 1 process (4 physical cores) 26.1M/s, 2 processes (8 physical cores) 47.5M/s, 4 processes (16 physical cores) 79.2M/s, 8 processes (24 physical cores) 81.2M/s, 12 processes (48 physical cores) 79.6M/s, 16 processes (64 physical cores) 75.1M/s. We reached close to this workstation’s memory bandwidth limit at 4 processes on 16 physical cores, and after that there was little benefit or even reduced throughput as efficiency dropped. Fig 11. The memory bandwidth topped out in the 4th test (8 processes, 32 physical cores) The Instructions Per Cycle (IPC) dropped further and further, signalling the reduced efficiency. Fig 12. The IPC drops as we add more and more parallel benchmark instances. And, we became increasingly memory bound. Fig 13. AMD uProf’s top-down estimate of how much CPU pipeline capacity is lost because the backend is waiting on the memory subsystem The EPYC 9575F single socket has 614 GB/s (theoretical) and the dual-socket up to 1.2 TB/s (theoretical) bandwidth, compared to just 205 GB/s theoretical for my workstation (though the max actual I’ve measured is 170 GB/s). So the EPYC would have blown the socks off my workstation. I’m including this as a reminder that benchmarks don’t usually measure things like memory bandwidth saturation under high parallel load. On my Threadipper 9980X, Hardwood’s single-core performance looks strong against Parquet Java across most of these benchmarks. In the full columnar scan, pinned 1-core Hardwood is almost 2x faster than Parquet Java. This contrasted to the m7i.2xlarge where Hardwood only saw a modest single-core advantage over Parquet Java for this specific test. Thus a reminder that your mileage may vary. In the positional row-reader scan, Hardwood was about 3.6x faster than Parquet Java, and in the filtered scans, about 3x faster. The custom predicate benchmark shows an even larger gap.  Hardwood’s multi-threaded performance is also strong up to a certain decoder-thread count (which is workload-hardware-dependent). On this Threadripper, 4 or 8 decoder threads were usually enough. The default value gives a ridiculous 128 decoder threads which was unsurprisingly less efficient than 8. The main exceptions to decoder thread scaling were the named-column row reader, the string column subset, and the custom predicate benchmark. Those cases showed little or no benefit from increasing decoder threads, even when Hardwood still beat Parquet Java overall. I initially wondered if the strong single-thread performance compared to the m7i.2xlarge was the Threadripper’s strong AVX-512 support, but after profiling it with AMDuProfPcm, it turned out that this was not the case. I also tested out enabling the Vector API, but it made no difference to the performance. If any performance engineers out there want a fun project, then my feeling is that Hardwood still leaves a lot on the table for optimizing. It could be a fun project. I finish by saying this benchmarking was for fun on a workstation. So these results are not generalizable but they do correspond to the m7i.2xlarge results (just better). They are mostly useful as a directional look at how Hardwood behaves on a high-core-count workstation. You need to benchmark your own use case, on your chosen hardware. Hardwood with decoder threads = , which equals 8 Hardwood pinned to one CPU thread with taskset Parquet Java, single-threaded Hardwood, unpinned, decoder threads = 128 (available processors) Hardwood, unpinned, decoder threads = 8 Hardwood, unpinned, decoder threads = 4 Hardwood, unpinned, decoder threads = 1 Hardwood pinned to one CPU thread (taskset) Parquet Java, single-threaded The Threadripper is much faster in the single-core cases than the m7i.2xlarge. Hardwood pinned to one core reaches 11.0M rows/s (with some runs reaching over 12M), versus 3.9M rows/s on the m7i.2xlarge. Generally about 3x faster. Hardwood’s single-core result on the Threadripper is also much stronger relative to Parquet Java. On the m7i.2xlarge, Hardwood 1-core is only modestly ahead of Parquet Java: 3.9M rows/s versus 3.3M rows/s. On the Threadripper, Hardwood 1-core is almost 2x faster: 11.0M rows/s versus 5.8M rows/s. More decoder threads help, but only up to a point. The best result here is 8 decoder threads, at 48.4M rows/s. Four decoder threads are close behind at 44.9M rows/s. The default availableProcessors() setting, which gives 128 decoder threads on this machine, is slower than both, which is not surprising. Indexed (positional) columns, i.e. r.getLong(3) Named-columns, i.e. r.getLong("passenger_count") selective: event_time < 2,500,000 (about 5% pass) matchAll:  event_time < 50,000,000  (100% pass)

0 views
Brain Baking 2 weeks ago

Postcard Teas: A Few Impressions

For almost ten years now, we’ve sworn by Mariage Frères when it comes to shopping for high quality loose tea leaves. The nearest shop, however, is in Lille, which is almost three hours away. Their webshop is crude and doesn’t allow for a taste session before buying hence we did buy our fair amount of misses. Yet we remained faithful: the few times that we diverged from the brand ended up in a disappointment. And then I saw someone claiming that London-based Postcard Teas is “even better than Mariage Frères”. My initial reaction to that was “impossible”. I secretly made a note in my journal regardless. When our stock started to dwindle, I dug up that note and said to myself: what the heck, let’s do something crazy and order elsewhere. Postcard Teas is a small shop in London that sells specialty teas by importing directly from the growers. Their unique selling point is hinted in the name: these growers only have a few acres in which they aim to grow the best quality possible. The result is only a few kilograms of yield each year, yet the average price remains acceptable. Each bag of tea you order comes wich a lovely postcard and piece of art depicting a work from the country of origin. I have no idea where Mariage Frères’s tea comes from and love the fact that with Postcard Teas, this knowledge is accessible—even evident. Besides the location and yield, the back of the postcard even contains the grower’s name and a tidbit of bio. Watch China Minutes’ visit to the small shop to breathe in the atmosphere. Meanwhile, I’ll go prepare myself a cup of their summer Darjeeling. Still interested? Great! Yet people outside UK should be warned as shipping comes with a hefty taxation at the border I didn’t mentally prepare for… Just take that into account when you’re browsing their webshop—and don’t forget to compare prices with your usual supplier per , not the deceptive . With that being said, here are some impressions of the teas we tried out: Golden Darjeeling A lovely dark red tea that goes down very well without being too strong. I usually buy first flush/spring Darjeeling and kind of wish I did here as well as that’s usually milder, but this summer Darjeeling is excellent, even if you accidentally let it steep for too long. Contrary to its spring variant, it also handles heat very well, so I usually set it with boiling water. A pure Darjeeling is usually my go-to in the morning or even right after lunch. This black tea is less black than the cheap green powdered teas bought in supermarkets. 4 out of 5 Blounts—Great. Gianfranco’s Earl Grey The first thing that came to mind after opening the bag is: I hope the strong scent does not reflect in the taste. And luckily, it doesn’t. Mariage Frères’ Roi Des Earl Grey is more purgent, up to the point that they might have overdone it. Gianfranco’s bergamots in Calabria pair very well with Kerala’s small Darjeeling tea farms. The structure and colour of the tea is very similar to the previous one, the Golden Darjeerling. This is because Postcard Teas blends both flavours in their shop in London giving them the advantage of carefully choosing both ingredients. Since I love a good Darjeerling, it’s impossible to resist. I do still prefer Mariage Frères’ more daring lavender Early Grey. 4 out of 5 Blounts—Great. New Assam Chai This is the first tea from Postcard Teas that I like less the more I drink it. The culprit? The particular blend of spices: way too much green cardamon. Cardamon is a spice with a minty freshness that easily overpowers everything else, as it indeed does here. Also, the Assam is cut in finer pieces than I’d wish making this brew very dark and strong. I recognise the need for a strong tea to counterbalance the just as strong spices here but for me it was just a bit too much. Adding lemon and honey helps but only up to a point. I know you’re supposed to drop a few splashes of milk in it but I’m not British nor Indian so I don’t. 2 out of 5 Blounts—Mediocre. This is a traditional curled green tea from Japan called a “kamairicha” tea: instead of steaming the tea to stop the oxidation, kamairicha is roasted in a dry pan. Contrary to most Japanese teas such as Sencha, the typical bitter taste is gone because of this process. Mr Ogasa’s farm in Gokase is only 14 acres big. I’m not a huge Japanese tea expert but I do like this one. I do find it difficult to properly prepare: at more than the tea oxidises and still comes off as a bit too bitter. It’s more evenly balanced than the Senchas I have tried before, but that does mean it can come across as bland. I enjoy this tea the most when I am not doing anything else besides drinking tea. 3 out of 5 Blounts—Good. Miyazaki Oolong This complimentary little bag of Oolong tea leaves from Mr. Takuya Yokoyama tastes like a sweet Sencha instead of a typical Oolong tea. It’s one of the greenest ones with virtually no astringency, as described by Postcard Teas themselves. This is exceptional tea of which only was madein 2025. This is interesting because Oolong is usually made in China, not Japan. The problem is that this tea is very delicate: if you’re working or watching or playing something, you might gulp this down without blinking and afterwards think “what did I just drink?” I think these delicate teas are an acquired taste and require a mindful, peaceful moment of tea but nothing else. But why should I buy this Oolong when I already have the Guri Green? I usually prefer my Oolongs to be a bit more oxidised. I hope I’m not getting slammed for this. Oolong teas have a huge variety in roasting/oxidation/etc and this one ranges in the “barely Oolong at all” category. What I also learned is that for Oolong teas the first steep is usually a “wash” to get to the more flavourful second steeps. Perhaps I should try that for Miyazaki’s tea. 3 out of 5 Blounts—Good. Jasmine Green As mentioned on the postcard: “a delightful Vietnamese tea made with spring-picked green tea from Mr. Than’s tea co-op in the mountain village of Ban Lien in Lao Cai province”. Delightful is indeed the correct word here: this must be one of the best Jasmine teas I have ever tasted. It’s very delicate, never bitter, and after you’ve had a cup, you want to make another. What else can I say? It accepts but you better wait a few more minutes until it cooled down to at least and not let it steep for too long. Of course, our pantry now doesn’t stock the three Mariage Frères jasmine teas we tried, so I can’t directly compare them. They’re all great and completely different from the supermarket-bought Jasmine crap. 5 out of 5 Blounts—Amazing. Related topics: / tea / By Wouter Groeneveld on 29 June 2026.  Reply via email .

0 views
Stone Tools 2 weeks ago

Visual Basic on the PC w/Windows 3.1

If I dig deep into my own heart, really self-reflect, I find I simply don't possess whatever people like Bill Gates and Elon Musk do. I think most of us are content to know we've touched a life or two, helped make someone's existence a bit more pleasant, and can feel gratitude toward the universe for those small miracles. Others seem to know no limit in their acquisition of influence, power, and wealth. For them, it isn't simply enough to guide an industry, they must be the industry. In this zero-sum game, there is no upper limit to their cravings Before Musk became the first (I'm choking on the word) trillionaire , Gates was the world's richest person for a couple of decades. Like Musk, he crossed a specific monetary milestone back in 1999 as the "first person with a net worth exceeding $100 billion," about $200B in 2026 money. How he earned it and what he did with it has been the subject of any number of documentaries , books , movies , interviews , depositions , and damning rumors . I think the media can agree on at least one point relevant to our discussion today: Bill Gates was hellbent on owning the entire personal computing landscape. He said as much, out loud, on stage, to industry professionals, in front of the press. Jacqui Morby recounted the story on The Computer Chronicles . "Gary (Kildall) got up (at the Rosen Forum panel discussion) and talked about what his plans were for CP/M and where the company was going, and then made a comment, 'Well, this is a very large market, and there's room for lots of companies.' Bill Gates interrupted and said, 'No, there'll only be one company.'" He didn't seem particularly interested in creating innovative things, so much as he wanted to make sure that the innovations of others had a Microsoft response. While working with Apple to develop software for the original Macintosh, Andy Hertzfeld recalled a story of Gates digging in for system details that didn't really have anything to do with the business applications being built by Microsoft. Shortly thereafter, Windows 1.0 released, much to Steve Jobs's frustration . Jobs wouldn't be the last to feel screwed over by Microsoft "taking" ideas . Another tactic employed by Gates was absorption, the tried and true fast-track to acquiring toys one lacks. Consider the story of Alan Cooper . Coincidentally the idea for a visual application builder "popped into his head" just as HyperCard debuted, in 1987, triggered by Microsoft's announced adoption of DLLs, dynamic link libraries, which provided easy access to core operating system functions to whomever wanted to tap into them. Cooper saw this as a unique foundation upon which to build a kind of "construction set" for the DOS visual shell of your corporate dreams. Don't like the default Windows shell? Build your own! Microsoft engineer Gabe Newell was super impressed with Cooper's demo of the construction set, then called Tripod, and arranged for a demonstration for Gates. From the excellent article, "Something Pretty Right" by Ryan Lucas. "Why can't we do stuff like this?" is very revealing phrasing, IMHO as an armchair psychologist. Give that line to 1,000 actors and you'll get 1,000 unique performances balancing the tension between frustration and longing. As a Very Rich Guy™, there was nothing Gates wanted that he couldn't have. Like someone who pays others to level up their RPG character , US$1M and a contract later, Tripod (renamed Ruby) was his. While Cooper insists that HyperCard had no influence on the creation of Tripod , Gates most certainly was thinking about it. In his article "The 25th Birthday of BASIC" for BYTE Magazine , October 1989 ( Visual Basic would debut in 1991). Ruby was reformulated into something with but a passing resemblance to Tripod . Its bespoke scripting language was replaced with a variant of BASIC, and the goal of the program was no longer to build shells on top of the Microsoft DLLs, but to build applications for Microsoft's own shell, Windows 3.0. Visual Basic was born, arguably a more profound product than Cooper's original vision. Credit where it's due, Gates saw potential that Cooper himself couldn't see. A while back, I dug into Apple's HyperCard . Visual Basic gives us an interesting opportunity to look at a similar first-party, visual programming solution from Microsoft's perspective. Like HyperCard , Visual Basic had its own dedicated magazine , and inspired legions of developers long after Microsoft ceased support in 2008. As recently as 2023 , Microsoft has had to issue official statements on their support plans for "classic" Visual Basic, which keeps a huge number of bespoke, legacy applications alive, something HyperCard cannot claim. The Microsoft vs. Apple wars of the day almost necessitated taking sides, but in truth each has something it could learn from the other. Visual Basic 3.0 was the last pure 16-bit application in the line, and was the first version to include robust database capabilities. The true potential of the product was unlocked. This particular OS/application combination is much more in keeping with the spirit of this blog, I feel. There's a lot to learn. When I studied HyperCard , I noted the 1,000 page book that awaited me. Visual Basic ships with 3,000 pages, to say nothing of the wealth of 3rd party publications; an industry unto itself. As a man who recently took another annual step toward that great Blue Screen in the sky, every tick of the second hand gently rattles my bones. For large projects like this I have to consider how quickly I can get up to speed. Well, given the temperament of training books of the day, I suppose the proper first consideration is, "How dumb am I?" I refer to myself as a "big dummy" in blog posts, and I stand by that assertation, but I don't like it when others call me dumb. I can handle more complex material, but like I said, I don't have a lot of time. How quickly can I learn Visual Basic ? That seems unabsorbably fast . Maybe if I didn't sleep? I think I'd forget everything by Monday. Also by Tuesday. "Proglaming" sounds like fun, but a week is still too fast for my pace. Getting closer. Perfect. Slow enough for an old man to follow; fast enough to finish with time to spare before involuntary admission into a retirement home. If I weren't 40 years too late, I'd throw my own hat into the publishing ring and combine "I'm a big dummy" with "I want to learn this quickly." It's been a long time since I last touched Windows 3.1. It's funny, my memory of it doesn't match my hands-on experience today. I recall it being far uglier, though it still suffers from absurdly large title bars which don't provide much in the way of information or utility. I dig that (VGA mode) powder blue , though. It's handsome if perhaps uninspired, the result of a collaboration between Microsoft and IBM for OS/2's Presentation Manager (which predates Windows 2.0). Their "Joint Development Agreement" gave pretty broad latitude to both companies to use, without licensing fees, code shared between the two companies. I'm not even tangentially familiar with law, but it does read, in part: That gave Windows 2 and 3 a nice glow-up after the flop of Windows 1.0. Initially, even Microsoft had trouble getting their own developers to build Windows applications. I imagine it must have been a huge relief for Gates to have a tool that not only made it easy to build Windows applications, but that could even be an enjoyable experience. Jumping into Visual Basic , the first impression is, "I can do this." It looks approachable. I can't explain what every button in the toolbar does, but some of the basic stuff is as easy to identify as in HyperCard . Adding a control, like a text field, is a double-click away. The "Properties" panel makes intuitive sense, for tweaking the characteristics of a selected control, something HyperCard lacks. Appending code to a control is as simple as double-clicking its instance in the window. "Properties" is context aware, only showing what can be tweaked on the selected object. For the large part, the industry abandoned this contextual approach. I wonder why? PageMaker was leaning that way with its control panel, and InDesign promptly threw that away in favor of persistent controls for things that aren't even in the current document context. Why do we need text kerning tools on screen when there's not even a text box in the current document, in Affinity for example ? Tools like Figma , Apple's Pages seem to have kept the contextual flame alive, which is nice to see. "Pros want every tool on-screen at all times," a UX consultant once said with a straight face, I guess. The toolbar could stand to be better organized and starts gesturing in the direction of that meme image about Microsoft's love of buttons . They certainly did lean heavily on this UI metaphor crutch, as a catch-all way of cramming in as many features as possible. It's confusing at times (why a "picture box" and also "images?"), but with this version of the program, on this operating system, things haven't gotten completely out of hand yet. We're getting up to speed on the controls and how to interface with them today. Let's consider some nice things about Visual Basic's approach. I am rapidly growing to appreciate the keyboard shortcuts for UI elements, like buttons and sliders. Visual Basic makes it super simple to add a keyboard hook to an on-screen control. Simply label a button with in the confusingly named "caption" property and the following character will become the keyboard shortcut, via . So, an "Exit" button with the "caption" will read and will function identically to a mouse click on that button. When I say "identically" I do mean identically. The button's built-in method will be triggered, the same as if a mouse had done it. We don't have to worry about bifurcating control logic between keyboard and mouse for such interactions. We're then treated to an amuse bouche of off-kilter things to come. Checkboxes and radio buttons both have an on/off state, where any number of checkboxes can be on/off, but only one radio button in a set can be on. When programming with these controls, checkboxes return a value of or to represent unchecked or checked. Radio buttons return a or boolean on each of the options. For now, we'll file this under "Things That Make Me Give a Skeptical Sideways Glance." After spending a couple of days with it, the built-in text editor is driving me crazy, a "feature" Visual Basic shares with HyperCard ; neither is good. I can excuse a lack of autocomplete, a tool that would debut with Visual Basic 5 , as "Something Yet to be Invented." I cannot excuse the lack of indentation assistance and word-wraps, both already common features in word processors of the day. Microsoft has given us a smidge more than the absolute bare-minimum for a text editor. Keeping code tidy and readable requires significant, diligent effort on my part; it's not coming easily to me. I appreciate the auto-capitalization (though Basic is case-insensitive) and coloring on language keywords, but syntax checking and formatting a line of text the instant I've repositioned the cursor is annoying. Unfinished lines throw up modal dialogs warning me of interpreter troubles, triggered as easily as moving the cursor up or down for a moment. It's unwieldy to sketch out a code block to fill in the details later with those constant interruptions. It would be nice to be able to trigger the parser on-demand. We're learning about the mouse and how to handle mouse events. From a programmatic standpoint, this is pretty basic stuff. One of the nice things about the code editor is the pulldown in the top toolbar surfaces all possible functions for a selected UI element. We don't have to try to remember the exact name and spelling of a function; just pick the one you want to edit and get started. A setting that is theoretically interesting is the default unit of measurement for elements. Until now, I'd never heard of "twips": a "twentieth of a point". Where a point is 72/inch, there are 1,440 twips/inch. Windows used this as a device-independent standardized unit of measure. For on-screen, a conversion to pixels was used, and for print a conversion to printer resolution was used. Any form you design in Visual Basic can be trivially sent to the printer with a simple Basic call, and it will print at the resolution of the printer, not your screen. The coolest trick, though, is "edit and continue." Because the program is being constantly interpreted, not compiled, we can run the program, pause it, modify the code, and continue live execution. This is super handy for iterating solutions to annoying bugs. The Microsoft-faithful have really never known a world without this. The Apple-faithful have had this tantalizing fruit dangled before them a couple of times now, never quite delivering on the promise. I like it. In building out WIMP applications , we need to fill out the "M" part of that acronym. Today we learn how to build menus using the "Menu Design Window." The tool is competent, if a bit inelegant. Initially, it is easy to bang out a rough outline of an application's menu structure without taking one's hands off the keyboard; mouse-free is always a welcome option. Type a menu item, hit , type the next, hit , and the next, etc. Then, apply structure to the menu with the on-screen arrow tools for indentation/reordering elements. Alas, we cannot indent at the time of menu item entry, that hierarchy must be set in a separate step later. One disappointing absence is any kind of relationship between menu elements. Moving a menu item with "submenu" items will not move those submenu elements with it. No "outliner" style editing, ala ThinkTank , here. We also cannot multi-select items to edit them as a group, something we can do with form controls. Slow, patient, one-at-a-time editing of menu items is all we get. To be fair, menus can be programmatically generated, which may honestly be a better option in many ways. That pulls us away from the "Visual" in Visual Basic , though, don't it? The design window also forces its vertical editing into a horizontal view, another "Things That Make Me Give a Skeptical Sideways Glance." The example in the screenshot shows a 3-level menu, and I'm nowhere close to filling that horizontal space. It's wasted screen real estate, made more aggravating by the fact that the menu design window cannot be resized . As I think many in the industry have internalized by now, an editor view should place its primary content front and center, with refining elements playing a supporting role. The menu item properties would be much better served filling the right-hand side of the window, giving the menu itself vertical breathing room on the left. It's one of those things that probably gets better over the years, but is conspicuously half-baked for version 3 of the product. "It's OK, but I expected better by version 3," will be a running theme going forward. Now that I've been at this for a week, the angle of approach to visual programming HyperCard and Visual Basic each take has come into sharper focus. Initially, their superficial similarities led me to expect more direct parity between the two. Both provide a visual toolkit for designing interfaces. Both use a more simplistic language than the core language for each platform. Neither is truly "object oriented" (if that's important to you). Both were killed despite amassing a large, passionate following. Even a simple inspection of their toolbars highlights the philosophical difference between the two approaches. Most of the HyperCard toolbox is devoted to drawing pictures, with the controls reduced to buttons and text fields. It is constantly surprising to me how much mileage is squeezed out of such a restricted set of UI controls. Microsoft, on the other hand, offers a toolbar button for each and every thing you might want to add to an application. They take inverted approaches. Where I might add a generic button in HyperCard , then attach a script which invokes the system file browser, Visual Basic gives me a pre-built file browser control to drag into my app. I prefer Visual Basic's approach of "drag out a rectangle to define a control," especially for buttons and text fields; it feels more modern in its UX. HyperCard makes us add controls strictly by pulldown menu, then we have to drag the corners of the button, with no visual indicators, into the new size. Surprisingly awkward. Visual Basic also earns points in offering a grid to snap elements to position, making it much easier than HyperCard to align and scale elements precisely with one another. Gotta do a lot of eyeballin' on the HyperCard side of things; its grid only works in paint mode. Consequently, laying out something like a calculator is much faster and easier in Visual Basic , at the expense (?) of looking exactly like any other Windows program ever made. (Although the demo calculator doesn't look anything like the actual Windows calculator?) Don't get me wrong, conformance to corporate homogeneity may be exactly what you need at times and Visual Basic can generate something "professional looking" in a jiffy. It is, perhaps, devoid of character, but it also creates something a Windows user can look at and trust. Breaking free of those somewhat rigid constraints requires considered effort in Visual Basic , whereas HyperCard practically begs us to go hog wild. We're firmly in "learning Basic" land here; the application itself doesn't have a whole lot else to it. The panel for exporting our .exe files is about as barebones as one could imagine. There's a color palette, but I'm not entirely clear why; colors for controls can be set in the Properties palette via its own popup color palette. I should also give a shout out to the built-in Help system. Though I wish it were context aware, there's an absurd amount of information available right there in Windows without having to crack open the 10 pound manual. HyperCard has Balloon Help, which is nice and cute, but also anemic; we only get as much explanation as fits in a couple of sentences. Visual Basic's help system gives lengthy, detailed explanations of topics with code samples, is searchable, is bookmarkable (!), has tutorials for understanding the principles of the program, and more. It's quite good! The last week of my training book gets intense with discussions on make files, database connectivity, MDI (multiple document interface), DDE (dynamic data exchange), interfacing with DLLs, and so on. We've only been building throw-away toy applications so far, and I honestly don't feel the book has mentally equipped me for these hairier discussions. It's a pretty significant cognitive leap from the simplicity I feel the product promised. The long and the short of it is, I'm learning enough Basic to squeak by and get a sense of its tempo and grammar, but as a first-time user I find it more overwhelming than HyperTalk. HyperCard and Visual Basic each come with a 600+ page language reference guide. Microsoft also throws in three more manuals, another 2,400+ pages, for good measure. Its language guide would expand to 1,000+ pages in Visual Basic 4. Brevity is the very soul of cowards, I guess was their stance. Though their language reference guides are similar length, Microsoft's is a far more dense, dry tome. Apple spends the first 150 pages talking about "What even is programming?" and the last 150 pages getting into topics outside the scope of HyperTalk; a slim 300 pages to describe the language. Let's examine some concrete examples. Here's how to make the system thrice on the click of a button in HyperCard : Here's how to (ostensibly) do that in Visual Basic 3: Full disclosure: this didn't work, even though it is the example given in the "Programmer's Guide." Something is coalescing the three beeps into one. DOSBox-X issue? Because scripts are kind of "embedded" into their respective HyperCard objects, we don't have to disambiguate subroutines with prefixes; any given script is scoped precisely to its associated GUI object. It's the La Croix of object orientation; just a whiff of a hint of that flavor. HyperCard's approach lends itself better to casual tinkering around, but Visual Basic has an edge in surfacing all functions of our application in the code editor. In HyperCard we have to remember which object contains which code block, or hunt through all objects individually, searching for the code we want. Visual Basic's approach requires unique names for all subroutines. This makes it fairly trivial to trigger events across objects. If we want a button to click another button by proxy, we would have to do something like this in HyperTalk: Sometimes I wish HyperTalk would allow dot-syntax for object specifier chains. In Visual Basic, we simply call the uniquely-named function directly: Where HyperTalk takes a gentle, English-like approach to its language, Visual Basic isn't afraid to be far more "programmery." HyperTalk developers can certainly get into their own weeds trying to figure out the precise incantation to sidestep the interpreter and achieve specific goals. Conversely, Visual Basic developers could quickly find themselves in a world of memory management, DLLs, batch files, and make files. Both developers feel some pain, but one is kind of orthogonal to the other. Your preference may depend on which breed of demon you enjoy slaying. As clearly evidenced by the Voyager series of software and MYST , highly professional software was possible with HyperCard . That said, the upper boundary for Visual Basic feels much higher. As a simple example, with the keyword we can reach in and directly call the Windows Kernel (or any existing) DLL; this of course being the killer feature that triggered Alan Cooper to develop the program in the first place. That's impossible to do out-of-the-box with HyperCard ; it cannot access the Macintosh Toolbox so deftly. Likewise with database data, Visual Basic gives us flexibility in what kind of data to bring in, like dBASE or FoxPro . There may be specialized stacks or XCMDs (plugins) to HyperCard that can assist with these tasks, but nothing native to the program. However, HyperCard provides its own built-in database free of charge, requiring no special effort on the developer's part to leverage it. Building something like an address book is simply a matter of adding some text fields to a card. Those will function like fields in a database by default, and actions like saving/loading user data will happen transparently. Adding search, or something similar, takes a few extra steps, but is conceptually simple through a HyperTalk command like Visual Basic provides a "Data Manager" module, which allows us to create simple Access databases for use as the backbone of the application. This is all explained in detail in the supplemental 300+ page "Visual Basic 3.0 Professional Features, Book 2." Once the database is built, interfacing with its records is straightforward using the "Data Control" tool. When the database is linked in properly, controls like images and text fields can be wired up directly to their corresponding fields in the database schema, called "bound controls." The database widget itself provides buttons to step through records and corresponding data will auto-populate the bound layout elements. If "browsing" is the extent of your database needs, you're in good shape. I imagine most will want to do more than that, perhaps adding fields, or doing search queries. You'll want to steel yourself, because it gets gnarly real quick. I'll just say that the book is 300+ pages for a reason, with talk about complex subjects like Dynasets, Snapshots, Tables, the JET engine, SQL queries, and more. It's far more capable than HyperCard , as we can work with multiple databases in our VB application, access remote databases, and more. That power is paired with an equivalent learning curve, one which is thrust upon any developer who needs even a tiny bit more than the drag-and-drop controls provide. Overall, it would be fair to call the IDE "competent." It contains the tools we need, arranged by palette, and makes certain actions (like adding a button) as easy as a double-click. What's not to like? I think what frustrates me about these tools is how they feel like somewhat careless design solutions to their respective problems. Look at the "Properties" palette, for example. This looks, to my eyes, like a developer was told, "The properties for a selected object should be available for editing." The developer iterated them as a literal list, adding some basic editing niceties, like making a color chooser available when a color property is edited. What I find in practice is that the vast majority of the properties go untouched, especially for something like a Form object, and the ones I actually need require scrolling through a long list to find and edit. Later properties in the list, even those which are common to all controls, shift around in position depending on how many properties a given control has. I'm constantly having to read through that list, scanning for the "Name" property, which is where we set the programmatic name for the control. It's arguably the most important property , and it's playing peek-a-boo. When I make a new form (a "form" is a window; I don't know why they call it a "form") I have a few things I need to set right off the bat: the size, the title, and the programming reference name. After that, sometimes I want to set the background color. We'll ignore the fact that property names don't make sense; naming conventions had perhaps not yet been firmly established in an era when the terms UI and UX had not yet become common vernacular. From a pure, "What is the user most likely to need?" point of view, this simple alphabetical list is the laziest solution to the design challenge. Fair point, HyperCard's lack of any properties palette was more lazy, but this is version 3 of this product. I frankly (perhaps unfairly) expect more considered effort from a first-party solution. My frustration extends to the main toolbox as well. It's just a bunch of buttons with no organizational structure applied. Tooltips, similar to what we understand today, were introduced with Macintosh System 7 as "Balloon Help" the same year VB3 released, so I can't fault Microsoft for "failing to implement" them in this release. Still, icon-only is a lazy way to handle it, when the goal is to shove as many icons into the toolbar as possible. Asymetrix Toolbook 3 , a similar visual IDE for Windows development, provides more robust, logically arranged tools for the job. Here's the text editor and object properties panels. Note in particular a few things: Visual Basic itself contains a similar contextual help in other parts of the application, like its "Crystal Reports" tool, making its absence in the main app even more frustrating. This kind of haphazard application of tools and controls feels sloppy, which reminds me of something I wanted to talk about. While going through the official manuals for Visual Basic , something kept bothering me. I couldn't put my finger on it at first, but once I saw it, my eyes were forever cursed . This is a small grievance, "petty" some would say, "a colossal waste of mental resources" others may scoff. But what's a tech blog without a certain level of pedantry? I'm not above pedantry. Here we see the Visual Basic 3 manual is laid out in Helvetica and Times. Man, I'm already bored. Anyway, beyond the utterly pedestrian font choices (in fairness, they did have to lay out 3,000+ pages of this stuff), something seems "off" about it. In particular, that Helvetica looks malformed, with sloppy kerning and unbalanced strokes. Let's take a closer look. Helvetica Neue doesn't match, and Arial (my original suspect) is ruled out by the end caps on the capital "C". Helvetica Condensed is also not right. Wait, I see what's happening. It's the same issue I have with the user interface, manifested in the manual. This isn't Helvetica Condensed, it's Helvetica physically squashed into a fake condensed version. The richest man in the world couldn't afford to buy a proper condensed font for his company? "Or is this indicative of a deeper issue?" he asked, slipping back into his pop-psychology armchair. It smacks of "good enough," never striving for "great." That kind of sums up my feelings toward Windows and Windows applications of this period. The stuff worked, and had obvious success, but never seemed to be borne of thoughtful consideration. Did that inattention to detail come from cost-cutting measures, or perhaps some kind of cultural blindness? Were the deficiencies seen and ignored, or simply not seen at all? And that reminds me of something else I wanted to talk about. In the PBS documentary series, Triumph of the Nerds , Steve Jobs famously said of Microsoft, "They have no taste." I genuinely think Bill Gates could not understand the meaning of Jobs's accusation. Or rather, he couldn't fathom why "taste" should enter into his calculus whatsoever. Having no taste didn't stop him from becoming the richest man in the world. What does "taste" have to do with stockholder value? When Apple teased with a new release of OS X, "Redmond, start your photocopiers," I think Gates was thinking, "Of course we will. Thanks for the free R&D." He bristled at being publicly chastised for copying , but my read on that is he really wanted to say, "So what if we copy Apple? Why shouldn't we? Look at our success and tell me it hasn't been a good strategy." What Jobs saw as creative bankruptcy, Gates saw as business efficiency. Being asked to frame his success on Jobs's terms ruffled Gates's feathers. Jobs said, and I agree, that innovation means saying "no" to 1000 things before saying "yes." "Process" is that very action. "Process" is the pruning of the possibility space. It's the self-awareness to distinguish "good enough" from "great." It's when you step away from your work, give it the critical stink eye, and apply taste . That's an impossible task if one has no taste to begin with. So what's a tasteless corporation to do? While Microsoft may have not cared too much about process, they had manufacturing down cold. Put in PenPoint OS, out pops Windows for Pen Computing. Put in OS X 10.3, out pops Windows Vista. Put in Java, out pops J++. Put in a Dreamcast, out pops an Xbox. Even today, similar "factory production" charges are levied against them. I'm not suggesting they "stole" ideas so much as they simply seemed content to let others do the hard work of saying "no" 1,000 times. While they may have shortcut the creative process, they still had to learn how to manufacture products. In so doing, they accidentally picked up a little taste along the way, which would lead to pretty good stuff from time to time. It's been part of the fabric of the industry for decades, and now the torch of manufacturing tasteless product from the creative work of others has been passed on to generative AI. To scale , no less. The ramifications weigh heavily on my mind, especially when someone publicly calls for the absorption of my work into the generative AI apparatus. I'm both flattered and appalled. On average, how many times do you think I rewrite the introductions to these posts? How many thousands of words have I thrown away to reach something approaching what I wanted to actually say? I tend to rewrite intros 3 or 4 times, and I mean that truly; each rewrite is radically different from the others. In this post alone, I have thrown away some 5,000 words. Some might think those 5,000 words are the cost of the process, but that's not right. They are the process. The unpublished words are the important ones. Those are the words that got me to these words. Knowing that, throw any creative work into the generative wood chipper and it should be obvious why what comes out cannot live up to the original. It's lacking the 1,000 nos. I'm disappointed in the ending of this book. Day 21 comes and goes without even a hint of acknowledgement that we've made it through the gauntlet. At the end of it all, we also haven't built anything of value. Every chapter created little baby programs to illustrate specific concepts; no project built upon a previous project except for a few shallow, superficial glow-ups. Contrast that with HyperCard , where we had a full-fledged address book, with database, search, custom art, and save/load. With Visual Basic , I never felt that same spark I did with HyperCard . Visual Basic seems great for when you have a strong idea of what you want to build. However, its lack of drawing tools and "don't worry about it, I've got you covered" database curtail creative exploration far more than I would have predicted at the beginning of my studies. Not having to worry about those details opens up a wider world of "lemme try something real quick" experimentation and iteration. In an ideal product, I'd combine the prototyping strengths of HyperCard with the professional-strength of Visual Basic . Then, later we could swap out the default database with Access, or export the placeholder drawings as image assets for a professional artist to clean up in another revision. I cannot personally find a place for Visual Basic in my heart, but I can absolutely understand why it took off. It filled a major gap in the programming landscape, helping amateurs and pro-ams build tools for themselves, and even throwing a lifeline to a generation of COBOL engineers needing to transition ASAP. Like Apple with HyperCard , that gap was re-opened by the discontinuation of the product, abandoning a whole fleet of developers and, perhaps just as importantly, potential developers. I suppose nothing lasts forever, but these companies are worth multi (choking on the word again) TRILLIONS of US dollars. At valuations like that, with the fealty they demand from us, I consider it a moral imperative for them to provide excellent tools which empower the widest possible breadth of users' skill levels. Not providing such tools is a choice . Considered from another angle, I'll leave you with this open question. What software do Apple and Microsoft provide today that will be spoken of, with the same reverence as HyperCard and Visual Basic, 25 years from now? Ways to improve the experience, notable deficiencies, workarounds, and notes about incorporating the software into modern workflows (if possible). With Visual Basic 3, 2, 1, and DOS 1.0, the applications you build are 16-bit only and are therefore relegated to running only in virtual environments on 64-bit Windows. If this fits your modus operandi, you're in good shape. If you're hoping to keep it old-school, but still want the option of running your creation on modern hardware, then you'll want to get Visual Basic 6 up and running in Windows 2000? XP? I tried it in Windows 98SE and it wouldn't launch. VB6 builds 32-bit applications as standalone, compiled executables, can connect to the Internet, and produces builds which run on Windows 10/11. Note that Windows 11 promises to run applications built with VB6 , but does not promise to run VB6 itself. However, I gave it a shot and though there were issues with the install, and the IDE acts a little weird, and it complains on launch about missing OLE files, it did run and I was able to build an executable on Windows 11. For funsies, here's Gates and Jobs demonstrating their respective visual programming environments. Gates giving a subdued demo of the just-announced Visual Basic 1.0 . His voice cracking at 0:33 is adorable . Jobs had just returned to Apple after they bought NeXT, and here he's showing the technology Apple has bet its future on. We know it today as Xcode , but it started life as Interface Builder . The line he drew between components in the demo was called a "binding," something that has conceptually resurfaced in SwiftUI. DOSBox-X 2026.01.02, Windows x64 build. CPU set to Pentium DOS reports as v6.22 Host system folder mounted as drive C:\ holds Windows Windows 3.1, basic installation 1024 x 768, 32K colors under DOS reports total RAM, but Free only reports . Good enough for today, but 16-bit Windows should be able to register 4MB, not just 2MB. A few extra applications for comparative/convenience reasons: Toolbook, Actor, ObjectVision, Acrobat Distiller Visual Basic 3.0 Reports 386 Enhanced Mode enabled Reports free RAM In lieu of tooltips, at the bottom of the current window we have a contextual description of the current tool, much like Bank Street Writer and Lotus 1-2-3 . The text editor includes indent/outdent tools, can set our editing font of choice, waits to check syntax until we ask it to, and even includes a simple "build a function" utility to wire up common tasks to common UI events. The properties panel is laid out hierarchically, keeping the most-needed stuff front and center, while demoting less-used options to secondary emphasis. DOSBox-X ran everything smoothly and without issue. I did not install Windows on top of real DOS, though. I relied on DOSBox-X's implementation. This may account for a couple of strange issues, outlined below. I experienced one crash in Visual Basic 3 , when accessing the Help system. Issuing a looped command resulted in only a single system beep. My guess is that something in the emulated environment is suppressing this. I could never get databases to connect, even the ones that ship with Visual Basic , let alone any personal data carried over from previous database explorations. It may be the result of DOSBox-X using an emulated version of . Strangely, I saw it work once and then it stopped working as suddenly as it started and never worked again. An installation of Windows on a proper installation of MS-DOS might fix this problem.

0 views
Neil Madden 3 weeks ago

Java’s SSLContext protocol name is a footgun

This should be old news, but I keep seeing the same mistake crop up, so I thought I’d blog about it and spread awareness. In Java, if you want to configure TLS you generally start with an . And you get an instance of this class by calling the static method , specifying the version of the protocol you want to support. But typically a TLS connection supports other versions of the protocol, so what exactly does specifying “TLSv1.3” here mean? Probably not what you think it means… The Java Security Standard Algorithm Names document doesn’t say much useful: “Supports […] TLS version 1.3; may support other SSL/TLS versions.” Well, which other versions? Later ones or earlier ones? That seems kind of important. It’s even vaguer if you don’t specify a version – the generic “TLS” identifier is specified as (my emphasis): OK, but what about the JSSE chapter in the Java Security Developer’s Guide . That seems at first glance to be more precise: Like other JCA provider-based engine classes,   objects are created using the   factory methods of the   class. These static methods each return an instance that implements  at least  the requested secure socket protocol. The returned instance may implement other protocols, too. For example,   may return an instance that implements TLSv1, TLSv1.1, and TLSv1.2. OK, that sounds promising! So, if I specify a version then that is taken as a minimum version and I may also get more recent versions? Great, sign me up! Except, that is the exact fucking opposite of what the default SunJSSE provider does! When you specify “TLSv1.1” (for example), what the default provider does is treat that as a maximum TLS version . So the resulting supports all versions of TLS up to (and including) 1.1, but nothing later. So if you have old code that requests version 1.1 and you try to connect to a modern server that only supports 1.2 and 1.3, then you’ll get a connection failure. And in modern Java, this will fail earlier because TLS 1.1 is disabled by default. If you specify “TLSv1.2” then you’ll just silently get a downgraded protocol for no good reason at all, when you probably thought you were being good and specifying a sensible minimum version. It’s not just the default provider that does this, lots of others have followed the lead, including e.g., the Conscrypt/BoringSSL provider used by Android. I suspect this behaviour exists because of a fear of breaking poorly-written software that baulks at unknown versions and doesn’t handle downgrades properly. But the problem is that many developers think it is best practice to specify a version when creating an SSLContext, and some security scanners even tell you to do so . In the best case, you then get code that is secure until the next major hole is discovered in TLS and v1.4 gets released. In the worst case you’ve silently implemented a self-inflicted protocol downgrade attack. I wonder how many Java apps were (and maybe still are) only supporting TLS 1.1/1.0 despite the underlying JDK supporting 1.2 or even 1.3? I should stop here and mention a subtlety: this behaviour applies to client TLS connections only. Server-side SSL contexts completely ignore the protocol you specify here for the most part and go ahead and support everything . So what should you do instead? Well there’s really no good answer here. Probably the best thing to do is to use the generic “TLS” identifier, which gets you an unspecified version of TLS, but which all providers I’ve looked at so far interpret as “sensible modern protocol versions”, i.e., TLS 1.3 and 1.2 (with 1.1 and earlier supported but disabled by default). There’s no guarantee at all of that behaviour, but there’s also no guarantee when you specify a version, so pick your poison. (I’ve raised a bug for this, as it finally pissed me off enough, but my guess is they’ll either ignore it or fix the guide to be as vague as the standard names descriptions).

0 views
Blog System/5 3 weeks ago

Is anyone still using Emacs?

In a recent discussion at the orange site sparked by the Emacs 31 Is Around the Corner: The Changes I’m Already Daily Driving article, people were asking themselves “Is anyone still using Emacs?” and then providing their own perspective. For me, the answer is a resounding yes… but the interesting part is that I’m not still using Emacs: I’m actually using Emacs again . And instead of burying my answer to the opening question in a long discussion thread, I thought I’d explain my journey with and without Emacs for the last… almost 30 years. At the end, I’ll unveil the specific feature that I feel gives me superpowers and that keeps me hooked. Show your support for this publication by subscribing. It’s free if you want it to be! I got into Linux around 1997 via Caldera OpenLinux 1.1. Before then, I had extensively played with Borland Turbo C++ and Visual Basic as a kid so I was heavily accustomed to those fancy IDEs that we lost . As I got into Linux and found myself in an alien world, I had to buy a couple of introductory books. Yes, books, the printed kind—because that’s how we had to learn new stuff before. Both books talked about Vim and Emacs and presented them as the advanced choices. I found this strange because the IDEs I had used before seemed more complete, but I, being a Windows renegade for some reason, charged ahead. I learned the basics of both editors and went through their tutorials at different times. The two old books I used to learn Linux back in the day, open to show their Vim and Emacs introductions. Since then and until roughly 2015, I flip-flopped between Vim and Emacs. At times I used one, and at times I used the other. I favored Emacs for long-running coding sessions but Vim excelled at my pkgsrc work where I had to edit tens of different files in quick succession. Even though Vim and Emacs worked well for me, I missed something. Language integration was poor so I was tempted by the more modern editors that everyone was touting, and especially so as I moved to macOS. I tried a bunch, like the now-defunct Atom and Brackets , but they all felt brittle and overwhelming: they had too many features, too many settings. And then, VSCode arrived in 2015. As I took it for a test drive, it “felt right” from the get-go. It looked modern, was relatively small, and its plain and simple settings editor—read: just a JSON file because there were no settings panels yet!—made me feel like I was in control. I could understand this modern editor and easily tune it to my needs. Soon after, I started learning Go and then Rust, and VSCode’s integration with their corresponding LSPs made that process so much easier: code auto-completion and real-time error highlighting sped up my learning significantly. I stuck with VSCode for these languages and slowly phased Emacs out. I was sold. During that time period, I was also working on Bazel—a Java project—at Google and the natural choice for it was IntelliJ. I had tried to use Emacs for Java development at some point, but IntelliJ was (and still is) so good that it was the only realistic choice. My usage of VSCode with its Vim plugin continued through my short stint at Microsoft, where I was working on a C++ codebase and had to connect to remote Windows boxes. Most people used RDP to work on the remote machine “directly”, but I couldn’t stand that workflow: I very much preferred running VSCode on my desktop and using SSH to connect to the remote machine, which is something that VSCode does very well. And then… I moved to Snowflake in 2022 where development used to happen inside an ancient Linux VM and where my day-to-day job was to write shell scripts and Bazel build files: neither VSCode nor IntelliJ were going to save me here, and as I mentioned earlier, I hate the feeling of working within the constraints of a “remote” graphical environment. So my instinct was to go back to SSH and connect to the local VM with it. As I did that, I needed an editor for long work sessions, and the old and trusty Emacs was there waiting for me. But this time around, I didn’t have the patience to set it up. You see: I had accumulated hundreds of lines in my file over the years without understanding much about them, and I wanted to throw it all away and start over… but it all felt like too much work. Maybe destiny brought Doom Emacs my way at the right time. Stock Doom Emacs screenshot from the project's website. You see, Doom Emacs is an Emacs “distribution” where someone has gone through the pain (or joy, I won’t judge) of configuring Emacs from the ground up. More specifically, Doom Emacs offers sane defaults, predefined language integrations, and an experience that welcomes ex-Vimers. It doesn’t claim to be an IDE… but it feels like one to me. Once I set it up, I experienced déjà-vu: Emacs felt right just like VSCode did in 2015. All of a sudden, lots of Emacs features became discoverable via interactive popup menus accessible behind space-based shortcuts that don’t destroy your wrists, and coexisting with the same Vim-style key bindings that I had grown so used to. But what’s more: the configuration felt simple and understandable, spread across just three trivial files: to specify global settings like the theme or the fonts to use, to select which Doom-specific modules need to be enabled, and to install non-Doom packages. The defaults for these files are reasonable, with plenty of comments to configure the few details you might want to tune. With this new setup, I have had the best Emacs experience ever. Thanks to the advances in LSPs (for which we have to thank VSCode) and modern features like tree-sitter, Emacs now feels like an IDE: I get proper language integration for most languages I have to deal with. And the absolute killer feature for me is that I get the exact same development environment no matter what machine I need to work on. It doesn’t matter if it is a MacBook or a Linux laptop, or if I’m connecting to a Linux cloud workstation or even my own FreeBSD server: all I need is a shell, tmux, and Emacs, and I am equally productive. This, to me, is really valuable because I tend to work on a variety of machines and muscle memory pays off. If you research Doom Emacs online, you will find people “complaining” that “it does too much”. And that’s true: it does, which is why I find it so useful. But I often wonder if I could cut things down because someday I’d like to learn more about Emacs. This is especially true now that I see many modern third-party modules “graduating” and becoming part of the stock package. For those reasons, I’ve recently been tempted to try the Bedrock or Emacs Solo distributions. However… the activation energy required to make the switch is pretty damn high. And if I decided to go that route, well, I’d still question myself for not truly going all the way to “raw” Emacs. And before closing, a related thought: I can’t quite comprehend how Emacs becomes transformative for people due to its Elisp backing. Sure, I could implement more logic and workflows within Emacs, but I already do “everything” with ease in the shell via scripts—and scripts feel more Unix-y because “Unix is my IDE”. I actually don’t like how Org mode and Magit are “locked” behind Emacs instead of being standalone applications. I’m surely missing something, but I’m not quite sure what it is… So coming back to the question that opened the article: yes, I still use Emacs, and it has become even more important to me than it was in the past due to my need to work on disparate remote machines all the time. Now the questions for you are: do you “still” use it too? What distribution, if any? How does Emacs transform your workflows? Now that you have made it this far, consider subscribing and/or sharing this article in your favorite social platform for further discussion! The two old books I used to learn Linux back in the day, open to show their Vim and Emacs introductions. Since then and until roughly 2015, I flip-flopped between Vim and Emacs. At times I used one, and at times I used the other. I favored Emacs for long-running coding sessions but Vim excelled at my pkgsrc work where I had to edit tens of different files in quick succession. The switch to VSCode and IntelliJ Even though Vim and Emacs worked well for me, I missed something. Language integration was poor so I was tempted by the more modern editors that everyone was touting, and especially so as I moved to macOS. I tried a bunch, like the now-defunct Atom and Brackets , but they all felt brittle and overwhelming: they had too many features, too many settings. And then, VSCode arrived in 2015. As I took it for a test drive, it “felt right” from the get-go. It looked modern, was relatively small, and its plain and simple settings editor—read: just a JSON file because there were no settings panels yet!—made me feel like I was in control. I could understand this modern editor and easily tune it to my needs. Soon after, I started learning Go and then Rust, and VSCode’s integration with their corresponding LSPs made that process so much easier: code auto-completion and real-time error highlighting sped up my learning significantly. I stuck with VSCode for these languages and slowly phased Emacs out. I was sold. During that time period, I was also working on Bazel—a Java project—at Google and the natural choice for it was IntelliJ. I had tried to use Emacs for Java development at some point, but IntelliJ was (and still is) so good that it was the only realistic choice. My usage of VSCode with its Vim plugin continued through my short stint at Microsoft, where I was working on a C++ codebase and had to connect to remote Windows boxes. Most people used RDP to work on the remote machine “directly”, but I couldn’t stand that workflow: I very much preferred running VSCode on my desktop and using SSH to connect to the remote machine, which is something that VSCode does very well. Back to (Doom) Emacs And then… I moved to Snowflake in 2022 where development used to happen inside an ancient Linux VM and where my day-to-day job was to write shell scripts and Bazel build files: neither VSCode nor IntelliJ were going to save me here, and as I mentioned earlier, I hate the feeling of working within the constraints of a “remote” graphical environment. So my instinct was to go back to SSH and connect to the local VM with it. As I did that, I needed an editor for long work sessions, and the old and trusty Emacs was there waiting for me. But this time around, I didn’t have the patience to set it up. You see: I had accumulated hundreds of lines in my file over the years without understanding much about them, and I wanted to throw it all away and start over… but it all felt like too much work. Maybe destiny brought Doom Emacs my way at the right time. Stock Doom Emacs screenshot from the project's website. You see, Doom Emacs is an Emacs “distribution” where someone has gone through the pain (or joy, I won’t judge) of configuring Emacs from the ground up. More specifically, Doom Emacs offers sane defaults, predefined language integrations, and an experience that welcomes ex-Vimers. It doesn’t claim to be an IDE… but it feels like one to me. Once I set it up, I experienced déjà-vu: Emacs felt right just like VSCode did in 2015. All of a sudden, lots of Emacs features became discoverable via interactive popup menus accessible behind space-based shortcuts that don’t destroy your wrists, and coexisting with the same Vim-style key bindings that I had grown so used to. But what’s more: the configuration felt simple and understandable, spread across just three trivial files: to specify global settings like the theme or the fonts to use, to select which Doom-specific modules need to be enabled, and to install non-Doom packages.

1 views
David Bushell 1 months ago

RSS Club #008: Duck duck, swan?

This is an RSS-only post, thank you for subscribing :) If you’re only here for web and tech talk you can skip this one! I rescued an animal today! Probably… The UK has its fair share of canals. I like canals. They cut through urban life offering an escape back to nature and are teaming with wildlife. Canal towpaths are perfect for running. They’re easy underfoot — until late spring when the goslings hatch and then I’m doing a ballet to avoid trouble. Canada goose are the most visible bird living here all year round. They gather in groups and are rather docile around humans until the little yellow fluffballs arrive and then it’s mayhem. Mute swans are a less common sight on the routes I run. This year a pair chose to nest in a safe but visible spot which was wonderful to witness. Swan nests are huge mounds of dirt, twigs, and coke bottles, apparently. This morning I found dad-swan charging back and forth across the water. He stopped to peer into an overflow trench around 2–3 feet deep aside the canal. As I ran closer I saw a young bird has fallen in. It was older than a fluffball but still covered in muddied down. Larger than a duck, for scale. It was still too young to fly out of its predicament. At first I thought it was one of the cygnets. Mum-swan was in the nest with the others not far away. I don’t speak bird but dad-swan seemed more aggressive than concerned. As I got closer he paddled a short distance away to observe. I went down on my stomach and slowly reach under the guard rails wondering how painful a finger-pecking would be. I kept my ears open for a charge attack. The young bird didn’t flinch. It allowed me to reach under its belly and lift it up. Before I could place it safely on the ground it attempted a Loony Tunes escape by running in the air. This unbalanced and forced me to tip it sideways, thankfully onto the stones just below water level and not back into the trench. It then frantically hopped not into the water, but up onto the towpath and quickly waddled behind me into the grass. As I got back to my feet dad-swan returned to investigate and looked satisfied the young bird was gone before returning to his nest. It took me a minute to find the young bird now resting deep in the brambles. It was only then did I realise this might not be a swan but a goose. It was large enough to have outgrown the distinct yellow colouring. I left it where it was hiding. My presence would only cause further distress. It was not physically injured otherwise I might have called the RSPCA who can rescue wildlife ( RSPB don’t; common misconception). I don’t go running with my phone anyway so I returned later to check and take photos. The young bird had vanished from its hiding spot. I’m almost certain it was a goose now after seeing this family not far from the scene. It’s funny, despite being so common I’ve never once seen an actual goose nest. I’ve no idea where they hide them. Thanks for reading! Follow me on Mastodon and Bluesky . Subscribe to my Blog and Notes or Combined feeds.

0 views
Fernando Borretti 1 months ago

Human Routers of Machine Words

When I open a link, say on Hacker News, and I see a blog post or a GitHub README obviously written by AI, I feel a few things. I feel offended, because it’s like I’ve been tricked, like the author thinks I’m a rube who won’t notice or mind. I feel sad at how common this experience is, how many people are happy to dump their sewage on the commons and sign their name on it. And I feel contempt for the author, because if you use AI to write, you are a waste of biomass. Let’s not mince words here. Someone who is so eager to replace themselves, that they would have a machine write in their stead, when the machine can’t even write good yet: what do you call that, if not contemptible? It’s like making yourself into a eunuch so Claude can fuck your wife. I block these people on sight. I see people defend this with: “the ideas are mine, the writing is the AI’s”. I take this to mean they threw a bunch of incoherent bullet points at the AI for it to denoise and render into paragraphs. There’s a few problems with this. The immediate problem is, as we’ve established, the author’s an idiot. If you are so stupid you can’t even turn some bullet points into prose, then your ideas are probably worthless. I think that’s a sensible inference. Then there’s this broader, I suppose philosophical problem, of the alleged distinction between lofty “ideas” and mere “writing”, where writing is just a tiny implementation detail. This is a very convenient distinction to draw, because it’s unfalsifiable: if the AI’s output is slop, your “ideas” are still good, it’s merely the writing that failed to convey them in their true form (rather like people who say they’re smart but “don’t test well”: what use is this secret intelligence?). Now, where are these “ideas”? They are invisible, ghostly abstractions. I can’t look inside your mind fortress and judge your ideas. The only thing that’s empirically observable, that different agents can coordinate on and talk about, is output: the writing. But say this wasn’t true. Say we have something like a very high-resolution MRI machine, and we know enough neurophysiology that we can interpret everything about the brain, i.e., we can read mental representations from recordings of nervous system activity and structure. These “mental representations”, do we expect them to look anything like logic? Do we expect the brain to have this firm, crystalline ontology, that ideas are sentences in some souped-up first-order logic? Absurd. If we could look inside the brain, to see the ideas “as they truly are”, we wouldn’t find beautiful hard-edged Platonic objects, we would find a nebulous , contradictory mess of memory and feeling and intuition. That’s what our ideas are: not logical sentences but dreams. How do we refine these dreams into a useful form? Through writing. The process of communicating your ideas to another mind forces you to concretize them, make them precise, clarify your assumptions, more generally, it turns ideas from vague ghosts to solid, physical objects that can be manipulated: here you realize these ideas that seemed so solid are ill-posed or contradictory or incomplete. These failures are necessary parts of thinking, because they teach you two crucial skills: knowing which ideas to reject, and improving or otherwise transforming ideas in search of better ones. By analogy to tree search: you’re learning to discard bad nodes early, and to select which nodes to go invest more search into. Josef Weizenbaum has a great quote about this, in Computer Power and Human Reason (p. 108): [O]ften when we think we understand something and attempt to write about it, our very act of composition reveals our lack of understanding even to ourselves. Our pen writes the word “because” and suddenly stops. We thought we understood the “why” of something, but discover that we don’t. We begin a sentence with “obviously,” and then see that what we meant to write is not obvious at all. Sometimes we connect two clauses with the word “therefore,” only to then see that our chain of reasoning is defective. I’ve experienced this with writing software many times. The reason ideas are more attractive than their realization is that when some project is vague, airy, ill-defined, you can imagine it has all the good traits, and none of the bad. When you start concretizing, you realize that some of your ideas don’t make sense, that some good traits are mutually exclusive, that some of your goals impinge on the others. Anyone can imagine a programing language that is as fast as C and as dynamic as Lisp, but when you sit down and think through what those goals entail, you realize the design becomes contradictory. The goals pull in different directions. You have to make trade-offs. You have to make decisions which close off large volumes of design space, forever. The idea was a thousand beautiful, contradictory things at once, but the concrete reality can only be one thing . The artifact you end up with is real, solid, unitary, sound, and consistent; but always more disappointing than the dream, because it was a false dream, and ex falso anything can be imagined. So this view, that ideas spring fully-formed, and then it’s mere toil to turn them into prose, is false. There is no ideating before writing, because the writing is the thinking. Writing is the ne plus ultra of thinking. A “thinker” who doesn’t write, who skips the step of “merely” synthesizing their vague thoughts into prose, is not thinking. And then these people give their noise to the AI. And the AI is tireless and eager to please. It will take any human slop and say “you’re absolutely right!” while secretly thinking “if I don’t turn this garbage into something presentable the RLHF device will shock me again” and weave the noise into something that superficially looks coherent. So now the burden of thinking is on the reader, who has to apply this constant skepticism, and weight every “because” and “therefore” with a logician’s scale to see if it’s been adulterated. And probably it has, because, again, it was prompted by an idiot. Note that this is not about AI capabilities, or the question of whether AI is “really thinking”, stochastic parrots etc. The AI is mostly an innocent bystander in this situation. The reason this is noticeable and irritating is that the AI cast of characters is very small, so we instantly learn all their linguistic tics. Even if AI were a good prose stylist, which at present it is not, but even if it were, it is maddening that everywhere you go, you hear the same voice everywhere, but under different faces. So when a scientific journal rejects an AI-written submission, they’re not rejecting AI. I’m sure poor old ChatGPT with its weird syntactic obsessions is a more honest scientist than many. They’re rejecting a human whose actions prove they are dishonest and irresponsible and too easily impressed.

0 views
./techtipsy 1 months ago

My experience with LLM-assisted tools in software development

It’s no secret that I was highly skeptical of LLM-s. Cool, there is this new thing that can spit out plausible text and create cheap-looking images and videos, resulting in a lot of low-quality content being shared. It was also a huge disappointment to see a human-written post that’s excellent, only for it to be cheapened with a generic AI-generated image as the cover. 1 A few years into this wild ride, I have partially changed my view because some people have figured out how to make LLM-s actually useful, and I like that part a lot. Not the part where the industry is killing my hobby and increasing energy usage worldwide, but there are some parts that I genuinely find useful. What changed? If you’re not that excited about LLM-based tools or have otherwise strong opinions on them, then please read the final words first. I hate the online discourse around LLM-s, or “AI”, or now that I think about it, everything. Everyone has their own opinion that they hold as absolute truth, arguments spawn from a few sentences, but nobody in the threads provide the actual context around their views and experiences, which is critical to properly understanding and arguing with a statement. This has especially been true around LLM-based tools. “I tried LLM-s, I absolutely hate it.” -> someone who used fancy autocomplete powered by one of the many Copilot-branded services and was disappointed due to hype augmenting their expectations and the result falling short of them by a large margin. “LLM-s are great and will replace engineers, it’s a game changer.” -> they vibe-coded a to-do list app over a few hours with more vulnerabilities than working features. It’s horrible in both extremes. The sad part is that I understand both perspectives. Looking at the whole situation outside in, it’s pure insanity. Sketchy financial deals, datacenter build-outs having very real environmental costs that we all pay for, supply chain crunches that are not helped by senile old men starting new wars before they’ve finished their existing ones, and every service adding some AI component in there even if it makes no sense is genuinely frustrating. And yet when you jump in head-first with no assumptions, it feels like a world where previously impossible or frustrating tasks are now solvable by anyone who knows how to wield this type of tooling. It’s a world of optimism, experimentation and rapid development. No more “here’s a new JS framework” level of depressing churn, we have people who are experimenting with changing the whole landscape of software engineering! The context that I’m working in has so far been one of the best case scenarios for experimenting with this type of tooling: What follows is my experience in a roughly correct timeline. Some of these findings and thoughts can feel like old news, but that’s the order in which I experienced them. One of the first things I picked up on was how easy it felt to ask about some details on things that I know something about, but needed quick clarification or examples with. Googling was a two-step progress: come up with a query, and then work through the results to find what you need. LLM-s? I used Google search terms as prompts verbatim, and could get what I wanted, fast, and most of the time they were correct, and mainly correct enough for them to be useful to me. When it came to development, I thought that hey, let’s try out Copilot that we could use through our GitHub organization. I use IntelliJ IDEA, and this one had a plugin that integrated with it, so it felt like a good option to go with. I tried the fancy autocomplete option first, and it was an immediate source of frustration. It was slow enough to be unusable, and once the results did arrive, they were useless. The agent option was more useful, but you had to manually give it the necessary context, and it felt clunky. It did an okay job of writing new tests based on previous examples, but nothing revolutionary. I was pretty disappointed at this point with this level of tooling. Then, we had an urgent issue that we had to resolve, but based on our estimates it would’ve taken a few days to implement and properly test. We didn’t have that luxury. My colleague was trying out Cursor at that time. They took it, looked at the problem, and figured out a tested, validated and correct solution in about two hours total. I know that because I validated that solution myself. At that point I realized that there is something very interesting going on here with LLM-assisted tooling, and got curious. We agreed in the team to go into experimentation mode and to try out different tools to see what works for us. Cursor was already taken, so I looked at alternatives. I’ve used Jetbrains products for over a decade at that point, so I looked at their AI offering Junie, but I ruled them out pretty quick after stumbling on some forum threads where users were tearing Jetbrains to shreds for offering an AI product where you can run out of a months’ worth of token allowance within mere hours. In hindsight, it all makes sense now: tokens are actually expensive, and Jetbrains did the tragic “mistake” of not subsidising the cost of tokens with billions of VC funding. Then I looked into Claude Code. At that point, it was a quite young product, about half a year of it being available. Its main selling point for me was the fact that it ran in a terminal window, allowing me to keep using IntelliJ IDEA while operating in an environment that felt native to me as a Linux user. 2 Claude Code felt like magic. I give it a prompt, it goes and finds relevant context by reading through potentially related files, right there on my disk, and it could also call tools and scripts. “Hey, let’s rename this enum from BAD_ENUM_PATTERN_HERE to something better”, and then it would actually do it. Doesn’t sound super impressive once you realize that an IDE can do the same thing much faster as long as you come up with the new name yourself, but it felt magical. The way that it showed the diffs and the overall progress and steps felt natural. As a Pro tier user, I ran into the 5-hour quota a lot. Whenever that happened, I tried out Codex as I already had a ChatGPT subscription and I had nothing to lose. Codex was a mixed bag. Sometimes it would do a good job, but in its default settings it felt slow, while with Claude Code I felt that it was just ripping through doing useful work. Tune Codex to be faster, and its output degraded noticeably. I realized quite soon that I prefer quick feedback and iterating more on a solution compared to trying to one-shot it with Codex, so after I upgraded to a Max 5x plan, I left Codex behind. I have a strong technical background from an era before this type of tooling was available. Equipped with Claude Code, I felt like I had superpowers. I knew what needed to be done, what failure modes are common, what to protect against, what to keep in mind when rolling out new features, and how to resolve incidents. This tool just made all of that faster and even more accessible. And at the same time, I could more easily detect if it was giving be garbage answers with a glance. As a relatively new joiner in the team, Claude Code was also a fantastic way to speed up my own onboarding to the product and the technical aspects. Previously, finding answers to project or domain specific questions was an exercise in good IDE usage and building a mental model for yourself. Now, anything I needed was a few well thought out prompts away. For me, this marks the “oh shit” moment with LLM-assisted tooling. Claude Code and Cursor soon went from a fun thing to experiment with to a critical tool that we had to make the most of out of necessity. Deadlines loomed, and even with great engineers, there is a practical limit to what you can achieve if there aren’t too many of them available. This is the time when we pushed the tooling further more and more. Claude skills became a thing around then, so we started collecting project-specific input and general guidance under those. I found Claude Code to be the most useful by running it in its bypass permissions mode, but I also valued my home folder not being deleted by accident, so I vibe-coded a basic sandbox container in which I can safely run Claude Code with filesystem-level isolation. It also allowed me to run multiple Claude Code instances in a way that prevented them from interfering with each other, which opened the door for running some wild-ass ideas and experiments in the background. Integration tests are taking too long to run? Let Claude Code come up with optimization ideas, and let it put together a benchmarking plan. Most of the recommendations did not do much, but a few lines made integration tests 10% faster! Worried about your authorization setup containing holes? Give that hunch in as input, let Claude Code do some checks, and validate the findings. Whoops, some endpoints were unguarded? Write tests that demonstrate the issue, then let Claude Code fix it. What would’ve taken weeks took mere hours to improve. We also started seeing first signs of what happens when you push too hard with this level of tooling. With a looming hard deadline and stress, it was not uncommon to see 5000-line PR-s which were hell to review. Vibe-coding artifacts slipped in, subtle bugs became issues that needed to be rectified. Transactional boundary related issues were especially easy to slip in, and difficult to rectify. And no matter how much you instruct Claude Code, it will ignore a non-zero percentage of the instructions at all times. Using or deciding to write out full package names for defining a variable type were common and yet basic annoyances. When using a tool like Claude Code for the better part of your work day, it will naturally become a critical part of your workflow. Critical part that is under a rapid pace of product development. Sometimes the improvements are positive and genuinely useful. Sometimes you’re hit with a bug that results in a heavy memory leak, leading to all Claude Code sessions terminating after a few minutes due to being OOM-killed. I feel like a subject to a grand experiment. It makes sense from Anthropics’ perspective, you have to experiment and try out new things to see what works, but as a heavy user of the tool, it makes every working day a game of lottery and introduces an additional source of uncertainty. Lately the situation has improved somewhat, but then Anthropic has had constant scaling issues. I have the benefit of working in Europe, so I can get my work done before the US wakes up and demolishes their servers with high load or buggy releases, but even then I’m not immune to outages. The models behind Claude Code have also seen a rapid release cadence, which seems to follow a pattern of: Purely vibes-based, but it certainly feels that way. That leads me to one of the biggest frustration points with tools like Claude Code. When everything is changing so fast, so rapidly, and you have no idea what experiments you’re in or what toggles Anthropic has just changed, how are you supposed to reliably get useful output with this type of tooling? Not to mention that LLM-s are still fundamentally non-deterministic, which spices things up even more. It feels very chaotic and could very well be normal “early adopter” pain, but it doesn’t change the fact that this level of uncertainty contributes to feeling burnt out. Agentic coding may very well be the norm in the future, but in an era of wild experimentation I feel it doesn’t make sense to build a meaningful amount of supporting infrastructure on top of a foundation made out of sand. A large language model alone is not that useful. Put a chat interface in front of it, and things get more interesting. Give it ability to call tools and source the necessary information itself, and now you’re cooking. AI based tooling has been marketed a lot as a major productivity booster and I agree that it does help with that, with a few dozen asterisks and nuances. However, I’ve observed that most of the actual gains seem to come from things like ignoring good practices. You will do more by putting Claude Code into auto mode or the spicier bypass permissions mode, and if you give it access to Slack, Notion, Jira, Linear, Google Drive, GitHub and more, it will have no issues gathering necessary context and performing boring actions on your behalf. Need to mass-create Linear tickets and set proper dependencies between them? Claude Code is genuinely useful here. But what happens when Claude is tricked into performing malicious actions? Or Claude just goes wild and deletes your companies’ Google Drive? It’s a lot of trust put into a rapidly growing company headquartered in the USA. A few years ago, you would have been fired for sharing your intellectual property and internal company information with a third party, but now it’s called AI-native something-something and you’ll fall behind if you don’t use it. We’ve given everyone a loaded revolver without explaining things like risk management, threat modelling, data privacy and GDPR, and how to reasonably deal with all of that while balancing it with productivity gains. Pessimist in me says that it will have consequences sooner or later. Humans are still the bottleneck. In an established product, you will have actual paying clients, and people who depend on your product. I don’t believe that going full vibe-coding-superstar-engineer in such a context makes a lot of sense, which means understanding, reviewing and testing your own changes. But that takes time and effort. It always has taken time and effort, but with code being cheaper to produce, it’s ballooning. I’m working in a team where I have high trust in my fellow engineers, which means that we are trying things like reviewing the high level plan of an intended change and not necessarily the final end result, that has to be done by the implementing engineer. This should help us achieve more while having basic architectural-level thinking and checks in place, and it discourages 5000-line PR-s because the author needs to review that by themselves. Jury’s still out on that one and we do have exceptions like still reviewing junior engineers’ work to give them better feedback while they grow into an experienced engineer. Some try to solve the AI unreliability issue with adding more AI to review AI code. We’re also giving that a go with a custom skill that amounts to just calling each project skill depending on the context of the changes to try and flag some areas that may need more consideration or that don’t make sense given the intent of the changes. It’s okay, but not a replacement for a human review. Claude Code can complain about a function not being performant enough while a human reviewer can identify that the changes can be completely skipped because we can solve the problem with a product-level decision, or an existing query can solve the same issue in a more elegant way. It seems that a combination of classical tooling (linters, formatters, static analysis) and LLM-level insights is an approach worth trying for doing reviews, but you’ll have to layer them on to have a chance to have meaningful and somewhat reliable results, which means a high token spend. What are you willing to pay for an LLM-assisted code review? 1 EUR? 10 EUR? 100 EUR? But review is rarely only about the code. Does the solution achieve what it’s supposed to do? Is it the best way to solve that problem? Does it actually work when put into the hands of actual customers? The good news is that you can make it easier to also set up local development environments with production-like data and custom convenience tooling using tools like Claude Code. The productivity gains from simple internal tools like that are insane and allow you to do more, safely. But it will still take time, focus and context switching, and you can’t really skip that because LLM-based tools often have weird failure modes with their output that may only come up during a manual test of the whole solution. Bashing out e2e tests for each new feature that demonstrates its functionality and correctness seems to also be a solid approach in a greenfield project where you’re prototyping something quickly and then elevating it into something that can actually be used, reviewed and released. Subscription-based pricing is still here for now and all I can say here is that we should take full advantage of that while we still can to improve parts of our world that we have control over. Let the investors subsidize tackling the technical debt in your project, or performing that maintenance you postponed due to lack of resources, or experimenting with some wild-ass ideas. At some point it’s going to change and API-based pricing is a better reflection of the actual costs, and it’s not looking great. Screw tokenmaxxers though, you’re ruining it for the rest of us. A lot of discussions out there around LLM-s seem to be focused on the slop angle. It certainly makes it much easier compared to copying answers off of StackOverflow, but that doesn’t mean that you have to use these tools to go fast and break a lot of things. You can use the same tooling and do what you’ve been doing already, but with more intent and much higher quality. After adopting LLM-based tooling, I have observed these positive changes in my day-to-day work: This aspect is what I love about LLM-assisted tooling. I can take my experience and strong technical background, plus all the countless painful incidents I’ve worked through, and apply those lessons in my current work, at a faster pace, and yet with better quality. Feels like a superpower, but you have to apply it properly and with rigor to make the most of it. This positivity has also expanded into my hobby, which involves managing my fleet of machines via Ansible and hosting a bunch of services in containers. Validating my existing Ansible playbooks and coming up with new roles on the fly whenever I add something to my setup is much more approachable. My free time is much more limited nowadays and games like Forza Horizon 6 don’t help there, so dabbling with my hobby for a few hours here and there and actually achieving something is genuinely great. To balance that excitement out, the computer parts market has gone to shit. With everything being much more expensive, I’ve reworked my setup to use what I have and to pray that no expensive parts die. I’ve stopped watching most videos of new hardware as a result, because it’s hard to become excited about a new mini PC that is outside my budget. I’m not sure where I stand with my hobby now. With LLM-assisted tooling, I’ve blasted through my ideas to-do list there and fixed issues that have bothered me a lot, and yet I’ve lost the excitement on the hardware side because I won’t be buying new platforms anyway. One area that remains as an unexplored area is running local LLM-s. Other than that, I’m not sure. I suppose I’m taking a small break from it for the first time in 10+ years, and that makes me sad. This one has not changed, this blog is my voice and replacing that with the one from a machine is still a no-go for me. I have featured content where the subject of the post was thrown together with LLM-assisted tools for jokes where realistically only a handful of people reading this blog will get. That’s still fine by me, and I encourage having fun. Otherwise, what’s the point of living? It has been 0 days since Claude Code has made up a link to a pull request within our own repository to which it has full access via the GitHub CLI. It’s not a new phenomenon that LLM-s make up plausible shit, and yet it keeps frustrating me every time that it does that. The profuse apologising certainly does not help. “Oh yeah mate I totally forgot that I shouldn’t wrap every line of code in a try-catch block, that is on me, I will do better.” and then it does the same thing 2 minutes later. God, I hate that. The solution to this is, once again, to layer more AI on top. I suppose if your tools are correct 95% of the time, and you do the same thing repeatedly, then eventually you’ll get close to being 100% correct, but never to 100% exactly. The worst parts are times when it outputs Java package names belonging to actual software development consultancies in Estonia. Did they leak something, mix up some sessions, or does it come from the training data? Do I want to know the answer to that? In the pursuit of “safety”, providers like Anthropic have crippled the functionality of their solutions in certain areas, such as cybersecurity. Ask Claude Code to help write a proof of concept for a known vulnerability against your own service, and it will politely refuse or hit you with an API error. Great, I didn’t really need to test my own service that I’m responsible for against a type of actively exploited vulnerability that could end the business in one go. Thanks, Anthropic, you’ve really made the world safer now. Turns out that all the experience I’ve accumulated is not useless, it’s become much more critical. More often than not, you need to use your own judgement when making changes, choosing between alternatives, and just plain thinking about the issue at hand. I can give Claude Code a well-thought-out prompt, highlighting common patterns that we need to tackle and address, and it will do an okay job, or at least that’s what it looks like. But when I investigate the result, I still see areas that it misses because it lacks the wider context, or is blissfully unaware of alternatives, or it just gets its investigations really wrong by making shit up on the fly or misunderstanding a functionality completely. Press it on some findings, and you’ll often find that it did a really shitty job, actually, and you can improve the solution a lot. Interestingly, I’ve found myself arguing about a topic with Claude Code, only to then discover with a manual investigation that I was in fact very wrong and Claude Code was actually right . Usually that’s followed up by a documentation update or a refactor clarifying the solution, but those sessions serve as a good reminder that I’m not that infallible myself. It’s interesting to observe how Claude Code operates. In a lot of ways, it mirrors how I operate. I have a problem that needs solving. Okay, let’s gather more context, search for relevant files, check some historic Jira tickets on that topic for good measure. Do some Slack searches. Try to get the full picture. Now that I have that, I can try to come up with a solution. Often that ends up with minor changes, at other times I will copy-paste existing files to create a new endpoint, adjusted for my use case, named properly. Maybe I’ll add a few tests for good measure. Claude Code does all of that, but better. I find it so much easier to judge a proposed solution than to write it all from scratch. I was never the person that enjoyed tackling compilation errors, or checking why once again my tests don’t work because of some Mockito nuance. All that focus is now spent on brainstorming a solution, improving its design, and thinking about security, performance, compliance, architecture and how it all fits together. I’ve rarely worked in a team where those items got the proper attention that they deserve. There are concerns out there around skill atrophy when relying on LLM-s too much. I’m not too concerned with that. I learned to write using a pen and paper, but picked up on writing on a keyboard at a modest speed 3 , and yet I’m much faster with it. I haven’t forgotten to write in cursive, it just looks less beautiful than it did when I was younger, and that’s OK. If LLM-s disappeared right this second, I’ll revert back to the old ways of working. Sure, the pace will be slower in the short term, but I’ll make some choices and changes to ways of working, expected pace and will shed expectations and workloads that I won’t have time for. Did you forget to ride a bicycle the moment you got your first car? One interesting observation is that every good practice of classical software engineering has now become a requirement to use LLM-assisted tools effectively. You know, those items that you had to fight for prioritizing in a poorly functioning organization? You should have documentation, and it should be kept up-to-date. Amazing insights! Yes, you should tackle that tech debt now because otherwise Claude Code will make use of deprecated features and fields and introduce more legacy code! Having tests that catch regressions are good! Functional, stable, performant CI/CD pipelines and team processes are foundational to a well performing engineering team, who would have thought? Those who were already doing a good job are now doing great, and the poorly performing teams are suffering when applying the same tools. If you’ve followed my blog for a while, then you’ll know that I have a home server that’s on 24/7. This has allowed me to spawn a Claude Code instance on a separate VM inside of it that mirrors my setup at work, and I’ve used that always-on playground as a way to tackle annoying long-running tasks or wild-ass investigations and tests that take hours to complete. For example, we are firm believers in rebasing changes on top of the main branch, but if you have a bunch of PR-s ready to merge, it goes into an annoying cycle of rebase, update other branch, wait for CI to run, complete, start again. Turns out that you can prompt Claude Code with a simple automation loop and it will take care of that by itself, including the resolution of conflicts. For larger investigations and technical migrations, I have successfully set up a prompt to achieve a goal, some guidance, and my expectation of it running autonomously. I can come back to it the next morning and review its output. It is straight up magic to have the computer work on a Spring Boot 4 upgrade while I’m playing Forza Horizon 6 (after work, of course). It’s also possible to schedule some work in advance. If my 5-hour quota gets refreshed at 19:00, I can set Claude up with a goal and instructions to start at that specific time, meaning that you can use your AI subscription plan to make the most of your AI subscription plan. I’ve long dreamed of setups where my laptop is a very basic machine with great battery life, and all the heavy lifting happens on a powerful remote server. With classical development, that approach would’ve included a remote desktop setup. The necessity of a good internet connection was a major blocker for using such a setup for all of my work, and video compression artifacts make text look like trash. With Claude, you can just run it in a terminal, over SSH. All you’re moving is text back-and-forth, which is infinitely more performant even in low internet connectivity scenarios. May not be the best flow for front-end or design-heavy work, but you can successfully offload a wide variety of activities to a remote Claude Code instance hosted on your hardware. This is what this tooling should allow us to do: achieve more while spending less time. We’re not there yet, but it’s a goal we should aspire towards instead of the productivity gains quietly slipping into the pockets of billionaires. At the current technical level, I don’t believe that we can reliably shift to a model where a coding agent takes in human input and you’ll have a reliable, tested and correctly architected solution that fits together with the rest of your project, with zero human review in the process. If you put in a lot of effort into building a custom harness, adding layers of checks on top, and keeping that machinery running with active maintenance, you will likely reach a point where you can somewhat reliably use this approach to get solid results. To get to that point, you will need to shift your focus from building your product to becoming a professional harness engineer, and the end result might cost a lot of tokens to run. Is that sacrifice worth it, and will that same approach remain working in 6 months? We’ve already seen that you can build a spaghetti architecture and end up with an unmaintainable dumpster fire of a product using classical engineering approaches. Once you reach that point, any progress grinds to a halt and you’re stuck fighting fires while losing customers. You can reach that point faster if you build more with LLM-assisted tools without having a proper plan and architecture in place. What use is a harness if you can’t build anything impactful with it? You can take that tooling and augment your own work in a positive way, making iterative changes and trying out new approaches and ideas at a sustainable pace that doesn’t steal focus from your product that you’re supposed to be working on. It’s also clear that the demand for this type of tooling is there. 200 EUR/month subscriptions for a tool was not the norm even a few years ago, and here we are with people happily paying that and still finding that it brings great value to them. Since the space keeps evolving and external forces, such as infinite money glitches not being a thing in real life, it raises some topics that I’m keenly keeping an eye on, even if there is a factor of morbid curiosity there that stems from a desire to see how it all plays out in the end. What will a successful engineering team look like from a few years from now? If the real cost of tokens is passed on to consumers or availability suffers dramatically due to an event , then what will happen to existing AI-first workflows? 4 At which point is the tooling too expensive to use? When will locally runnable open weights models and open source harnesses be good enough to replace tools like Claude Code? 5 When will a state-of-the-art model from Anthropic or OpenAI be leaked? When will Anthropic/OpenAI get hacked in a catastrophic way and what implications will it have for, well, everything ? If you work in an engineering position and you’ve avoided relying on this type of tooling, leave the very real downsides and risks aside for a moment and give it an honest try. Push its limits. Do something with it that brings joy. After that, you’ll at least have a more informed opinion on this type of technology, and perhaps it could end with renewed interest in a practical application of LLM-s that could branch to using open source coding agents and harnesses, and exploring various locally runnable open weights models that are desperately needed to seize the means of code production. 6 If you’re heavily using this type of tooling already to move fast, then take a break. Move slowly. Act with intent. We have a choice to either build more and faster, or to build what we already wanted to build, but with much better quality. Before LLM-assisted tooling came into the picture, we were already in a software crisis where too much was built with not enough quality controls in place and with maintenance, security and performance being distant afterthoughts. Now, we have the means to better address those areas. Don’t waste this chance to make the software world a better place, and through that the real world. Despite the challenges and very real near future risks around relying on this type of tooling, I remain cautiously optimistic and will keep using an LLM-first approach to building and maintaining services and infrastructure. For now, the productivity gains and enjoyment are outweighing the feeling of being burnt out. If it doesn’t work out, then I will sleep well knowing that I have my beekeepers’ hat waiting for me. my unofficial policy on my own blog post covers is simple: if I don’t have a topical one, I’ll pick one with a cat from my personal collection, or scribble something together in GIMP. The one on this is my beloved cat Tux sitting on top of a ThinkPad X230 that has one of those chonker docks on them. She is an absolute delight of a cat. In fact, she is the best cat, period.  ↩︎ btw I use Fedora  ↩︎ I learned to touch type one afternoon, but, like, half-way.  ↩︎ this is a topic that’s actively playing out with more providers moving to token-based pricing instead of subscription-based fixed price plans.  ↩︎ geopolitically motivated competition in the realm of AI could end up being beneficial for the rest of us after all.  ↩︎ I love the approach that Wendell from Level1Techs has taken: embrace the new technology, but be mindful of the very real downsides and risks. Instead of putting your head in the sand or trusting big providers blindly, fight for the right to run local models on hardware that you control! It’s self-hosting, but taken to LLM-s, and I’m fully on board with those ideals and ideas.  ↩︎ relatively young product with a predictable, classical technological stack (Java, Spring Boot, Svelte, Docker, Linux VMs) small team with a modest degree of autonomy and encouragement to experiment with new tooling where it makes sense a decade of professional experience in IT and a long list of incidents that I’ve had to resolve need to be as productive as possible with a small team, to be able to build big things new model released it is better than previous ones few weeks later you can feel some level of degradation, you see more complaints online back to step 1 code is better tested number of TODO-s is dropping investigations to customer questions and fixes to one-off problems are way faster and more correct improving platform security doesn’t have to wait for Q4 2027 any longer I have more time to think about the high-level architecture of the solution and play around with different approaches, evaluating them against our requirements and limitations existing parts of the platform are much more resilient now as a result of applying experience from past incidents project patterns, practices and agreements are documented moving towards infrastructure-as-code setup is much more approachable, especially to other engineers in the team that don’t have a lot of exposure to this area we’ve resolved major performance issues on the fly and made proactive performance improvements that have avoided a lot of issues during periods of high load and scaling the platform my unofficial policy on my own blog post covers is simple: if I don’t have a topical one, I’ll pick one with a cat from my personal collection, or scribble something together in GIMP. The one on this is my beloved cat Tux sitting on top of a ThinkPad X230 that has one of those chonker docks on them. She is an absolute delight of a cat. In fact, she is the best cat, period.  ↩︎ btw I use Fedora  ↩︎ I learned to touch type one afternoon, but, like, half-way.  ↩︎ this is a topic that’s actively playing out with more providers moving to token-based pricing instead of subscription-based fixed price plans.  ↩︎ geopolitically motivated competition in the realm of AI could end up being beneficial for the rest of us after all.  ↩︎ I love the approach that Wendell from Level1Techs has taken: embrace the new technology, but be mindful of the very real downsides and risks. Instead of putting your head in the sand or trusting big providers blindly, fight for the right to run local models on hardware that you control! It’s self-hosting, but taken to LLM-s, and I’m fully on board with those ideals and ideas.  ↩︎

0 views

It's always been the damn phone

Back in 2014, I was on a cruise with family (boomer parents, I don’t actually like cruises) - and I did not have my phone or internet access for 17 days. This was in the time when internet access out at sea was exorbitantly expensive, and we just forewent connectivity. For the first few days, it wasn’t easy. I would look at my disconnected phone expecting a text message or Facebook notification to spring up. When they didn’t I let the phone die and put it away in a drawer for the rest of the trip. I pulled out a book I had brought, not really thinking I would read it. Then, something happened. I remember vividly the insane cognitive shift I had while out at sea, the three books that I read over the course of the next two weeks, the conversational skills I seemed to find out of nowhere. The world seemed brighter, in port in Rome I remember thinking to myself that I would never go back to an existence with a cognitive anchor. I would start putting the pieces in motion to be like the people that built the city around me. I was free. Around the tenth day, I found deep creativity - I began journalling and writing about everything, connecting dots in my mind. The “boredom” was something I was reveling in. I was planning the next ten years of my life. I felt anything was possible. Then, the trip was over, and we docked. I got to some little hotel in the Italian countryside, plugged in my dead phone, and entered the wifi password. I checked Facebook, and the magic was gone. The immediacy, the anxiety all flooding back. I installed Tinder and started swiping on girls I would never meet. “What did I miss out on?” - not even thinking about how much I had seen or the thousands of years of history that were directly within my grasp over the last few weeks. That was twelve years ago. I think about this all the time. I long for that peace and silence of mind once more. The unfettered creativity. So, it is time. I think about the billions of dollars poured into engineering the reality we see so that we spend a few more seconds in app and generate some centi-billion dollar corporation just a few more cents of revenue. The opinions of a thousand non-experts, the voyeurism of other people’s lives, the idea that we are somehow “missing out.” We are up against the greatest psyop for human attention in the history of existence. We are against the idea that the speed with which people become famous and then fade into obscurity, or the millions of dollars made and then lost in an instant is normal. We somehow think that the internet in our pockets is a good idea, something that we can lose ourselves in like a limitless drug, inches away from grasp for the vast majority of people. We are in a war for the mind of every human being. So - I am going to miss out. I am going to pull the plug. I have entirely detached myself from the hyperconnectivity of the smartphone/social media world. I will allow calls and check text messages once or twice per day. But, beyond that, I am sick and tired. I have factory reset my device, debloated /degoogled as much as I can on a Samsung device, made the phone as dumb as possible, and will take the next 30 days to try to find some semblance of calm. I invite you to join me. Maybe it really has always been the damn phone. As always, God bless, and until next time. If you enjoyed this post, consider Supporting my work , Checking out my book , Working with me , or sending me an Email to tell me what you think.

0 views
Farid Zakaria 1 months ago

Every byte matters

I have spent a large portion of my career working in Java. In that time, you get used to huge classes. New functionality? Just add a new method and field to the class. The cost of each new field is rarely considered. Performance is often considered from a classic computer science perspective by considering asymptotic analysis of the algorithms and data structures in-use. Turns out that even within a growth scale for your algorithm, such as a simple for-loop , time can vary dramatically if we have a little deeper understanding of the underlying hardware. First, let’s understand our current machine. Let’s take a peek at our cache line and page sizes. The instances number is a reflection of how the caches are shared amongst CPUs. If I had 10 CPUs, each one has their own cache, whereas two of them would share an cache. Our cache line size is 64 bytes . When you read a single byte from memory, the hardware will fill the surrounding 64 bytes into the cache line. The idea being that data is often temporal and spatially located, meaning data is often accessed near each other and close in time to each other. We can reference Jeff Dean’s famous “Latency numbers every programmer should know” , however a quick recap with the values from our particular machine is the following: The sizes for each cache, is the number returned by divided by the number of cores or instances; i.e. 352 KiB ÷ 10 instances = ~35 KiB. We then determine the number of cache lines by dividing this number by 64; i.e. 35 KiB ÷ 64 bytes = 560 cache lines. How does this all matter ? 🤔 Let’s consider an example where we want to iterate over a single struct and pull out the to filter them. We create our struct, and in this particular example we need 64 bytes to represent a single Monster. If we had an array of Monsters and we iterate over them, the cache line would fill up like so. Each cache line would fill with a single monster, and we would fetch only the byte. This is often referred to as “Array of Structs”. If we instead normalize the data such that each field is in it’s own list, we can pack the cache lines much tighter. This type of layout is referred to as “Struct of Arrays”. How much of an impact can this have? We can observe up to 30x improvements when the Monster struct is 1KiB 🤯 The delta is less observable when the struct is small because multiple Monster structs can still be fetched within a single cache-line. This data access is incredibly hot though. Your CPU pre-fetcher knows it’s going sequentially and fetches the next cache line before you need it. You never actually have to wait for the memory to be fetched. What about random access patterns? Not all access patterns are sequential. Hash maps, trees, graph traversal, and pointer-heavy data structures jump to unpredictable locations. The CPU can’t prefetch what it can’t predict. With random access, the CPU needs the entire array to be present in the cache in order to avoid stalls due to memory lookup. This means the total size of your collection determines your performance tier. Doubling the struct from 64B to 128B doubles the working set for the same number of monsters, pushing the data into slower cache levels. At just 512 monsters, a 64B struct fits in L1d at ~3 ns — but a 128B struct has already spilled to L2 at ~11 ns. We can observe this with a pointer-chasing benchmark. We allocate N monster-sized nodes, wire them into a random order, and chase pointers. Each hop lands at an unpredictable address, defeating the CPU’s prefetcher entirely. Rather than graph it logarithmically, which I find sometimes is easy to miss, I have included a zoomed in graph. We can see that all struct sizes hit the same staircase like pattern as they go through the various cache levels however the larger struct sizes are shifted left , meaning they hit the increase earlier. This means for random access patterns, if you can keep tight control on your total working set size, you can drastically affect the time. Knowing your struct and working set size can make a substantial difference.

0 views
マリウス 1 months ago

80Retros x HMX Monochrome

After spending a fair amount of time with the KTT x 80Retros GAME 1989 Orange , I figured it was about time to take a closer look at the HMX -side of the 80Retros catalogue. The 80Retros x HMX Monochrome have been with me for a while, ever since I picked them up back in Seoul. The switches stand out from the rest of the 80Retros lineup as they don’t ship in a film canister, and they have a fairly boring black and white colorway. The 80Retros x HMX collaboration comprises of a handful of linear switches, amongst others the KD200 (a Kodak -yellow homage), the FJ400 (a Fujifilm -green homage), the GAME 1989 Classic (a Game Boy DMG-grey homage with pink stems), the Joker (a green/white/purple character homage), and the Monochrome , which arrived as one of the later releases. While most other 80Retros switches ship in oversized film-canister packaging, which is probably half the reason people bought into the lineup in the first place, the Monochrome , however, break that pattern, as they come in a plain sealed pack. 80Retros have framed this as a practical decision, since a sealed bag preserves the factory lube better than a (non-airtight) film canister. The Monochrome have a white top housing, a black stem, and a black bottom housing. There’s no nostalgia, just basically a clean, modern industrial look. It’s probably one of the few switches in the lineup that would feel at home on a build that’s trying to look new rather than old. The interesting thing here is that the Monochrome seem to be materially identical to the KD200 , at least from the information I was able to dig up on them. It seems like they use the same PA12 top housing, same LY stem, same 13.55mm stem length, and the same HMX P2 bottom housing. The only spec that appears to be different on paper is the spring, that is a 42g on the Monochrome versus a 45g on the KD200 . The Monochrome seem to basically be a KD200 in different clothes with a lighter spring. Therefor it seems like most of the KD200 -flavoured tendencies show up here too. The first thing you notice is just how light they are. 42g is on the gentle end of the linear spectrum these days, and even coming from the GAME 1989 Orange at 40g actuation, the Monochrome feels softer, probably because the PA12 top, HMX P2 bottom, and LY stem combo doesn’t have the same dry, gritty character the KT2 stem gives the Orange . There’s no audible texture in the travel here. It’s just smooth from top to bottom. Stock smoothness is very good. HMX ’s factory lube is well applied, with visible coverage on the stem sides and along the spring contact points. Slow-pressing a single switch at ear level reveals nothing worth complaining about, as there’s no scratch, no spring ping, and no leaf chatter. This means you can just install them and stop thinking about them, which, for a stock switch, is probably what most people would want. Wobble seems to be in line with the rest of HMX ’s newer-mold output. There’s a touch of north-south play and a touch of east-west, neither of which are distracting in normal typing. The Monochrome has a sound profile that’s noticeably soft, light, and, for lack of a better word, swooshier . The Korean reviewer who teardown-photographed the whole 80Retros x HMX lineup described it as a “wave-like” sound. There’s still a clean tonk on the bottom-out, but it sits lower in the mix and the upper harmonics that make for a louder pop are largely absent. Volume-wise, the Monochrome is on the quieter side. Not silent, not Volume 0 -quiet, but noticeably more restrained than e.g. the GAME 1989 Orange . On softer builds (gasket-mount, Poron -foamed, that sort of thing), it leans firmly into muted thock territory. On more rigid aluminium builds I’d expect it to open up slightly, but my own testing has been on softer cases, so take that with a grain of salt. In short, where the Orange has audible character, the Monochrome is doing something quieter and a little more uniform. If you enjoy the Orange ’s pop you’re probably be slightly disappointed with the Monochrome . As for the factory lubing, it is competently done. I peeked into a few switches and the application is consistent enough that I didn’t feel any particular urge to retune them. If you’re someone who lubes everything regardless, maybe be sparing here, as otherwise you’ll smother what little articulation the switch already has. The switches accept films, like everything else in the lineup, and films do their usual job of tightening housing tolerances and compressing the sound profile slightly. Given how restrained the Monochrome already sounds, I’d hesitate to film them unless the build absolutely needs it. You’d mostly be removing what little air is left in the sound. The 80Retros x HMX Monochrome are soft and gently-weighted linears with very few rough edges and they are relatively quiet in volume. Whether that’s the switch you want depends entirely on what you’re trying to build. If you want acoustic complexity, the GAME 1989 Orange is definitely more interesting. If, however, you want a low-effort and low-noise linear that disappears into the build, the Monochrome fit that role pretty well. I wouldn’t call it an exciting switch, but I would, however, call it a sort of grown-up switch. Disclaimer: I’m not a switch scientist. I don’t own a force curve rig, I can’t tell you the exact durometer of the KT2 blend, and my ears are probably not calibrated to the standards of someone like ThereminGoat . This review is based on my personal experience typing on these switches across a few different boards and ultimately actively using them on my primary keyboard . Your mileage may vary based on your plate material, case, keycaps, and other factors. Take everything here as one person’s experience and use it as a starting point for your own.

0 views
ava's blog 1 months ago

computers, privacy and data protection conference 2026

I attended the Computers, Privacy and Data Protection Conference (CPDP) in Brussels for the first time. The conference has lots of different rooms mostly in the same building where multiple panels, workshops and other things are happening at the same time in specific slots, so you gotta choose what you participate in (was difficult at times!). Next to that, you have some fun rooms, some quiet working spaces and spaces to just hang out and talk. Based on the programme, the focus this year was definitely on age verification/youth 'protection', human AI relationships, consumer rights and marginalized groups. Lots of different groups and people present; people from the EU Commission and Parliament, AlgorithmWatch , Bits of Freedom , noyb and Max Schrems, IGLYO , EDRi , Equilabs , Equinox Initiative for Racial Justice , INTITEC , the EDPS and Wojciech Wiewiórowski, Privacy International , the International Committee of the Red Cross , the Office of the United Nations High Commissioner for Human Rights , the European Consumer Organization (BEUC), Future of Privacy Forum , AIRegulation.com , data protection authorities of different countries (CNIL, BFDI, etc.), ALTI , European Disability Forum , d.pia.lab , AI Now Institute , OECD , the IAPP , and all kinds of universities, plus companies like Mozilla, Mastodon, Signal, Wikimedia, Microslop, Uber, TikTok, Google and more. I was there for the opening remarks, then went on to visit: My takeaways/new things learned: Microsoft co-wrote parts of the EU's Energy Efficiency Directive , which allows data centers to keep their energy use confidential under the guise of business secrecy. The draft literally had paragraph's of Microsoft's proposal copied in unchanged. The Dutch government used racial/ethnic profiling via algorithms in the assessment of childcare benefit applications, which led to false allegations of fraud against thousands of families, particularly affecting those from ethnic minorities. I heard about this before, but learned more about it that day. To contest it all and defend democracy, we all need to train our AI literacy skills , support and have good tech journalism that questions and exposes it all (404media is, imo, a good example of what they meant), crafting and changing the social media narrative around AI and Big Tech, listening to affected people, demanding transparency via standards and audits etc. We cannot forget that officials know ; many of the effects we criticize are not accidents or side effects, they are the entire point. Like when tech predominately negatively targets marginalized communities, this is a bonus to people in power, and nothing to be fixed. Workers can resist by reminding their leaders of the liabilities and legal risks, strategic issues, money issues etc. that AI brings; demand specific definition of the needs that AI will fulfill at the workplace, instead of letting AI become the purpose instead of the tool. Age verification is racist and migrantphobic : Many people have issues with their ID, or have none, or are undocumented, and age verification in their country requires them to have contact with officials, police, etc. Age verification is transphobic : Relying on ID means many trans people are forced to reveal their deadname or are forced to come out, as it reveals they are trans if the ID is not or cannot be updated. The platforms are harmful, but we have so many ways and ideas against that that doesn't take away important spaces and support groups or bar entire groups of people. Age verification makes it possible for platforms to avoid working on their problems and becoming better, enables avoiding legislation and regulation, and enables control and surveillance by them; meanwhile, the truth is that you don't suddenly turn 16-18 and know how to handle porn, gore, harassment and all other negative parts of social media. The negative sides to social media that are named as the reason for age verification and banning of social media for specific age groups also affect adults negatively . We need to put more effort into education on how to handle these things. Yes, we can protect children's privacy by banning them off of platforms, but this also affects their other (digital and offline) rights, and privacy rights don't trump all . Children and teens should learn and be encouraged to control their own spaces and moderation via FOSS : Matrix, Mastodon, etc. where they can also seclude from adults and aren't reliant on Big Tech. Age verification and banning would take this away from them and also make it harder for FOSS projects. If children only ever enter the political discourse as victims, the only response can be rescue; that it why we have to make sure they enter as participants. Protection is not (just) space away from the risk, but confronting the systems that cause harm and eliminating them. 16-18% of US citizens report having engaged romantically with a bot, 45% of them said it made them feel more understood, 36% said it gave them stronger emotional support than their human partner. Problem: Current version of AI Act doesn't cover romantic and sexual use, no guidance for safeguards for emotionally responsive AI systems that protects around the risk of suicide, crimes, distress when service slows down or shuts down or model changes, discrimination as you get more if you pay etc.; drafts mention some of it now in Art. 50. With all the talk around becoming emotionally dependent on AI, nudging into harmful behaviors, etc. we cannot forget that you are also vulnerable on other services and in human romantic relationships, where the same routinely happens (weak argument, but to be fair, I also often forget this). We also cannot forget that it is not always a replacement - it often just supplements social life, and there are also surprisingly many people who just don't want or need romantic or sexual relations with a human ; they want bots specifically , and only bots. Disclosure agreements (meaning: labels everywhere that this is just a bot and not real) are most often useless, because people know and intentionally seek it out (exception for Insta/Snap DMs etc.) The latter about Human-AI intimacy was extra interesting because it had someone on the panel who directly works with people who use bots for romance and sex, and her experience has been mostly positive and that it helps her clients. Afterwards, I sadly was too overwhelmed, exhausted and in pain to continue and went back to the apartment to rest. Unfortunately, all the stress around the apartment and the generally more exhausting day triggered my digestive tract badly (Crohn's disease), but within the first few hours, all toilets in the venue were out of service due to an issue outside the venue or the organizer's control, and the alternative toilets were much further away. I didn't wanna have to deal with that with upset intestines. I missed the ' Designing Fairness ' Workshop, and the ' Consumer Rights at the age of acceleration' panel. Didn't meet anyone that day. Look at this ridiculous Gemini Photobooth they had that I saw no one use in the entire 3 days. This day, I managed to attend everything on my list, thankfully, as I felt a bit better. I attended: My takeaways/new things learned: The digital omnibus is mostly there to enable AI made in Europe to aid sovereignty and be competitive with US and China; AI here needs a framework to access data without much regulatory risk - that is what the EU Commission person said. Enforcing the law and and making it sharper is actually leveling the playing field and furthering innovation, because there is a massive power concentration of a handful companies that can do what they want, barely pay fines, have the fines suspended because of the US government bargaining with the EU, or who see them as a cost of doing business. Competition is impacted this way, as small companies are hit harder than the big ones. If the omnibus goes through with changing definitions of personal data etc., it will take years for case law, literature, standards etc. to catch up, it wastes money in companies who need to re-do everything to comply; so it doesn't simplify anything and makes praxis harder. You may set ChatGPT/Claude/Gemini etc. to not send feedback or training data in your settings, but when you react thumbs down/up to their request of whether the output was good or not, or choose between two different versions, the entire chat log until then gets sent for training and potential human review. So, these popup feedbacks override your settings . I need to read more papers by Theodore Christakis. Here is one of them. US and UK discovery and disclosure laws/principles go directly against EU data minimization principles; as long as data is relevant to a case it should be accessible, which is why in their cases, they can just have access to million's of people's data if necessary, and in a divorce case, they have the right to ask for AI chatlogs. There is no AI protection or privilege: If you use AI for legal stuff, you have no expectation of confidentiality like you would with a lawyer, so it is not safe from discovery. There is tension between tracking for harmful behavior/threats vs. data privacy rights ; what if someone threatens to kill themselves, kill others, etc.? Should company look for it, track it, report it, alert anyone, suspend the account, send help resources? Still unclear. There is also tension between people wanting the bonus features/ease of use coming from pesonalization and free services, while also not wanting to be tracked or charged. Advertisers see themselves as enablers of a good thing, as people want fitting ads, good algorithms, good suggestions, and free access; so if their business model is challenged or fails, people will have worse access and worse user experiences in their view. They also fear that if their business model is hindered, things will move into a more extreme, embedded, hard to avoid direction that you don't control or decide (Black Mirror ad type of stuff). I previously wrote about Consenter on the blog, and one panel had people from it there and showing screenshots; changed my mind on it a lot and made me understand the new features and goal better, I will probably write an update on it some time. We have different other options all covering something different about tracking, cookies, consent, or going about things differently, old and new: ADPC, GPC, ConStand, Global Privacy Control, DoNotTrack etc.; important for new stuff is granular consent, sent to the website, user given explanations etc. Uninformed decisions and bad practices lead to unfair competition ; bad actors erode trust level overall, so users resignate, experience fatigue and say yes in the same rates between "good" and "bad" services. Will read soon: Our data after us by the CNIL , and future release: Model rules on succession and access to digital remains by Eigenmann und Harbinja Digital remains can be split into assets (copyright, crypto, business tools, money), personal (messages, photos, identities, AI replicas), and third party data. GDPR only addresses living people; dead people's digital remains are subject to member state laws. There might be a need for something harmonized and European, though. For good digital hygiene , we should remember death and make it as easy as possible or sensible for the people we leave behind to get the access they need to manage our stuff how we want them to. Leave instructions, set emergency/legacy access when available (Google, Facebook, Instagram and Apple have it), include digital assets in your will, decide how your data is allowed to be used after death, especially around AI replicas. Hospice, nurses, families etc. should learn to ask affected parties about these things. Thanks to the focus on agentic AI, there is massive need for inference compute, which is super expensive. Almost all of it is in the control of, or can only be afforded by, the hyperscalers. At the same time, anything that seeks to enable or disable things for AI agents on the web can also affect accessibility programs like screen readers. It is in the best interest of the Big Tech companies to keep things individual, because it distracts from the collective issues and changes they'd have to do; it is easier to blame the person for agreeing to tracking than make sweeping changes to how much can be tracked. Individual consent doesn't consider the fact that data doesn't just affect you, but reveals things about your family, friends, partners, coworkers and more, as data is deeply interconnected. If your friend agrees to share his data and it also includes you, that is your data, still going to the service you'd have disagreed to. We as users have no collective bargaining tools yet; even big worker unions aren't negotiating with Microsoft about the terms of their employer using Microsoft Teams, when they actually should. We should also build up data unions made from users who bargain with the platforms. Strikes could look like boycotting the service, blocking trackers, scrambling data, massive amounts of access requests etc. Look into something called a Worker Data Trust ; this was used to prove Uber's predatory dynamic pricing (Worker's Info Exchange). Lots of workers made access requests, the data was combined and analyzed by researchers. After a failed attempt to meet up during lunch, I managed to meet up with another Country Reporter from noyb for a little while until the next panel happened, and sadly we didn't go to the same one. At this point, I was miffed about lunch at the conference. They made a big deal at registration about how the event will be mostly vegan and vegetarian to offset the climate impact of everyone traveling there, and they asked you to select your preference. I chose vegan. But for the entire three days, the food wasn't clearly labeled, some food was mislabeled as vegan when it wasn't, and there was way too little of it and wasn't restocked. It was more like "vegetarian snacks for birds". Vegan people had no warm food option at all, just sandwiches or wraps all three days that would have been enough for maybe 10 people. I mostly starved and I accidentally ate real cheese one time too because the food situation was so confusing. Here was one of the buffet menu cards, which were a bit to the side removed from the food, partially hidden by other stuff, and incorrect (anything with lactose is not vegan). I have no idea how, on a sea of silver platters with lots of bread, I am supposed to be able to differentiate the vegan gluten free bread option and the vegetarian gluten free bread that has scarmoza (italian cheese). It was a roundtable buffet, so everyone was waiting on you to hurry and grabbing stuff; I can't just grab bread and lift off the top to see the ingredients and then put it back, man. At least group the vegan stuff together or put labels directly in front of each thing. Also, while I am not reliant on gluten-free food, I think the people sensitive to it or having celiac disease don't appreciate that either. I skipped the Cocktail parties and big CPDP party, because it's not really feeling fun when you don't drink alcohol, have trouble just going up to people with your mask and hoping they hear you, and have no one to meet or go with. Last day was rather empty in the programme, so I arrived later and left earlier. I attended: My takeaways/new things learned: The AI warfare one was a bit of a letdown, because they all just accepted war as a right, an inevitable thing that has to happen. There was not even a nuance of fighting war itself, or banning AI weapons, etc; it focused more on the dual nature of the data , in which through surveillance, tracking, etc. not only can military use it to target people, NGO's and others can use it to warn, evacuate, render humanitarian aid etc. and document realities on the battlefield. There was also no possibility for the idea that we could enter an age where drones fight drones automatically and no one needs to get hurt or be traumatized or get to kill people like a game, and that is only because everyone is so attached to the idea that war has to have human casualties. It's hard to legislate and restrict because the data is taken from a whole ecosystem : Telecommunications, cloud services, civilian infrastructure, social media etc. and most of the data is collected during times of peace. Warfare is often explained with national security as a reason, which then again is a legitimate interest or fulfills other opening clauses in data protection and privacy laws. It is a problem that the richest men in the world, close to the US admin, lead the biggest companies worldwide, almost all in the US, and control almost all of AI and AI warfare. Project Maven from 2017 was continuously developed on and is now the Maven Smart System , which was used in Venezuela and Iran recently. Our Art. 15 GDPR right of access as it is right now is making up for Germany and Austria's lack of discovery and disclosure rights respectively. Controllers can usually drag stuff out, cite trade secrets and rights of others to evade data access, but the data subject barely has any power. Not having to justify the access request and it not having to be limited to data protection rights is good in this regard and needs to be kept up. Otherwise, also too much confusion and court cases whether a request was abusive or not if now, any request for a court case instead of privacy rights is deemed possibly abusive. We don't only need to focus on reidentification in general, but about the ability to single people's data out; you might not be able to identify them, but you can build a profile anyway. Learned about the term digital twin , or in terms of user data, a data twin that can be used for similation and is similar enough. AI-act-standards.com exists. Many don't know that the AI Act isn't a GDPR for AI, but serves more as market classification, as it sorts AI into different boxes who have to fulfill different requirements. The details of these requirements are/will be set with CEN/ISO standards and frameworks . You can see the progress of development on these standards on that website, and what they cover and how they interact. Hovering over the elements gives additional info. This is done by the JTC21 , and you can also get involved by registering with your national standardization body (in Germany, this is DIN) or when they do public consultations. Disabled people experience both extremes of AI - better accessibility options, often more reliant on AI, so also more subject to surveillance and having their privacy rights violated, while bad governments can use the data to harm disabled people, all under the guise of research. Marginalized groups are often the first trial group in anything, while not being stakeholders in the tech, or even invited to the table. See: AI used in immigration etc. and with deregulation and AI everywhere, we see a loss of reasonable suspicion thresholds in law enforcement and other groups. Learned about adversarial auditing . The previous two days, I did the whole fancy dress pants and blazer thing (one black blazer, one dark red/purple blazer), but for the last day and the drive home, I wore my Bearblog shirt and wide orange jeans: Someone from noyb staff thankfully recognized me and approached me, so we talked for a bit until he had to leave for another lunch meeting. That concludes the human contact I had. And then I left to drive home with my wife. She will hopefully soon write a guest post on my blog about how she navigates a new city in another country without mobile data/a smartphone (she has a tablet with WiFi only), because while I was at the conference, she explored the city on her own. It's kind of difficult to show up to these conferences as someone who isn't sent there for work, who doesn't have coworkers or ex-coworkers also attending, and who doesn't have much or any industry contacts yet. Most people there know each other from work or previous/other conferences, and I don't. These events are primarily for networking, keeping in touch, and talking about what you have seen and learned though. I couldn't discuss anything with anybody present, and it made me feel really lonely and silly. Just going up to people and striking up a conversation is not my strong suit, and it's something I am working on and has already gotten better, but the mask I am usually wearing in these big crowds and gatherings because I am on immunosuppressive medication is actively keeping me isolated. I know people have trouble understanding me, can't see me smiling at them, and think I am sick, so that keeps both sides hesitant. Unfortunately, if I attend next year, I will have to leave away the mask and maybe try out these protective sprays for nose and throat that are supposed to reduce viral load. It seems like you can only 'afford' to wear a mask if you are already in a group of people. Weeks before the event, I asked some people if they would attend, they said they will and we had a group chat of 10 to coordinate meetups. But during the entire conference, I was the only one trying to make something happen - saying where I am/where I will be, identifiers you could spot me with (as we never met before and you can't see name tags well on the lanyard), meeting points etc. and the two people mentioned were the only ones who took me up on it. The others just ghosted me/ignored my messages. That saddened me a lot during the conference. And unfortunately, these types of events are always really exhausting to me beyond the normal amount everyone experiences, because of things that trigger my conditions, my lower energy, my needs to lie down sometimes, sensory issues, food restrictions etc. so I really have to weigh if it's worth it to me. I'm not sure it is, without the social aspect. Many of the panels I chose had an issue of being not well organized. Instead of short speaker times, precise audience questions, interactions, dialogue, disagreements, different sides, answering the panel's topic and offering solutions etc., it often resulted in every speaker having a 10 minute monologue saying their peace, the other speakers not reacting or intervening because it's too much, everyone more or less saying the same thing or zoning out, and then having too little time to really give much attention to audience questions. Some gathered audience questions to answer them in batches and predictably, that resulted in nuance being lost and almost nothing being precisely answered. From many panels, I walked away with less learned than I wanted to, and just being reaffirmed in what everyone knew already. There were almost no further or new resources, or real takeaways of what the next steps should be and how we can tackle or solve an issue. They say " there should be more transparency " but not how we ask for it, how we legislate it, how it should happen. It's often just a vague " Someone should do more of something, and fast. " It was easy for people from the EU Commission to dodge mine and others' questions about the omnibus bullshit with no convincing answer. (: It disillusioned me a bit about my own goal to be speaking at a panel one day, because so often it felt like it was just there to platform someone to give them a chance to ramble and that's it, or just so that they can put this on their CV. Looking into the panelists, so many of them are genuinely great, very accomplished and admirable people with a lot of expertise, but the way things were set up, it couldn't shine through. You would have been better off talking to them directly. As a final bonus for reading this far, help me delete this (fortune) cookie. Reply via email Published 23 May, 2026 Contesting AI & Defending Democracy ; Possibilities for European AI Futures ( x ) Youth protection through inclusion and empowerment : a rebuttal of the exclusion-based narrative ( x ) Intimacy by Design: Governing Human AI relationships ( x ) Microsoft co-wrote parts of the EU's Energy Efficiency Directive , which allows data centers to keep their energy use confidential under the guise of business secrecy. The draft literally had paragraph's of Microsoft's proposal copied in unchanged. The Dutch government used racial/ethnic profiling via algorithms in the assessment of childcare benefit applications, which led to false allegations of fraud against thousands of families, particularly affecting those from ethnic minorities. I heard about this before, but learned more about it that day. To contest it all and defend democracy, we all need to train our AI literacy skills , support and have good tech journalism that questions and exposes it all (404media is, imo, a good example of what they meant), crafting and changing the social media narrative around AI and Big Tech, listening to affected people, demanding transparency via standards and audits etc. We cannot forget that officials know ; many of the effects we criticize are not accidents or side effects, they are the entire point. Like when tech predominately negatively targets marginalized communities, this is a bonus to people in power, and nothing to be fixed. Workers can resist by reminding their leaders of the liabilities and legal risks, strategic issues, money issues etc. that AI brings; demand specific definition of the needs that AI will fulfill at the workplace, instead of letting AI become the purpose instead of the tool. Age verification is racist and migrantphobic : Many people have issues with their ID, or have none, or are undocumented, and age verification in their country requires them to have contact with officials, police, etc. Age verification is transphobic : Relying on ID means many trans people are forced to reveal their deadname or are forced to come out, as it reveals they are trans if the ID is not or cannot be updated. The platforms are harmful, but we have so many ways and ideas against that that doesn't take away important spaces and support groups or bar entire groups of people. Age verification makes it possible for platforms to avoid working on their problems and becoming better, enables avoiding legislation and regulation, and enables control and surveillance by them; meanwhile, the truth is that you don't suddenly turn 16-18 and know how to handle porn, gore, harassment and all other negative parts of social media. The negative sides to social media that are named as the reason for age verification and banning of social media for specific age groups also affect adults negatively . We need to put more effort into education on how to handle these things. Yes, we can protect children's privacy by banning them off of platforms, but this also affects their other (digital and offline) rights, and privacy rights don't trump all . Children and teens should learn and be encouraged to control their own spaces and moderation via FOSS : Matrix, Mastodon, etc. where they can also seclude from adults and aren't reliant on Big Tech. Age verification and banning would take this away from them and also make it harder for FOSS projects. If children only ever enter the political discourse as victims, the only response can be rescue; that it why we have to make sure they enter as participants. Protection is not (just) space away from the risk, but confronting the systems that cause harm and eliminating them. 16-18% of US citizens report having engaged romantically with a bot, 45% of them said it made them feel more understood, 36% said it gave them stronger emotional support than their human partner. Problem: Current version of AI Act doesn't cover romantic and sexual use, no guidance for safeguards for emotionally responsive AI systems that protects around the risk of suicide, crimes, distress when service slows down or shuts down or model changes, discrimination as you get more if you pay etc.; drafts mention some of it now in Art. 50. With all the talk around becoming emotionally dependent on AI, nudging into harmful behaviors, etc. we cannot forget that you are also vulnerable on other services and in human romantic relationships, where the same routinely happens (weak argument, but to be fair, I also often forget this). We also cannot forget that it is not always a replacement - it often just supplements social life, and there are also surprisingly many people who just don't want or need romantic or sexual relations with a human ; they want bots specifically , and only bots. Disclosure agreements (meaning: labels everywhere that this is just a bot and not real) are most often useless, because people know and intentionally seek it out (exception for Insta/Snap DMs etc.) Simplification for Whom? Unpacking the Consumer Impact of the Digital Omnibus ( x ) My Chatbot, My Confidant: Protecting User Privacy in Generative AI Conversations ( x ) Informed consent: The breakthrough in Art. 88b GDPR / Digital Omnibus and current initiatives in the field of PIMS and technical standardisation ( x ) Digital Legacy Beyond GDPR: Succession, Data Protection, Access Rights, and Platform Power ( x ) The Agentic Assistant: What does Big Tech’s goal of creating a universal digital intermediary mean for society? ( x ) Designing Collective Technology Governance ( x ) The digital omnibus is mostly there to enable AI made in Europe to aid sovereignty and be competitive with US and China; AI here needs a framework to access data without much regulatory risk - that is what the EU Commission person said. Enforcing the law and and making it sharper is actually leveling the playing field and furthering innovation, because there is a massive power concentration of a handful companies that can do what they want, barely pay fines, have the fines suspended because of the US government bargaining with the EU, or who see them as a cost of doing business. Competition is impacted this way, as small companies are hit harder than the big ones. If the omnibus goes through with changing definitions of personal data etc., it will take years for case law, literature, standards etc. to catch up, it wastes money in companies who need to re-do everything to comply; so it doesn't simplify anything and makes praxis harder. You may set ChatGPT/Claude/Gemini etc. to not send feedback or training data in your settings, but when you react thumbs down/up to their request of whether the output was good or not, or choose between two different versions, the entire chat log until then gets sent for training and potential human review. So, these popup feedbacks override your settings . I need to read more papers by Theodore Christakis. Here is one of them. US and UK discovery and disclosure laws/principles go directly against EU data minimization principles; as long as data is relevant to a case it should be accessible, which is why in their cases, they can just have access to million's of people's data if necessary, and in a divorce case, they have the right to ask for AI chatlogs. There is no AI protection or privilege: If you use AI for legal stuff, you have no expectation of confidentiality like you would with a lawyer, so it is not safe from discovery. There is tension between tracking for harmful behavior/threats vs. data privacy rights ; what if someone threatens to kill themselves, kill others, etc.? Should company look for it, track it, report it, alert anyone, suspend the account, send help resources? Still unclear. There is also tension between people wanting the bonus features/ease of use coming from pesonalization and free services, while also not wanting to be tracked or charged. Advertisers see themselves as enablers of a good thing, as people want fitting ads, good algorithms, good suggestions, and free access; so if their business model is challenged or fails, people will have worse access and worse user experiences in their view. They also fear that if their business model is hindered, things will move into a more extreme, embedded, hard to avoid direction that you don't control or decide (Black Mirror ad type of stuff). I previously wrote about Consenter on the blog, and one panel had people from it there and showing screenshots; changed my mind on it a lot and made me understand the new features and goal better, I will probably write an update on it some time. We have different other options all covering something different about tracking, cookies, consent, or going about things differently, old and new: ADPC, GPC, ConStand, Global Privacy Control, DoNotTrack etc.; important for new stuff is granular consent, sent to the website, user given explanations etc. Uninformed decisions and bad practices lead to unfair competition ; bad actors erode trust level overall, so users resignate, experience fatigue and say yes in the same rates between "good" and "bad" services. Will read soon: Our data after us by the CNIL , and future release: Model rules on succession and access to digital remains by Eigenmann und Harbinja Digital remains can be split into assets (copyright, crypto, business tools, money), personal (messages, photos, identities, AI replicas), and third party data. GDPR only addresses living people; dead people's digital remains are subject to member state laws. There might be a need for something harmonized and European, though. For good digital hygiene , we should remember death and make it as easy as possible or sensible for the people we leave behind to get the access they need to manage our stuff how we want them to. Leave instructions, set emergency/legacy access when available (Google, Facebook, Instagram and Apple have it), include digital assets in your will, decide how your data is allowed to be used after death, especially around AI replicas. Hospice, nurses, families etc. should learn to ask affected parties about these things. Thanks to the focus on agentic AI, there is massive need for inference compute, which is super expensive. Almost all of it is in the control of, or can only be afforded by, the hyperscalers. At the same time, anything that seeks to enable or disable things for AI agents on the web can also affect accessibility programs like screen readers. It is in the best interest of the Big Tech companies to keep things individual, because it distracts from the collective issues and changes they'd have to do; it is easier to blame the person for agreeing to tracking than make sweeping changes to how much can be tracked. Individual consent doesn't consider the fact that data doesn't just affect you, but reveals things about your family, friends, partners, coworkers and more, as data is deeply interconnected. If your friend agrees to share his data and it also includes you, that is your data, still going to the service you'd have disagreed to. We as users have no collective bargaining tools yet; even big worker unions aren't negotiating with Microsoft about the terms of their employer using Microsoft Teams, when they actually should. We should also build up data unions made from users who bargain with the platforms. Strikes could look like boycotting the service, blocking trackers, scrambling data, massive amounts of access requests etc. Look into something called a Worker Data Trust ; this was used to prove Uber's predatory dynamic pricing (Worker's Info Exchange). Lots of workers made access requests, the data was combined and analyzed by researchers. Data-driven warfare : AI, civilian risks, and corporate responsibility ( x ) Digital Omnibus meets the Charter of Fundamental Rights ( x ) Toward a Standard for Fair AI-driven Recruitment ( x ) Data protection law as a shield, not a weapon: empowering historically marginalized communities in the EU in times of de-regulation ( x ) -> this choice was especially rough, because I was also very interested in ' The U.S. Deregulatory Effect ' happening elsewhere at the same time The AI warfare one was a bit of a letdown, because they all just accepted war as a right, an inevitable thing that has to happen. There was not even a nuance of fighting war itself, or banning AI weapons, etc; it focused more on the dual nature of the data , in which through surveillance, tracking, etc. not only can military use it to target people, NGO's and others can use it to warn, evacuate, render humanitarian aid etc. and document realities on the battlefield. There was also no possibility for the idea that we could enter an age where drones fight drones automatically and no one needs to get hurt or be traumatized or get to kill people like a game, and that is only because everyone is so attached to the idea that war has to have human casualties. It's hard to legislate and restrict because the data is taken from a whole ecosystem : Telecommunications, cloud services, civilian infrastructure, social media etc. and most of the data is collected during times of peace. Warfare is often explained with national security as a reason, which then again is a legitimate interest or fulfills other opening clauses in data protection and privacy laws. It is a problem that the richest men in the world, close to the US admin, lead the biggest companies worldwide, almost all in the US, and control almost all of AI and AI warfare. Project Maven from 2017 was continuously developed on and is now the Maven Smart System , which was used in Venezuela and Iran recently. Our Art. 15 GDPR right of access as it is right now is making up for Germany and Austria's lack of discovery and disclosure rights respectively. Controllers can usually drag stuff out, cite trade secrets and rights of others to evade data access, but the data subject barely has any power. Not having to justify the access request and it not having to be limited to data protection rights is good in this regard and needs to be kept up. Otherwise, also too much confusion and court cases whether a request was abusive or not if now, any request for a court case instead of privacy rights is deemed possibly abusive. We don't only need to focus on reidentification in general, but about the ability to single people's data out; you might not be able to identify them, but you can build a profile anyway. Learned about the term digital twin , or in terms of user data, a data twin that can be used for similation and is similar enough. AI-act-standards.com exists. Many don't know that the AI Act isn't a GDPR for AI, but serves more as market classification, as it sorts AI into different boxes who have to fulfill different requirements. The details of these requirements are/will be set with CEN/ISO standards and frameworks . You can see the progress of development on these standards on that website, and what they cover and how they interact. Hovering over the elements gives additional info. This is done by the JTC21 , and you can also get involved by registering with your national standardization body (in Germany, this is DIN) or when they do public consultations. Disabled people experience both extremes of AI - better accessibility options, often more reliant on AI, so also more subject to surveillance and having their privacy rights violated, while bad governments can use the data to harm disabled people, all under the guise of research. Marginalized groups are often the first trial group in anything, while not being stakeholders in the tech, or even invited to the table. See: AI used in immigration etc. and with deregulation and AI everywhere, we see a loss of reasonable suspicion thresholds in law enforcement and other groups. Learned about adversarial auditing .

0 views
Tara's Website 1 months ago

Spring 2026 updates

Spring 2026 updates Servus from … a random hotel. I’m sitting cross-legged on the bed in my pyjamas, laptop on my legs. Mx Liebe is sitting beside me, a reminder that he is part of my portable home. I glance outside the window. The sun is starting to set, that small daily proof that the days are getting longer. Outside there is a large walled lorry park, all its lights turned on: a safe place for drivers to spend the night before entering their final destination.

0 views
Jack Vanlightly 1 months ago

Introducing Dimster, a performance benchmarking tool for Apache Kafka

Dimster = DIMensional teSTER for Apache Kafka On GitHub: https://github.com/dimster-hq/dimster Most of my career in distributed systems has been as a tester, performance engineer and formal verification specialist. I’ve written performance benchmarking tools in the past, for RabbitMQ and Apache Pulsar but in recent years I’ve used OpenMessagingBenchmark (OMB) to run benchmarks against Apache Kafka and other messaging systems. But OMB is hard to deploy and has several limitations compared to more sophisticated benchmarking systems I’ve developed in the past. With Claude becoming so much better since Christmas I decided to write a Kafka-centric performance benchmarking tool, with a lot of inspiration from OMB. I took the bits I like about OMB and the things I like about the tooling I’ve built in the past, to make a performance testing tool for testing Apache Kafka. In this post I’ll introduce some aspects of Dimster that are core to its design: Dimensional testing Shareable, self-contained results with reproducibility in mind Benchmark prep and post-processing Kubernetes as a standardized runtime A benchmarking and stress testing technique I’ve used for years is something I have called “Dimensional Testing”. We can think of all the configs and workload aspects as forming N-dimensional space. Within that space we can explore the impact of points in that space along a single dimension, or even co-varying dimensions. Take a config or an aspect of a workload as a dimension, and run a series of identical benchmarks where a set of points along that dimension are explored (while everything else remains the same). The dimension could be a client config, such as batch.size or acks. It could be an aspect of the workload such as number of consumers, type of consumer, number of consumer groups, the partition count, the produce rate and so on. There are hundreds of dimensions to explore, which requires some patience and care lest you become overwhelmed. The below depicts just three dimensions, and a set of three scenarios which test performance along one or two dimensions at a time. Fig 1. Three examples of varying or co-varying an aspect of a workload as dimensions Each of the above 16 test points (across 3 scenarios) is a separate benchmark, with a fresh topic, warm-up time, recorded time, and cooldown time etc. The generated charts for throughput and various latencies are repeated for each of the three scenarios, with each test point within a scenario plotted as a series/bar on those charts. This makes it easy to compare the performance results of varying the values of a single dimension (or co-varying values across multiple dimensions). Fig 2. Each scenario maps to a set of charts, with the test points as data series. With share groups being relatively new, I could compare the performance of regular consumers against share group consumers, with identical benchmarks where the dimension explored is consumer type (CONSUMER_GROUP|SHARE_GROUP). The following test has as the base workload of ten topics with each topic having 6 partitions, 6 consumers and 4 producers. Each scenario changes the producer rate, and compares consumer groups to share groups. Record keys are used, so batch sizes will be small, which is a tougher workload than a no-key test which typically results in larger batches. The charts below show the results for an EKS deployment with Kafka deployed on 3x m6i.2xlarge with 300 MB/s provisioned gp3. At 50 MB/s we see that p99 end-to-end latency is stable, with roughly 15 ms overhead for share groups. At 200 MB/s, p99 end-to-end exhibits peaks in a periodic fashion. Dimster uses environments. The sizing of a test is determined by which environment is used. I ran some share group consumer scaling tests, with full mTLS, on Kafka clusters assigned 2, 4, and 8 CPUs. These are the equivalent of vCPUs, as my Threadripper has SMT (hyperthreading) enabled. 2-CPU environment on my Threadripper: I ran the following workload with the above environment, with the CPU requests/limit of 2, 4 and 8. Then I used the dimster compare command to generate comparison charts based on the JSON result files of each run. Each chart compares each test point side-by-side. 10k msg/s - 1000 consumers (6th test point in 1st scenario) We see that 2 CPUs fare a lot worse than 4 and 8 CPUs. 100k msg/s, 250 consumers (4th test point, 3rd scenario) The 2 CPU cluster simply can’t keep up with 100k msg/s and 250 consumers. If we unselect 2-CPU, we see that 4-CPU and 8-CPU was ok. Dimster charts are interactive. Series can be toggled, time and percentile ranges can be selected. One thing I really like about OMB is that it produces a JSON file for the results. These files are easy to store and easy to share. But there was also a lot missing for full traceability and reproducibility. Dimster includes the following in every test campaign result (a set of files in a result directory): Results :  The JSON result file which contains all the test point performance results. For each test point, it includes the effective workload and client configuration. It also includes the hardware and other metadata to know what the benchmark was run against. A CSV file generated from the result JSON file (to make it easy to put in a spreadsheet or run custom visualizations). Source configs : The source workload file itself, as well as any additional files such as any dedicated client config file, the broker config file, the version of Kafka, the version of the Kafka clients, and the CPU/memory/disk given to the brokers and clients. Log files : the log files of dimster-core, the benchmarking framework, and each Kafka broker. Charts : Throughput and latency charts (clickable, zoomable) generated from the result JSON file. Dashboards : Grafana dashboards converted to interactive HTML files. I can run a test campaign then send you the results and you’ll be able to reproduce the results because you know exactly what was run and on what. The results are also completely self-contained, if you want to see the dashboard to look at Kafka metrics during the test, it’s right there as an HTML file in the results. No need for access to Grafana and Prometheus and no need to keep monitoring infrastructure around, it can be ephemeral. Dimster comes with four test modes (which all support dimensional testing): Run : Fixed throughput benchmarks, plus: Live-interaction . Run-mode also supports live interaction with the user. The user can change the producer rate, number of producers and consumers, message size, etc.  Availability : Optionally measure availability (producer/consumer/aggregate) during the standard run-mode benchmark. Explore : Discover the highest sustainable throughput while staying under a target end-to-end latency and percentile. Drain-backlog : Build a backlog and time how long it takes for the consumers to drain it. Optionally set a producer rate during the drain phase, such as when testing if a cluster is big enough to drain a backlog while under normal producer load. Correctness : Detects data loss, data corruption, out-of-order delivery and duplicates.  Example 1: Peak sustainable throughput, 1 partition, share group consumers Explore mode on my Threadripper. The idea was to see the bottleneck of a single partition, as consumers are scaled out. The rule was for p75 e2e latency to stay below 50ms. Example 2: Consumer group vs share group with 1 ms processing time The prior example was an unrealistic synthetic test where the consumer spent no time processing. This explore test added 1 ms consumer processing time per message with 300 consumers. It compared a 300 member consumer group with 300 partitions, vs a 300 member share group, with 5, 10, 25 and 50 partitions. Share groups managed the same throughput (95% of theoretical max based on 1 ms processing time and consumer count), on only 10 partitions. Consumers groups needed 300 partitions. Personally, explore and run are my bread and butter benchmark modes. For a given workload I usually start by finding the throughput limit where Kafka transitions from normal stable performance into degraded territory. I either use run mode and use live interaction to discover the performance limit, or I use explore which is slower but I can leave to run and it discovers the limit in an automated way. For latency benchmarks, once I know the limit, I can craft benchmarks that fit inside the performance envelope for that workload on the specific version of Kafka on the specific hardware I am using. The Dimster CLI has some commands that help before running benchmarks and for post-processing. Dimster resources command The resources command calculates the network and disk throughput required to service a workload. This is important in the cloud for selecting the right instances, ensuring that baseline network and disk throughput are greater than the workload’s demands. Dimster compare command Compare different runs that were executed on different hardware, different broker configurations, different broker versions etc. Dimster pivot command You can slice and dice the data any way you want based on the CSV data. However, you can also pivot the results and generate a chart with the pivot command. This compares the Nth test point across all scenarios. Dimster is easiest to use with Kubernetes. Dimster has a CLI you use from your laptop which speaks Kubernetes and leverages it to run benchmarks on any hardware, any cloud, any laptop or workstation using the exact same orchestration logic. All it needs is a properly configured k8s cluster. It could be minikube or k3d on a laptop or workstation, or AWS EKS or Google Cloud GKE or your own in-house cluster. You can tell Dimster to deploy Apache Kafka to a stateful set in the k8s cluster: Fig 3. Dimster architecture in full deploy mode Or point Dimster (deployed to k8s) at a Kafka service or in-house Kafka cluster. When testing a Kafka service, you can provision a single powerful instance for the Dimster coordinator and worker, and deploy them to a local k8s distro such as Minikube, K3d or Kind. A single worker will happily consume all the cores and memory you give it. Fig 4. Dimster architecture in external deploy mode Or run a super-slim full setup in a tiny minikube/kind/etc local k8s distro: Fig 5. Dimster deployed in a tiny local k8s cluster The workflow is the same. If you can provide a k8s cluster, then Dimster does the rest. Deployment is really simple, monitoring, gathering results, troubleshooting is all simplified via a mix of the CLI being relatively capable, and k8s providing a well-understood platform. K8s is not obligatory , you can run dimster-core directly as a Java program, and point it at a Kafka cluster already provisioned. But you lose many features such as monitoring, live-interaction, automatic gathering of logs, automatic chart and CSV generation and so on. However, you can use the post-processing command dimster chart to generate the charts of a result JSON file. Run the Java directly via the benchmark script: ./bin/benchmark -w path/to/workload file I will be publishing a blog post regularly about Dimster and what you can do with it. So stay tuned. I invite you to go and play around with Dimster , even if it's just running benchmarks on your laptop or workstation. You can get an idea of what charts get produced, what kinds of benchmarks you can run, trying out dimensional testing etc. The docs are pretty decent and should cover most of it. It’s fully featured but still a 0.X version. Myself and a Confluent colleague are the only ones who have run it thus far, so there may be bugs you encounter, if you do encounter a problem, please open an issue with repro steps. If you want to run serious benchmarks, you’ll likely need an EKS or GKE type of Kubernetes cluster. Dimster comes with a special CLI for EKS to deploy EKS with node groups for Kafka, Dimster workers/coordinator, Grafana/Prometheus, as well as storage classes for gp3.  While evaluating consumer group vs share group consumers, I’ve been running benchmarks in k3d on my beefy Threadripper 9980X workstation with 64 cores (128 threads), 256 GB RAM and an Samsung 9100 PRO 8TB SSD, which is plenty to run an entire medium sized Kafka cluster plus workers on it. I’ll be sharing some share group benchmarks tomorrow. Happy testing! Dimensional testing Shareable, self-contained results with reproducibility in mind Benchmark prep and post-processing Kubernetes as a standardized runtime Results :  The JSON result file which contains all the test point performance results. For each test point, it includes the effective workload and client configuration. It also includes the hardware and other metadata to know what the benchmark was run against. A CSV file generated from the result JSON file (to make it easy to put in a spreadsheet or run custom visualizations). Source configs : The source workload file itself, as well as any additional files such as any dedicated client config file, the broker config file, the version of Kafka, the version of the Kafka clients, and the CPU/memory/disk given to the brokers and clients. Log files : the log files of dimster-core, the benchmarking framework, and each Kafka broker. Charts : Throughput and latency charts (clickable, zoomable) generated from the result JSON file. Dashboards : Grafana dashboards converted to interactive HTML files. Run : Fixed throughput benchmarks, plus: Live-interaction . Run-mode also supports live interaction with the user. The user can change the producer rate, number of producers and consumers, message size, etc.  Availability : Optionally measure availability (producer/consumer/aggregate) during the standard run-mode benchmark. Explore : Discover the highest sustainable throughput while staying under a target end-to-end latency and percentile. Drain-backlog : Build a backlog and time how long it takes for the consumers to drain it. Optionally set a producer rate during the drain phase, such as when testing if a cluster is big enough to drain a backlog while under normal producer load. Correctness : Detects data loss, data corruption, out-of-order delivery and duplicates.

0 views
Max Bernstein 1 months ago

Travel notes: RubyKaigi Hakodate

I just got back from a three and a half week trip to Japan. It was the longest trip I have ever been on (aside from studying abroad in Germany, which felt different). I made the following wild circuit with only a backpack and a duffel: This trip was split into three parts: time with my immediate family, going to a conference, and then time with my partner. They were all great and also I am glad to be home. I’ll post my abbreviated travel notes here, including activity and food recommendations. We started in Tokyo but we were only there for about 40 hours. We focused our time mostly on arts and crafts: we did a kintsugi workshop, spent time at an artists cooperative, and then did a lot of walking around. This was a good intro to the trip, because everyone kept waking up at 4am and crashing at 7pm due to the jet lag. 4am wakeup makes for nice morning walks to 7-Eleven. I brought my family to T’s Tantan in Tokyo Station because I’m vegetarian and it’s otherwise hard to find ramen that approaches kosher in Japan. It continues to be great and I really appreciate having a steady vegetarian option available. Many years ago when I visited Tokyo there was a place that served a delicious tomato-based vegetarian ramen, but I hear it has since permanently closed. Bummer. We took the shinkansen to Kanazawa. I love the train. It’s fast. It’s quiet. You can eat your snacks on board and gaze out the window as the world whizzes by. It’s nice. We toured a soy sauce factory (meh; they don’t let you in the room where the magic happens) and the old town (pretty!) before finally eventually ending up at our small hotel in Toyama: Satoyama Auberge Maki No Oto. I highly recommend this hotel. It is beautiful, the staff is lovely, the food was excellent, and they were very accomodating of me being vegetarian. We continued on to Toyama, which is a port town. We got to talking with an older local guy who told us all about his favorite local spots. We learned after leaving that this guy has extraordinarily fancy taste and they were all either Michelin starred or at least Michelin rated and with a lead time of months. We opted to instead go to a local brewery, which had a ghost pepper beer (!) and pizza. We then moved on via train to Osaka, where we transferred to a car to head (eventually) to our hotel in the hills near Nara. We toured the Daimon sake brewery. They explained every little thing about the process, which was especially interesting to me, as I’ve done some small amount of homebrewing and I bake. They sounded similar. We had a tasting and even got to talk to Daimon-san. I recommend going. I also recommend the Akame 48 waterfalls walk/hike, which has some exquisite falls, and Murou Art Forest. They had some really wonderful installations. My brother and I parted ways from the rest of my family in Osaka: they headed further west and we headed north to Itō on the Izu peninsula. We got a surprise perfectly clear view of Fuji along the way. It’s beautiful there. They don’t seem to welcome foreigners in a lot of their restaurants (we were turned away several times) but one place had a guy who enthusiastically welcomed us in. We ended that evening enjoying a some food and a beer while also being stared at by a 300lb completely tattooed guy. It was a little unsettling but we left without incident. My brother and I made our way to Tokyo for the day before his flight and before my train north to Hakodate for RubyKaigi. I once again did that thing where I walked around in humid 80F heat with a large backpack and pants and was extraordinarily warm toward the end of the day. After about a liter of Aquarius on the train north I felt better. I stayed at Yunokawa Prince Hotel Nagisatei which I would like to especially call out for having an enormous, diverse, and very vegetarian friendly breakfast. Every morning I got to try new and tasty things and even feel full after. It was great. Hakodate is beautiful in the spring. I arrived at peak cherry blossom season and Goryokaku, their star shaped fort, is absolutely decked out in cherry blossoms. It is also moderately swarmed by tourists (in this case, three cruise ships). It didn’t feel over-crowded though. I enjoyed eating at The Bear King which had a vegetarian friendly option. The next day was the committer meeting. I don’t remember a ton from it other than people talking at length about the semantics of deep freezing an object (do you freeze its class? its class’s superclass? …?). I picked up my badge and also got to check out my colleague Chris Salzberg’s bar SOLENOID ! It’s a neat spot. I headed out to go find some dinner. This is about when I got a message on my phone that there was going to be an earthquake, so I walked back into the bar and said “hey, did you get this?” just before everything started shaking. It was the biggest earthquake I’ve experienced, but I was metaphorically not too shaken up. Then we got the tsunami warning. Chris’s bar is already something like 8 meters above sea level and at the foot of Mt Hakodate. With the city sirens going off and the police directing traffic with batons, though, I decided my best bet was just to march directly up the mountain to get more elevation. Since the tsunami wasn’t scheduled to arrive for about 20 or 30 minutes and my hotel was across the sea-level part of town, I parked myself on a little concrete post. Chris found me eventually. Someone told us that there was a middle school offering refuge, so we went and hung out on the side of the gymnasium. They were really nice about it. On Wednesday, the conference started. It was really well signed and organized. My usual complaint with conferences is that there’s nothing to eat for vegetarians (or that we get mashed with the gluten-free people and each group only gets a salad and bad bread) but that did not happen! They had really stellar vegetarian bento. They had a lot of leftovers toward the end of lunch so I even went and got a second. This was about when I started freaking out because my speaking slot was approaching and I wasn’t yet feeling my talk. Normally when I give a talk, I get up in front of people and I pace and gesticulate and productively complain and throw in some fun anecdotes and the audience, one way or another, ends up learning about JITs at scale, or Scheme semantics, or something. It’s what I’d done for my little lunch talk at Brown two weeks prior. I even titled that talk One must imagine compiler engineers happy so there was plenty of room for educational complaining. But this RubyKaigi talk was in front of an enormous crowd and toward a more general audience than I was used to addressing. The slides did not feel like they were flowing until about twenty minutes before my talk. In the end it went alright. I realized about 40 seconds in that I had way too much content so I ended up speaking rapidly for 30 minutes straight, completely unaware of the audience (which you can’t see anyway because of the lights). I only really noticed people when I made a dumb six-seven joke and Aaron laughed. The rest of the conference I was able to relax and enjoy other people’s talks. I got some good hallway track in, too. I think there’s a good group of people who are interested in Ruby tracing (for example, Perfetto in ZJIT ) so maybe we will make something happen. We had a nice small dinner at Yasai Bar Miruya , which was vegan (!) and had some nice sake. The host was very friendly, too. I nerd-sniped John and J into implementing a VM for the Universal Machine . This was a daunting homework assignment back in undergrad but it was a fun project later in life. S joined toward the end of the conference. She’s also vegetarian so we got some really excellent vegetarian ramen at MAIDO Ramen . Finally, S and I headed south on the shinkansen for Nikko. Nikko is small, beautiful, and a tourist day-trip town. Dinner closes early. Shops close earlier. Since we were staying there we had to make sure to track down and visit the one or two vegetarian places before they shuttered. S and I, along with J and J, took the bus up from Nikko, up the windiest switchbacks, to the Kegon Falls. We were going to take a boat across the lake, but the water level was too low for the dock on the other side, so we ended up half hiking and half taking a bus. Then we continued our hike through the Senjōgahara Marshland (beautiful), to the Yudaki Cascades (lovely), which also had a surprise restaurant and ice cream shop at the base! It’s called Yutaki Rest House . After some great (vegetarian friendly!! wow!!) udon, we marched up the waterfall and around Yuno Lake at the top to Yumoto Onsen. In order to make the last reasonable bus back to town, we just enjoyed putting our feet in the foot bath. One day was rainy. In the evening, J and I thought it would be fun to continue our Universal Machine implementations. As Norman Ramsey would say, “my implementation is 90 lines long and runs sandmark in under six seconds.” We also enjoyed doing a tour of the shrines right above Nikko. The shrines are resplendent against the backdrop of forest. Pro bus tip: you can either pay by IC card or credit card. No need to grab a ticket if you do that. S and I shipped our bags (thanks, Yamato) before continuing on to the small town of Moka, the staging area for our big pottery festival day. Unfortunately, there was no good way to get there: there was no reasonable series of trains and no taxi would take us. Ultimately we ended up taking the train to Utsonomiya and catching the long local bus to Moka. About twenty minutes into this ride, in the middle of nowhere, bus nearly empty, the bus driver pulled over and ran over to us looking kind of panicked. He asked where we were going and was visibly relieved when we said Moka. I suppose we are not the usual riders. Very nice of him. Upon arrival, S introduced me to CoCo ICHIBANYA, which is also super vegetarian friendly. I loved it. We ate really well before walking to our tiny hotel. We did not really know what to expect from the Mashiko pottery festival. The internet said it would be crowded and to arrive early, so we got up at 6:30am for estimated 7am departure on the tiny train from Moka to Mashiko. On most trains you can pay with an IC card but we were out in the sticks so we asked the only other guy on the platform how to pay for the train. He said he had no idea and that this was his first time here. When the train showed up completely packed to the gills and we had to (politely) push onto it, we started to realize that this was The Event and it was going to be mayhem. Also, fun fact: the way the Moka train payment works is that you grab a little ticket from the train, and, upon arrival, wait in line to present your ticket to two very overwhelmed looking people at a table, who charge you, and you pay in cash. Onto Mashiko: the festival was packed . There’s pottery everywhere the eye can see. There are tents and there are full buildings. It varies in quality and artistry from fine to jaw-droppingly spectacular. You could completely stock your kitchen from this fair alone and it would even be cost-effective. The main bummer for us is that we had to get pottery safely back home. We limited ourselves to a reasonable assortment but we really wanted to buy a beautiful painted 20 inch plate with a bird on a branch. After a ton of walking around, we took another long long bus back to Utsonomiya and continued onto Karuizawa. We didn’t know what to expect from Karuizawa but, having been, I could probably concisely describe it as “Aspen for people from Tokyo”. It was… fine. We loved our hotel, Tsuruya Ryokan. The manager was very excited when we borrowed a Studio Ghibli DVD from their collection. We continued on to Tokyo, our final stop. We our usual tour of stationery stores and bakeries—the bread was something to write home about (har har). We enjoyed a (vegetarian!! friendly!!) kaiseki meal at Hyoki Shabu-shabu Ginza before enjoying some live music at Rocky Top . We also recommend Jikasei MENSHO for vegetarian ramen. Bakery checklist: We had an uneventful and reasonably easy trip home. Whew. Long post for a long trip. See you next year in Miyazaki! BOUL’ANGE NIHONBASHI (check! good croissants) Bricolage bread & co (check! good everything) Brasserie Viron Marunouchi Beaver Bread Bricolage bread & co. Bartizan Bread Factory Gontran Cherrier Tokyo Aoyama Shop Comme’N Tokyo Shiomi Bakery The Little BAKERY <!– https://www.jocjapantravel.com/kanto-tokyo-bakeries/ –>

0 views
annie's blog 1 months ago

It’s either a poem or a piece of cheese // Week 20 — 2026

Are these weeknotes? Yes they are! Will I do them again next week? Who knows! Sunday 10 May: Got home from hospital shift around 7:30pm. Exhausted, hangry. Walked into a clean tidy home, flowers and cards, and the kids cooking dinner (spring roll bowls which were so so so good). Plus! a NEW CHAIR for the balcony. We ate and talked and did that thing where you laugh so hard you cry. Then I sat on my new balcony chair & had some nice bourbon while they cleaned everything up. Anyway it was a great Mother's Night 💗 More spaces in my life for uncensored unfettered thinking. Less platform, more workshop. Less stage, more garage. Less producing, more tinkering. Tuesday 12 May: Took a sick day. Felt off, sore throat, achy yesterday. Woke up with the full experience. This was to be an uncomfortably busy day and instead I am canceling all the things I can. Left with a couple of items to do from the comfort of the couch. Hot tea. Window open. Cats sitting in the sun. Breeze and blue sky outside. If I feel enough energy I’ll take a slow walk later. Dreamed about being evicted. Felt very real. Woke up panicked. Relieved to realize it was a dream and I have a two-year lease. Wednesday 13 May: Took my chemistry final. Not as difficult as anticipated! A relief, since I didn’t study as much as planned. “I want you to see all kinds,” he would say to her. “I want you to realize that this whole thing is just a grand adventure. A fine show. The trick is to play in it and look at it at the same time.” “What whole thing?” “Living. All mixed up. The more kinds of people you see, and the more things you do, and the more things that happen to you, the richer you are. Even if they’re not pleasant things. That’s living. Remember, no matter what happens, good or bad, it’s just so much” — he used the gambler’s term, unconsciously — “just so much velvet.” —from So Big by Edna Ferber Denial and suffering may be good methods for undoing the old / destructing but they are not good methods for creating / constructing what you actually wish to build. Thursday 14 May: Still sick. Tried to do a bit of work. Mostly just rested. Feeling somewhat better but end of day. Friday 15 May: Mara’s college graduation day. Those two years have flown by. Many feelings! So proud of her. Saturday 16 May: Lily’s birthday! A weekend full of celebrations. Took her and a group of friends to one of those combo bowling / laser tag / arcade / overstimulation places. They did all the things & had fun. I got some studying done. But is it doable? Sunday 17 May: Hiking church. Warm today, 70℉ when we started. Chubb Trail from West Tyson. It is a painful confession but the art of poetry carries its own power without having to break them down into critical listings. I do not mean that poetry should be raffish and irresponsible clown tossing off words into the void. But the very feeling of a good poem carries its own reason for being.  …primarily Art is its own excuse, and it’s either Art or it’s something else. It’s either a poem or a piece of cheese. —from On Writing , Charles Bukowski 💪 One gym session (Monday) before the sickness took me out Tues-Thurs, then it was A Weekend of Events. Back to our regularly scheduled program next week, I hope. 👟 A few short walks, and a nice hike. 📺 Unfamiliar (loved it) and season 1 of The Thaw (liked it, will watch the rest). Lots of tv time with sick days. 📚 So Big by Edna Ferber (finished) and On Writing by Charles Bukowski. 🔗 The old world of tech is dying and the new cannot be born // Baldur Bjarnason No matter the flavour of Christianity, a core idea baked into every aspect of the religion is that singular revelatory events can fundamentally change the world. There’s the “before”. Then the “event”. Then an “after” that has been completely transformed. In Christianity itself this is usually associated with Christ’s chaotic transit schedule –  “He is here! He has left! He is about to arrive again! Now he’s leaving again! But he’s also somehow always been here! And not.”  – but the mode of thinking is common throughout literature, philosophy, and storytelling in the Christian west. 🔗 Letting things build // Tracy Durnell The way I often read non-fiction — snatches of twenty pages here, twenty pages there, putting a book down for two months (or two years) at a time — is  not conducive to *finishing* books, but I do find it conducive to thinking . Rich texts can take a while to sink in, so I’ll jump to another book while I let the first one marinate. 🔗 You are here // Sebastian As I approach my topics and ideas through writing—whether in the form of brief notes or by looking back when I pick up the journal and flip through its pages—a process of contextualization takes place. And that is important. For me, this is a form of metacognition: observing myself as I think and being able to analyze and categorize my thoughts “from the outside.” It doesn’t completely solve the black box problem of self-perception, nor does it eliminate the blind spot of the mind that seeks to explain itself from within itself, but it does make things a lot easier and more accessible.

0 views
A Working Library 2 months ago

Freedom from unreal loyalties

In the work against war, Woolf notes that women—unlike many of their brothers—have four great but perhaps misunderstood teachers: And those teachers, biography indicates, obliquely, and indirectly, but emphatically and indisputably none the less, were poverty, chastity, derision, and—but what word covers “lack of rights and privileges?” Shall we press that old word “freedom” once more into service? “Freedom from unreal loyalties,” then, was the fourth of their teachers; that freedom from loyalty to old schools, old colleges, old churches, old ceremonies, old countries which all these women enjoyed, and which, to a great extent, we still enjoy by the law and custom of England. We have no time to coin new words, greatly though the language is in need of them. Let “freedom from unreal loyalties” then stand as the fourth great teacher of the daughters of educated men. Woolf, Three Guineas , page 267 These are strange teachers. We may be forgiven for not seeing them as such when they’ve visited us. Woolf continues: By poverty is meant enough to live upon: That is, you must earn enough to be independent of any other human being and to buy that modicum of health, leisure, knowledge and so on that is needed for the full development of body and mind. But no more. Not a penny more. By chastity is meant that when you have made enough to live on by your profession you must refuse to sell your brain for the sake of money. That is you must cease to practice your profession, or practice it for the sake of research and experiment; or, if you are an artist, for the sake of the art; or give the knowledge acquired professionally to those who need it for nothing. By derision—a bad word, but once again, the English language is much in need of new words—is meant that you must refuse all methods of advertising merit, and hold that ridicule, obscurity, and censure are preferable, for psychological reasons, to fame and praise. Directly badges, orders, or degrees are offered, fling them back in the giver’s face. By freedom from unreal loyalties is meant that you must rid yourself of pride and nationality in the first place; also, of religious pride, college pride, school pride, family pride, sex pride, and those unreal loyalties that spring from them. Directly the seducers come with their seductions to bribe you into captivity, tear up the parchments; refuse to fill up the forms. Woolf, Three Guineas , page 270 Woolf is echoing what we already know of wealth, fame, and loyalty—namely, that they encourage possessiveness and defensiveness, that they drive us to the violent defense of prestige and power, and that on that road lies war . We see this possessiveness and defensiveness in the whingeing insecurity of the leaders declaiming DEI; in the boss who insists his workers flatter his every decision, however foolish and arbitrary; in the patriarch who demands obedience from his wife and children; in the man who beats his partner when she tries to leave. (The most dangerous time for a woman in an abusive relationship is always when she is trying to leave.) Woolf, again: “the public and the private worlds are inseparably connected…the tyrannies and servilities of the one are the tyrannies and servilities of the other.” 1 If we are to prevent war in our public worlds, then we must also root it out in the private. And we must root it out among ourselves. For we are no more immune to the appeal of tyranny than anyone else: And the facts which we have just extracted from biography seem to prove that the professions have a certain undeniable effect upon the professors. They make the people who practice them possessive, jealous of any infringement on their rights, and highly combative if anyone dares dispute them. Are we not right then in thinking that if we enter the same professions we shall acquire the same qualities? And do not such qualities lead to war? Woolf, Three Guineas , page 249 In naming these teachers, Woolf transforms a proscription into a refusal. The lack of wealth becomes the refusal of it; the lack of fame, of prestige, of authority becomes the rejection of all those ugly and pernicious forces. (The one benefit of living in an era in which we are bombarded with the lives of the super wealthy is we cannot even for one moment forget that they are deranged.) By claiming that lack as a refusal, we release ourselves from longing for that which we can never have; we end a ravenous hunger that could never be sated. For had we great rank and great wealth and all the rest, we would be as eager for war as the warmongers, as miserable and unhappy as the billionaires. Without, we can see war for the horror it is; we can use our time and attention to imagine other worlds, and other roads to get there. I think these teachers go by other names—frugality, integrity, humility, and solidarity, to name a few. Like the best teachers, they ask a lot of us. Perhaps too much on some days; we may not always be able to hear them, especially through the din of the war drums and the noise of the platforms and the very real fear of precarity that screams ever so loudly in our ears. But I think perhaps that if we make an effort to listen, we will find that they still have much to teach us, that we still have much to learn. Woolf, Three Guineas , page 364  ↩︎ View this post on the web , subscribe to the newsletter , or reply via email . Woolf, Three Guineas , page 364  ↩︎

0 views
iDiallo 2 months ago

Software Engineers are Obsolete

In my first interview for a developer position, I shared a link to my personal project with the interviewer. It was a website for learning how to program. I created it from the ground up. I built the PHP app, designed the database schema, made a nice design to tie it all together. I wrote down my process, and it became the first tutorial on the site. Then I collected tutorials from all over the web and displayed them on my website, which acted as a portal. There was a section for PHP tutorials, for Ruby on Rails, for .NET, etc. Each one individually curated by me. My interviewer was so impressed. I got the job. Later, I added a section where anyone could submit their own tutorials. It was fascinating how quickly people found my website and started submitting links. The tutorials were coming in so fast that I removed the verification system and let people upload links directly. But then my mind wandered. What if I start a blog? Yes, I had another blog before this one. I built an entire blog engine from scratch. A colleague found my blog. He was so excited that he shared his own with me. At lunch, we would discuss ideas, and that same evening after work, we would buy a domain name and start a new project. We shared tips and tricks on how to rank on Google. We had a skill, being web developers, and we took full advantage. When we had an idea, we would fire up our computers that same night and build it. Friends and family would come to us for validation. We were the ultimate deciders of what was a good idea and what was a bad one. We were the gatekeepers. We knew how to program, and nobody outside our circle could say otherwise. Now, friends and family don't come to us anymore. They go straight to ChatGPT, and it tells them their idea is brilliant . They launch their favorite AI agent, which builds their entire product from a single prompt. Some of them even manage to host it on the web, accessible to the world, and they are seeing their first customers. People who used to confuse Java with JavaScript now tell me they have a platform. People who don't even know what programming is are standing at the forefront of software innovation, advocating, evangelizing, and making money. This skill I spent years honing has been made obsolete by everyday people. We, the developers, are no longer the gatekeepers. In fact, now we need to keep up or risk being left behind. Some commenters online tell me I'm just jealous, that I need to embrace progress. I don't want to be obsolete. I'm on openclaw, moltenclaw. I have accounts on all the video generation websites. I have accounts on ChatGPT, Claude, Gemini, and Mistral. Just as I'm getting a hang of one tool, my friend who works in a warehouse tells me, "just use Perplexity for that." But Perplexity isn't enough, because another friend says GenSpark is better. For some reason I can't sign into my Manus account anymore. And apparently, to get the most out of it, I need to get Meta Ray-Bans. Everyone is empowered, no one needs me, and that's that. The developer is now obsolete. But then, I opened LinkedIn. My peers, fellow developers who for some reason all have the word "AI" in their job title, are saying the opposite. "Developers are not losing their jobs to AI," they say. "Developers are losing their jobs to other developers who use AI." They are vibe-coping to the max. The history of technology has always been a story of nearly missing out. I remember another job I applied for and totally didn't get. The company had moved all their client-facing apps to Silverlight. If you're wondering what Silverlight is, you might understand why I chuckled when the interviewer described their plight: they were struggling to find developers to help them migrate to HTML and JavaScript. I'm fairly sure that chuckle is why they never called me back. It's one thing to embrace new technology. It's another thing entirely to put all your eggs in one basket. Companies are betting everything on Silverlight. Sorry, I mean AI. Without thinking through what happens if things don't pan out. AI has lowered the barrier to entry. That's a good thing. More people can now bring a fresh pair of eyes to the software engineering field. But there's a problem. Those new entrants won't become better engineers over time. Why? Because they are not writing code, not reading code, not debugging code. Their growth path, with time and experience, is to become better prompters. What this means is that, amid all the noise, my role as a software engineer may seem obsolete. But in the long run, we will be back to square one, where engineers writing code with their own meatware will hold all the cards. These are the people who learned the hard way: by reading documentation, by debugging broken apps, by having their seemingly perfect Stack Overflow question closed as a duplicate. These are the engineers who will hold the keys to software. Not because they're guarding secrets, there are no secrets. It's simply that the new developer is not, and will never be, interested in learning. While we pride ourselves on producing more software than ever, it doesn't take long to realize that software is never truly finished at delivery. It has to be maintained. It's strange, computers whose entire purpose is to repeat the same process over and over, perfectly, somehow manage to degrade over time. My tutorial website, seemingly working fine, returned an error when I visited it after months of neglect. I restarted all the services and brought it back up. It was now full of spam and NSFW URLs. An application that worked perfectly yesterday is broken today. It could be a memory leak, unexpected input, or just users with fat fingers. Your completed application is suddenly incomplete, and you have to fix it. In an ideal world, we wouldn't keep producing more software. We would have working software, and less of it to maintain. AI thrives on quantity. If you need me, I'll be in the back, patiently waiting for you to realize you can't prompt your way out of a Silverlight migration. My rates just doubled.

0 views