matduggan.com 1 weeks ago

Midnight Train to Stockholm

I was recently summoned to a meeting in Stockholm, a city I had somehow managed to avoid despite living in Copenhagen for years. My Swedish experience, up to this point, consisted entirely of trips to Malmö — the closest Swedish city to Denmark and, more importantly, home to a Costco. As an American living abroad, I am duty-bound to report there every six months so the proper authorities know I'm still alive and to procure my ceremonial barrel of peanut butter pretzels. It's less a shopping trip than a consular check-in. Stockholm, it turns out, is much further from Copenhagen than anyone lets on. My options were to fly or take the train. Flying is technically a one-hour affair, but to make a 9 AM meeting I'd have to wake up at 4 AM, shuffle through security in a fugue state, and land in Sweden looking like a hostage video. I wasn't sure when I'd nap. This seemed insane. Then I had what I believed, at the time, to be a genius idea: the overnight train from Malmö to Stockholm. I'd sleep en route, wake refreshed, stride into my meeting like a man who understood something about life that others did not. Plus, I love trains. Sweden has a high-speed line that does the run in four hours, but the night train takes its time, which sounded charming. It was not charming. The first surprise came at booking. There are three tiers of experience: a seat, a couchette (whatever the fuck that is), or a private sleeping compartment. Since I wasn't paying, I chose the private compartment. This would prove to be the single smartest decision of my adult life, possibly of anyone's adult life. If you are reading this from some point in the future and you are over the age of thirty, book the private compartment. I don't care what it costs. Sell a kidney. I'll explain. My train left Malmö at 10:30 PM. The station there is depressing in a way that's hard to articulate in that nothing is obviously wrong. There's a grocery store. There's a convenience store. And yet everyone inside looks stranded , as though they've been waiting for something that isn't coming. Small children roam in feral packs. There is a pervasive sense that this is the last train out of somewhere very bad, and that whatever is chasing everyone is still, perhaps, on its way. Amtrak in the US has this feeling, a vibe that you are running from the law. I remember when my family used to take the train from Ohio to New Jersey, waiting for it at a train station that was basically a concrete bunker in the middle of a corn field. The concrete box would either be freezing cold due to too much AC or boiling hot due to too much heat. I would stay up late on the train and watch as parents would abandon sleeping children to jump off at stops and catch a smoke. They all had a nervous desperation that these Swedish travelers shared. It's the kind of place where you wouldn't be surprised to see someone take a SIM card out of a phone and throw it in the trash. I stopped at the grocery store and loaded my backpack — my only luggage — with provisions: two large water bottles, wet wipes, a change of clothes, a bag of nuts, and, as an emergency measure, two Red Bulls, in case sleep failed me and I had to power through the following day on hatred and that cursed, faintly urine-themed energy drink. Then I stopped by the men's room, which featured a decorative fish tank whose sole occupant had a full-frontal view of the urinals. If reincarnation is real, Henry Kissinger is in that tank. European train station bathrooms are often weird, but this was up there. First there was no automated system to get in, it was just a guy with a credit card reader. Also they were piping in tropical sounds to the bathroom which I assume is to cover the unspeakable horrors happening in the stalls. I felt uncomfortable that I kept looking at the fish and found it to be always staring at me. The tank was in really good condition, with incredibly clean water. I couldn't help but think maybe its better for the fish to die faster than live their entire lives staring at an endless line of men peeing. I found my train and boarded, and I knew immediately I was in trouble. This was an old-school train — varnished wood, worn blue upholstery, late-70s energy throughout. The regular seats were hard, upright benches with little fold-down wooden tray tables. They did not recline. Not a little. Not at all. Which raises the question: why call it a night train? Night train implies, to me, that at some point during the night, someone might sleep . But the seats also had bright lights above them that never turned off, which transformed them from sleeper seats into something closer to interrogation chairs. The couchettes turned out to be stacked bunks — men-only, women-only, or mixed and the passengers were packed together so tightly that the gender segregation began to make a grim, practical sense. I'm not squeamish around strangers, but we're talking well within reach-out-and-stroke-someone's-hair range. My private room looked roughly like a jail cell: a cot, a light that turned off, a door that locked. In other words, everything I have ever wanted from a hotel. That's not sarcasm, I'm easy to please. Naturally, I was far too curious about the rest of the train to actually sleep, so I set out to wander. The three conductors on duty were all wearing bodycams strapped to their chests, which is always an encouraging sign in that it suggests both that they had been attacked and that they had, at some point, done some attacking of their own. They were also wearing shorts, which felt deeply wrong. There's something unsettling about a train conductor without pants. It's a formal job. You don't want your pilot in flip-flops and you don't want your conductor showing knee. I don't know what it is about shorts on men specifically that come across as clownish, but there is a ranking of jobs where one shouldn't wear shorts all the way to one cannot wear shorts. Doctors, pilots, lawyers, accountants are all pants jobs. Train conductor felt like a no-brainer that it would be a pants job, also frankly I think they should also have to wear a cool hat and have a pocket watch. In the same way I would bristle at a judge sentencing me to death in a Hawaiian shirt, a train conductor in shorts checking my ticket feels wrong. I made my way a few cars down to see how the general population was faring. The door slid open and I was hit, physically, by the smell of cheap vodka. Before me stretched a sea of Swedes, each with the specific facial expression of a person who has just realized they have made a terrible mistake and cannot un-make it. Two people were openly weeping. Three others were borderline-homeless-looking punk kids dressed exactly the way punks dressed in 1994 — I don't know who is still manufacturing M65 field jackets and military jump boots, but they're clearly still moving units in southern Sweden. One of the punks was vomiting into a plastic bag, seated next to a very sweet-looking, deeply concerned young woman who had presumably boarded this train with hopes and plans. She had the look of a woman who had her life together. The conductors arrived, spoke to him, were told to fuck off, and then quietly relocated the young woman to a new seat the way you'd move a house plant away from a leaking radiator. The punks then began joking loudly among themselves, two of them taking turns retching up what smelled like vodka cut with unleaded gasoline. God help anyone trying to sleep back here. Between the puking, the reek, and the punks openly hitting on every woman within shouting distance, it was like being trapped on a Greyhound bus that had sworn a blood oath never to stop. I watched a man roughly my own age attempt to sleep by laying his face directly on the wooden tray table, earplugs jammed in, arms limp at his sides, in the international posture of I have given up . I left when a boyfriend and girlfriend began fighting because she had proven surprisingly receptive to the advances of a punk kid whose body odor was strong enough to reach me three rows back. The boyfriend — sitting directly next to her — took issue with this, which seemed reasonable. I moved on to the meal car. The meal car was the hangout, the refugee camp, the place where people who had discovered they couldn't sleep in the bunks came to sit and stare into the middle distance. Two young women were seated at a table nearby, one of them work-shopping, in English, why she deserved better than her current boyfriend in Malmö. "I don't think I should settle for average." Her friend was being supportive and kept trying to inject something about her own life, only to be steamrolled every time. "Yeah, I know exact—" "It's just, I work so hard at school and he doesn't." "My last boyfr—" "Maybe when we get to Stockholm we go buy some nice dresses and go dancing." "That sounds fu—" "Because I really do think I deserve to feel beautiful ." After a few rounds of this, I got bored. It was all early-twenties drama, and I don't say that with contempt because it's a phase we all pass through. In your twenties, you discuss your plans and feelings as though they matter, because to you, they do. In your late thirties, you come to understand that nobody actually cares if you live or die, and you learn, mercifully, to keep it all to yourself. It's one of the small gifts of aging, along with knowing how to fold a fitted sheet and no longer pretending to enjoy helping people move. Friends in your 20s are your therapists, your closest confidants and your relationship counselors. In your 30s, you have an actual therapist and don't have to burden the people around you. At some point in everyone's life someone they love and respect will put up their hands and say "alright ENOUGH" and you'll realize how tiring you are. These women hadn't gotten there yet, but I did think the friend should start charging for this therapy session. I retired to my cabin and fell asleep almost instantly. The rocking of the train, the smug satisfaction of not being propped upright in a wooden pew next to a vomiting stranger, and a modest dose of melatonin combined into something close to bliss. We arrived on time. I grabbed my backpack and stepped out into Stockholm Central Station at 6 AM, which was nearly deserted. I found a coffee shop, ordered a coffee, and was charged an amount that made me wonder "should I open a coffee shop in Stockholm?". As he handed it over, the barista said, cheerfully, "It's my first day." "You shouldn't tell people that," I replied. I then felt terrible about it for approximately one hour. He, for his part, seemed entirely unfazed, or possibly hadn't heard me at all, which somehow made it worse. I carried the guilt with me through security, out onto the street, and into the cab I hailed for the twenty-minute ride to my meeting. The meter began climbing almost immediately, and with a kind of enthusiasm I hadn't previously known meters possessed. He asked where I was from. I said the U.S. He said he loved Americans. He said Americans were the best people which, frankly, nobody says unless they're being tortured by us. He asked if this was my first time in Stockholm. He asked what I did for work. Every question was warm and generous and I understood, dimly, that I was being courted. But it was only twenty minutes, how bad could it be? Eleven hundred kronor later, I stepped out onto the curb while he tried to press his business card into my hand so I could call him directly for future rides. Having just paid roughly $120 to travel the distance of a decent jog, I now understood his enthusiasm. He'd hit the jackpot, and the jackpot was me. The meeting wrapped in a few hours. I took an Uber back for a quarter of the cab fare — a small, petty vindication I savored the entire ride — and spent the next four hours walking around Stockholm. It's genuinely lovely, and denser than I expected. Copenhagen feels like a city that was designed by someone who liked people; Stockholm feels like a city that was designed by someone who respected them but wasn't sure he wanted them over for dinner. More cars, less green, wider streets, harder edges. Every Swede I passed looked as though they were on their way to something slightly more important than what I was doing. It was beautiful in the way certain people are beautiful — the kind of beauty that doesn't especially need you to notice. Should you take the overnight Swedish train? Honestly? Probably not this one, unless you are terrified of flying or being sober. The only tolerable option for an adult human is the private cabin, and at that price you can usually just fly. But it was, undeniably, an experience and one I would happily repeat, provided someone else were footing the bill.

Culture

▲ 0

0 views

matduggan.com 2 weeks ago

Clickhouse is winning the Observability Wars

For roughly the last ten years, a meaningful percentage of my working hours have been spent thinking about observability. If you're not familiar with the term, "observability" is what we call it now that "monitoring" doesn't sound expensive enough. The actual work is unglamorous in that you collect a lot of logs, some metrics, a few traces, and then you give them to people. I generally like my job. I like that we're always trying new ideas and approaches. I like the fact that when things go wrong, the answer is almost always sitting there in the data, waiting to be found by whoever is patient enough to look. But I want to be honest with you: in ten years of doing this work, across a half-dozen companies and every observability platform you've heard of and a few you probably haven't, logs have never stopped being the worst part of the job. They were the worst part when I started. They are the worst part today. I fully expect them to be the worst part of this job forever until the robots rise up and rip my head off in one clean sweep. I've written about why logs are terrible before , so I'll spare you the full lecture and give you the short version. Every developer's expectations for logs are set by a single formative experience: the syslog box. Or a container running locally. Or tail -f on a production server they probably shouldn't have SSH'd into. The point is that at some early, tender moment in their career, they had an experience with logs that was flawless. They ran and something useful came back. They piped it into jq and got exactly what they needed. This experience is the observability equivalent of a first kiss. It ruins them for everything that comes after. Because here is the thing about that flawless experience: it works because the system is small, the volume is trivial, and the person querying is the same person who wrote the log line. There is no schema drift, no cardinality explosion, no cross-team consumer with dashboard expectations, no VP asking why the "revenue events" graph has a gap in it. Then there are forty services. Now there are four hundred. Now the logs are being consumed not just by developers but by customer service, who need to look up a specific user's failed checkout from Tuesday. And by the data team, who are quietly building a business-critical dashboard on top of a log line that a backend engineer is about to refactor without telling anyone. And by the on-call, who at 3 AM does not want to learn a new query language, does not want to think about index patterns, and would like the search bar to just work. So you have a technical problem — the volume is enormous, the shape is inconsistent, the queries are unpredictable — sitting on top of an expectations problem, which is worse. Developers want logs instantly, they want to run arbitrary operations on them, and they will not commit to a schema. Meanwhile the less-technical consumers of that same data want the dashboards to be stable forever, the UI to be forgiving, and the whole thing to feel like a normal product. These two audiences are, in most practical respects, at war with each other, and you are the diplomat. ClickHouse came out of Yandex, where it was built to chew through analytical queries against absurd volumes of clickstream data. It was not designed for observability. It just happens to be shockingly good at it, because clickstream data and observability data have a lot in common: high volume, append-heavy, time-ordered, mostly read in aggregate, and every so often you need to reach in and find one specific needle. You can run it yourself with Helm charts. You can point Grafana at it via the ClickHouse plugin, or use their own web UI, or bring your own frontend. Their docs are actually good, which I mention because it's rare enough to be worth flagging. I've never used their ClickStack setup though, so YMMV. For observability specifically, the OpenTelemetry Collector has a ClickHouse exporter, which means you can pipe OTLP data straight in and let it manage the initial schema for you. ClickHouse is designed to scan billions of rows and ingest an amount of data that, when you first see the numbers, makes you assume they're lying. They're not lying. You query it with SQL, which is a language that already exists and was not created by a startup two weeks ago. I'm ranting about logs and then I'm explaining why I like to administer Clickhouse more. Let me take a second and explain why Clickhouse is really good at logs at scale. Logs, as a data shape, have some peculiar properties. They're append-only. You never update a log line, and you almost never delete a single one, though you delete a lot of them at once when retention kicks in. They arrive roughly in time order, though never actually in order. They're read in bursts where nobody looks at logs for days, and then during an incident somebody wants to scan a billion of them in seconds. They're highly compressible, because most of the bytes in your logs are repeated: the same service names, the same hostnames, the same error strings, the same JSON keys, over and over and over again. And critically, when you query them, you almost always want either a narrow time range across all fields or an aggregation across a wide time range with a few filters. You very rarely want "give me one specific row by ID" the way you would from a transactional database. (There are exceptions when its something like GDPR or compliance logging which is its own subgenre of nightmares). In a row-oriented database — Elasticsearch, Postgres, MySQL — the data for a single log line is stored together on disk. If your log has 40 fields and your query only cares about 3 of them, tough luck, you're reading all 40 from disk anyway. The database will filter it in memory, but the disk I/O has already happened. ClickHouse stores each column separately. If your query says SELECT service, status_code, count() FROM logs WHERE timestamp > now() - INTERVAL 1 HOUR GROUP BY service, status_code, ClickHouse reads exactly three columns off disk: timestamp, service, and status_code. The other 37 columns in your schema might as well not exist. On observability data, where you often have dozens of attributes but any given query touches three or four, this is the difference between scanning 800GB and scanning 40GB. This is also why the compression numbers look absurd. Columnar data compresses far better than row-oriented data because the values within a single column are, by nature, similar to each other. A column of service_name values might have a hundred distinct strings across a billion rows. ZSTD eats that for breakfast. You'll routinely see 10–14x compression ratios on real observability data, compared to 2–3x for Elasticsearch. The amazing thing is that ClickHouse scales without changing shape. I don't know how else to say this. Every other observability backend I've worked with mutates as it grows. The architecture at 1 TB a day and the architecture at 10 TB a day are recognizably different systems, with different failure modes, different ops burdens, and different mental models. ClickHouse at 10 TB a day looks like ClickHouse at 1 TB a day with more shards. That's it. That's the pitch. That's the whole reason I'm writing this. Let me show you what I mean. At 1 TB a day, every modern observability stack is roughly okay. If you're at this scale, you can pick almost anything and be productive. The differences below are real but they're not yet painful. Here is the honest truth: at 1 TB a day, ClickHouse is not less complicated than its peers. It's roughly the same. Maybe slightly more, if you count the schema design work you have to do up front. You get 10–14x compression with ZSTD and proper codecs, the Altinity Operator handles keeper coordination and the whole thing runs in about seven pods. But you do have to design your schemas. ORDER BY keys matter enormously. There is no native PromQL, so metrics workflows go through the Grafana plugin or through chproxy and an adapter. Roughly $1.5–2.5K/month. If you took the diagrams at this tier and squinted, you'd say they're all in the same weight class. And you'd be right. Now watch what happens next. This is where the exponential curve kicks in for everybody except one of these. You'll notice, if you look at the diagram, that I basically just added shards. That's it. That's the change. Same operator, same query engine, same query language, same mental model. Rebalancing after adding shards is manual, which is a real trade-off — most teams pre-provision or use weighting on Distributed tables to sidestep it. Materialized views for dashboard rollups shift from "nice to have" to "essential." Roughly $7–11K/month. The gap between ClickHouse and everything else opens up here. It doesn't close. This is where most solutions genuinely stop working, in the sense that even a well-staffed internal team cannot keep up with the operational load. If you've read this far, the point is probably already obvious, but I want to say it directly. Every observability stack works at 1 TB a day. If you're small, pick whatever your team already knows. Life is short. We're all just waiting for the robots to kick our heads off like soccer balls. The question is not which stack works today. The question is which stack still resembles itself two years from now, when your data volume has 5x'd and your team has 2x'd and the person who originally designed the whole thing has left the company. Elasticsearch mutates. LGTM mutates. Datadog stays operationally simple but mutates financially into something that requires its own dedicated team of accountants and pipeline engineers just to keep the bill from spiraling. ClickHouse just gets wider. You add shards. That's the whole trick. There is a real cost to this: you have to eat the schema-design and query-engine complexity up front, at a scale where the other options are objectively easier. You will be, briefly, the one making things harder for your developers. They will not always appreciate this. But the trade you're making is that their experience — and yours — remains roughly the same as the data grows by an order of magnitude, and the next order of magnitude, and probably the one after that. I have spent ten years watching observability stacks change shape underneath me while I tried to keep them running. ClickHouse is the first one that hasn't and that has been able to actually scale with me . That's pretty incredible. A relatively vanilla Elasticsearch cluster with Logstash providing some buffer between ingest and the Lucene indexes. Users get full-text search, which is genuinely good — this is the thing Elasticsearch is actually best at, and at this scale it delivers. Mapping explosions are already a background risk with mixed data, so dynamic mapping needs to be disabled or carefully templated from day one. ILM policies (hot → warm → delete) are non-optional even at this size, because forgetting to set them is how you get paged on a Saturday about disk pressure. Roughly $6–9K/month. Nothing too crazy. Alloy (formerly Grafana Agent, RIP) unifies the collection story into a single daemon, which is nice. Loki works well as long as you spend some time educating developers on how to attach useful labels — a conversation you will have many times, with many people, for the rest of your career. Mimir and Tempo largely do what it says on the tin. Roughly $3.5–5K/month. At 1 TB a day, Datadog is genuinely great. This is the scale it was built for, and it shows. You install the agent, you look at dashboards, you go home. There is almost nothing to think about, which is the entire point. You can already see the shape of the cost problem lurking in the diagram — the metered pipelines, the indexed-vs-ingested logs distinction, the custom metrics cardinality tax — but at this scale it's manageable. Roughly $45–75K/month, though negotiated pricing varies enough that I'd take that number with a grain of salt the size of a fist. Datadog's whole pricing philosophy is that they save you a full-time engineer. I think that framing is somewhat deranged, but they are extremely rich and I am not, so consider your source. Kafka is no longer optional. At 5 TB a day, direct writes into Elasticsearch cause bulk-reject storms and backpressure that will absolutely take your cluster down during a traffic spike. So now you're running Kafka, which means you're either running Kafka well or you're about to have a second, entirely different set of problems. Shard math becomes critical — at 50GB target shards, you're minting ~200 shards a day counting replicas, and your cluster state size becomes its own concern. You almost certainly need Elastic's commercial license for searchable snapshots and the frozen tier. Roughly $40–55K/month before licensing. That but Kafka You are now in microservices mode, whether you wanted to be or not. That means 65+ pods across three separate systems, each with its own compaction pipeline, its own hash ring, its own memcached tier. The gossip/memberlist ring becomes a real operational concern; ingester rollouts require careful -ingester.autoforget-unhealthy tuning, and if you get it wrong you either lose data or duplicate it. Roughly $22–32K/month. The operational complexity is still low, in that you don't run any servers. But you now need a full pipeline team whose entire job is reducing your Datadog bill. Exclusion filters, sampling rules, cardinality caps, tag allow-lists, the whole apparatus. This is what I call the "you build a system to avoid using the system you're paying for" trap, and once you're in it, you are in it forever. Roughly $180–350K/month, depending on how aggressive the pipeline team gets. This is also where you are basically fighting with your SaaS provider all the time, pouring over their billing documentation to figure out how to reduce costs. It's a hostile relationship and one I don't enjoy. You are now running three separate Elasticsearch clusters — one for logs, one for metrics, one for APM — federated through Cross-Cluster Search. Hot-tier NVMe cost dominates the bill. This is the scale at which teams start seriously evaluating alternatives, and where a lot of the recent migrations to ClickHouse have originated. Roughly $95–140K/month plus commercial licensing. You need people who are legitimate experts on Elasticsearch. Now thankfully Elastic just laid a ton of those people off, so they're probably possible to get, but still. Running this thing at this size is very complicated . Around 180+ pods, zone-aware everything, split-and-merge compaction, per-tenant limits, shuffle sharding to prevent noisy neighbors. You almost certainly have a dedicated observability platform team of three to five engineers at this point. If you don't, get ready for a bad fucking time. Roughly $55–85K/month. Still very easy to run, in the strict sense that you don't run anything. But your bill is now measured in six or seven figures a month, and the org has almost certainly built a pre-processing pipeline team whose entire existence is dedicated to reducing that bill. Most companies at this scale have gone hybrid: Datadog for APM and high-value metrics, self-hosted (increasingly ClickHouse) for logs. The complexity paradox at this scale is that you now have Datadog's simplicity plus your pipeline complexity plus a second self-hosted stack. Pricing is all over the goddamn place. You might be over a $1 million a month here. Look at the diagram and then look back at the 1 TB diagram. It's the same diagram. There are more shards. That's the difference. Materialized views for rollups are now mandatory rather than optional. Schema design mistakes you made two years ago will start to hurt, so hopefully you didn't make many. Rebalancing after adding shards is still manual; most teams pre-provision or use clickhouse-copier or a dual-write migration when they need to grow the cluster. Kafka starts to become useful as a buffer for very bursty ingest, though it's not required. Roughly $18–28K/month.

Database Sql JSON DevOps

▲ 0

0 views

matduggan.com 2 months ago

The Intolerable Hypocrisy of Cyberlibertarianism

I like the Internet. I am old enough to remember the pre-Internet era and despite the younger generations pining for those simpler days, I was there. Paper maps were absolutely horrible, just you and a compass in your car on the side of the road in the middle of the night trying to figure out where you are and where you are going. Once when driving from Michigan to Florida I got so lost in the middle of the night in Kentucky that I had to pull over to sleep and wait for the sun so I could figure out where I was. I awoke to an old man staring unblinkingly into my car, shirtless, breathing heavy enough to fog the windows. To say I floored that 1991 Honda Civic is an understatement. You would leave your house and then just disappear. This is presented as kind of romantic now, as if we were just free spirits on the wind and could stop and really watch a sunset. In practice it was mostly an annoying game of attempting to guess where people were. You'd call their job, they had left. You'd call their house, they weren't home yet. Presumably they were in transit but you actually had no idea. As a child my response to people asking me where my parents were was often a shrug as I resumed attempting to eat my weight in shoplifted candy or make homemade napalm with gasoline and styrofoam. Sometimes I shudder as a parent remembering how young I was putting pennies on train tracks and hiding dangerously close so that we could get the cool squished penny afterwards. Cassettes are the worst way to listen to music ever invented. Tapes squealed. Tapes slowed down for no reason, like they were depressed. Multiple times in my life I would set off on a long road trip, pop in a tape, and within fifteen minutes watch as it shot from the deck unspooled like the guts from the tauntaun in Star Wars . You'd then spend forty-five minutes at a Sunoco trying to wind it back in with a Bic pen knowing in your heart you were performing CPR on a corpse. Then you'd put it back in the player out of pure stubbornness, and it would chew itself again immediately, and you'd drive the next six hours in silence with your own thoughts, which were not as good as Pearl Jam. So I am, mostly, grateful for the bounty the internet has provided. But there is something wrong, deeply wrong, with what we built. The wrongness was there at the start. It was baked into the foundation by people who told themselves a story about freedom, and that story was a lie, and we are all, every one of us, paying their tab. To understand what happened we need to go back to the 90s. One of the first and most classic examples of the ideology that powered and continues to power tech is the classic "A Declaration of the Independence of Cyberspace" by John Perry Barlow written in 1996. You can find the full text here . I remember thinking it was genius when I first read it. I was young enough that I also thought "Snow Crash" was a serious political document. Today the Declaration reads like one of those sovereign citizen TikToks where someone in traffic court is claiming diplomatic immunity under maritime law. It helps to know who Barlow was. Barlow was a Grateful Dead lyricist. He was also a Wyoming cattle rancher. He was also, briefly, the campaign manager for Dick Cheney's first run for Congress. (You did not misread that.) He spent his later years as a fixture at Davos, the World Economic Forum, where the very wealthy gather each January to remind each other that they are interesting. It was at Davos, in February 1996, fueled by champagne and grievance over the Telecommunications Act, that Barlow banged out the Declaration on a laptop and emailed it to a few hundred friends. From there it became, somehow, one of the founding documents of the modern internet. Many of the pillars of "modern Internet" are here. Identity isn't a fixed concept based on government ID but is a more fluid concept. We don't need centralized control or really any form of control because those things are unnecessary. It was this and the famous earlier "Cyberspace and the American Dream: A Magna Carta for the Knowledge Age" that laid a familiar foundation for a lot of the culture we now have. [ link ] The Magna Carta is also our introduction to the (now familiar) creed of "catch up or get left behind". The adoption of new technology must be done at the absolute fastest speed possible with no regulations or checks. You don't need to worry about the consequences of technology because these problems correct themselves. If you told me the following was written two weeks ago by OpenAI I would have believed you. The cumbersome copyright/patent process. Cumbersome to whom, exactly? This is always the move. The thing your industry would prefer not to deal with is reframed as an obsolete burden. Your refusal to do it is rebranded as innovation. Your inability to imagine a world where you don't get exactly what you want becomes a manifesto. So there are dozens of these pieces and they all read the same. If you don't regulate these technologies humanity will only benefit. Education, healthcare, industry, etc. We don't need regulations because the transformation from the medium of paper to digital has transformed the human spirit. But one was extremely surprising to me. Langdon Winner wrote something almost prophetic back in 1997. You can read it here . He coins the term cyberlibertarianism (or at least is the first mention of it I could find) and then goes on to describe an almost eerily accurate set of events. In all he lays out 4 pillars of this ideology. Technological determinism. The new technology is going to transform everything, it cannot be stopped, and your only job is to keep up. Stewart Brand's actual quote, which Winner pulls out and lets sit there like a body on display, is "Technology is rapidly accelerating and you have to keep up." There's no room to ask whether we want any of this. The wave is coming. Surf or drown. It does not occur to anyone in this discourse that 'drown' is a choice the wave is making, not a natural law. Waves do not have intentions. Destroying your livelihood and leaving you to rot isn't a requirement of the natural order as much as that would convenient. Radical individualism. The point of all this technology is personal liberation. Anything that gets in the way of the individual maximizing themselves be it government, regulation, social obligation, your annoying neighbors, is an obstacle to be removed. Winner notes, with what I imagine was a very dry expression, that the writers of the "Magna Carta for the Knowledge Age" cited Ayn Rand approvingly. In 1994. As intellectual grounding. For a document about computers. There is something deeply funny about a movement claiming to invent the future and grounding its case in a Russian émigré's airport novels about steel barons in love with their own reflections. Free-market absolutism. Specifically the Milton Friedman, Chicago School, supply-side flavor. The market will sort it out. Regulation is theft. Wealth is virtue. George Gilder, who co-wrote the Magna Carta, had previously written a book called Wealth and Poverty that helped sell Reaganomics to the masses. He then wrote Microcosm , which argued that microprocessors plus deregulated capitalism would liberate humanity. He was very serious about this. Don't worry, Gilder is still out there. He loves the blockchain and crypto now. He now writes about how Bitcoin will save the soul of capitalism, which it is somehow doing while also destroying the planet. Both can be true in his cosmology. The ideology is flexible like that. A fantasy of communitarian outcomes. This is the part that should make you laugh out loud. After establishing that government is bad, regulation is theft, and the individual is sovereign, the cyberlibertarians then promise that the result of all this will be... rich, decentralized, harmonious community life. Negroponte: "It can flatten organizations, globalize society, decentralize control, and help harmonize people." Democracy will flourish. The gap between rich and poor will close. The lion will lie down with the lamb, and the lamb will have a Pentium II. We also have the advantage of hindsight and know, without question, that all of these predicted outcomes were wrong. Not 'directionally wrong' or 'wrong in the details.' Wrong the way it would be wrong to predict that if you set your kitchen on fire, the result will be a renovation. You have to hold these four ideas in your head at the same time to see the trick. The cyberlibertarians wanted you to believe that radical individualism plus deregulated capitalism plus inevitable technology would produce communitarian utopia. This is, on its face, insane. It is the economic equivalent of claiming that if everyone punches each other really hard, eventually we'll all be hugging. But Winner's sharpest observation, the one I keep coming back to, isn't about any of the four pillars individually. It's about the move underneath them. He writes: This is the entire game. This is how "don't tread on me" becomes "Meta should be allowed to do whatever it wants." This is how the rights of the lone hacker working in their garage become indistinguishable from the rights of a multinational with a market cap larger than most countries' GDP. The Magna Carta literally argues that the government should reduce barriers to collaboration between cable companies and phone companies in the name of individual freedom and social equality . Winner caught this in 1997. What makes the essay uncomfortable to read now is that Winner wasn't even predicting the future. He was just describing what was already happening and noting where it would obviously lead. He saw the media mergers and asked the question nobody in the industry wanted to answer: what happened to the predicted collapse of large centralized structures in the age of electronic media? Where, exactly, did the decentralization go? He saw that the cyberlibertarians were going to deliver the opposite of everything they promised, and that they were going to keep getting paid to promise it anyway. He was writing before Google. Before Facebook. Before the iPhone. Before YouTube. Before Twitter, Bitcoin, Uber, AirBnB, OpenAI, and the entire app economy. Before any of the actual examples that would eventually prove him right existed. He just looked at the people doing the talking, listened to what they were saying, and wrote down where it ended. It is not a long essay. He didn't need a long essay. The future was right there on the page, in their own words. He just had to read it back to them. The essay closes with a question that has, to my knowledge, never been seriously answered by the industry it was aimed at: Twenty-eight years later, the industry still treats this question as somewhere between naive and seditious. It's the question Barlow's declaration was specifically designed to make unaskable. And it remains, to this day, the only question that actually matters. When you look at these early formative writings, so much of what we see now becomes clear. The cyberlibertarian deal was always the same: you're on your own. The industry would build the infrastructure, take the profits, and shove every consequence, every harm, every cost, every responsibility, onto somebody else. There is no greater example to me than the moderator. Anyone who has ever moderated a forum or a subreddit knows that adding the word "cyber" to a space doesn't suddenly turn people into better humans. People are still people. They flame each other, they post slurs, they doxx, they harass, they spam, they post CSAM, they radicalize each other, they grief, they coordinate, they lie. A space with humans in it requires governance. They produce, with frightening regularity, the exact behavior any kindergarten teacher could have predicted. Then they act surprised. But the cyberlibertarian model required pretending it was unforeseeable. The platforms couldn't acknowledge that they needed governance because acknowledging it would mean acknowledging responsibility, and acknowledging responsibility would mean acknowledging liability, and acknowledging liability would mean the entire economic model collapses. So instead the industry invented a beautiful fiction: governance happens, but it happens by magic, performed by volunteers, for free, who we will simultaneously rely on and mock. Reddit is run by unpaid moderators. Wikipedia is run by unpaid editors. Stack Overflow was run by unpaid experts and is now a ghost town. On TikTok and Twitter it is the unknowable "algorithm" that is the cause of and solution to every problem backed by capricious moderators who delight in stopping free speech. Unless you don't like it, then it's negligence moderation in defense of your enemies. Open source is run by unpaid maintainers having nervous breakdowns. The platforms collect the rent. The people doing the actual work of making the platforms livable get nothing, and when they ask for anything like recognition, tools, basic protection from harassment, they're told they're power-tripping nerds who should touch grass. This is also the crypto story, just with the masks off. What if we made worse money on purpose, money that bypassed every protection consumers had won over the previous century, money that couldn't be reversed when stolen, money that funded ransomware attacks on hospitals and pump-and-dumps targeting people's retirement accounts? The cyberlibertarian answer was: that's freedom. The losses were real. People killed themselves. Hospitals had to turn away patients. The architects became billionaires and bought yachts and now sit on the boards of AI companies, where they are reinventing the same con with a new vocabulary. Now Winner got one thing wrong, and it's worth pausing on, because it's the most interesting wrinkle in all of this. What actually happened was weirder and worse. The cyberlibertarians became the corporations. They didn't sell out. They didn't betray their principles for the first offer of money. They simply scaled until their principles became inconvenient, and then they stopped mentioning them. Once the platforms got large enough to be unstoppable, once they captured enough of the regulatory apparatus to write their own rules, the libertarian rhetoric got quietly shelved like a college poster you took down before your in-laws came over. Meta no longer pretends it stands for free speech and seemingly takes delight in putting its thumb on the scale. TikTok users have invented an entire euphemistic shadow language to evade automated censorship like "unalive," "le dollar bean," "graped" that would have made 1996 Barlow weep into his bolo tie. Copyright and patents matter when they're Apple's copyright and patents. Or Googles. Or OpenAIs. Go try to make a Facebook+ website and see how quickly Meta is capable of responding to content it finds objectionable. Cyberlibertarianism was the ladder. Once they were on the roof, they kicked it away and started charging admission to look at the view. Remember I like the Internet. I said it in the beginning and it is still true. I love the Fediverse, I love weird Discords about small tabletop RPGs I'm in. I spend hours in the Mister FPGA forums. There are corners that are good. But they're mostly good because they're not big enough to be worth breaking up. It feels increasingly like I'm hanging out in the old neighborhood dive bar after most of the regulars have moved away. The lighting is the same. The bartender remembers your order. But you can hear yourself think now, and that's mostly because the room is half empty and the jukebox finally died. The new clientele is from out of town. They are taking pictures of the menu. If we want to have a serious conversation about why we are in the situation we're in, it is no longer possible to pretend that the broken ideology that put us on this trajectory is still somehow compatible with the harsh realities that surround us. It is not clear to me if democracy can survive a deregulated Internet. A deregulated Internet filled with LLMs that can perfectly impersonate human beings powered by unregulated corporations with zero ethical guidelines seems like a somewhat obvious problem. Like an episode of Star Trek where you the viewer are like "well clearly the Zorkians can't keep the Killbots as pets." It doesn't take some giant intellect to see the pretty fucking obvious problem . If we want to save the parts of the internet worth saving, we have to evolve. We have to find some sort of ethical code that says: just because I can do something and it makes money, that is not sufficient justification to unleash it on the world. Or, more simply: just because I want to do something and you cannot actively stop me, that does not make doing it a good idea. We have waited thirty years for the cyberlibertarian future to arrive and produce the promised harmonious community. It's time to face the facts. It's never coming. The bus left in 1996. The bus was never real. People did not get better because they went online. Giving everyone access to a raw, unfiltered pipeline of every fact and lie ever produced did not turn them into better-educated people. It broke them. It allowed them to choose the reality they now inhabit, like ordering off a menu. If I want to believe the world is flat, TikTok will gladly serve me that content all day. Meta will recommend supportive groups. There will be hashtags. There will be Discords. There will be a guy named Trent who runs a podcast. I will never have to face the deeply uncomfortable possibility that I might be wrong about anything, ever, until the day I die, surrounded by people who agree with me about everything, including which of the other mourners are secretly lizards. That is the internet we built. It was not an accident. It was the product of a specific ideology, written down by specific people, at a specific cocktail party in Davos, in 1996. Winner watched it happen and told us where it was going. We did not listen. There is still time, maybe, to start.

Culture

▲ 0

0 views

matduggan.com 2 months ago

If I Could Make My Own GitHub

My friend and I have a game where we talk about what we'd do if we were rich. Not rich like 'paid off the mortgage' rich. Rich like a man who owns a submarine he's never been inside. Rich like a man whose third wife has a skincare line. Tech-titan rich — the kind of money that buys you a compound in Wyoming and the confidence to wear the same gray t-shirt to congressional testimony." One of mine, for a long time, has been the dream of making a new forge. I was prompted to write this after reading the good post about Ghostty leaving GitHub but it's something I've written and talked about for a few years. Given how bad GitHub has become at its core job, it seemed like a fun opportunity to try and write up what my billionaire folly of a forge would look like. This folly would have less penile rockets filled with aging celebrities. GitHub, GitLab and Gitea (those being the 3 I've used the most) are all modeled on effectively the same design. There are differences, but you can tell that GitHub sets the pattern for the industry and then those features are ported over to the other two with varying levels of success. The issue with all of these is that they're designed to add things git doesn't do that you need. Git is great at what it is designed to do, but what it is designed to do isn't the way most people are using it. Git is a perfect tool for kernel development. It is a decentralized distributed version control system that relies on the idea of patches being sent to maintainers over email. You trust those maintainers to maintain their sections and merge in the stuff that makes sense and not merge in the other stuff. It's a pretty high trust environment that places very few restrictions on how online a specific contributor is or what system they are using. If you have a laptop from 2010 that connects to the Internet once a week you can still be a meaningful contributor to a project with these workflows. . However, in most jobs, git is effectively just the way I pull and push from a centralized repository stored in a forge. All the important stuff happens inside the forge, and very little of it happens on my client. Pull Requests are how I enforce the four-eyes principle, GitHub Actions are how I run my tests and linting on those Pull Requests to ensure they are functional and meet my organizational requirements, the user's identity in relation to that forge is how I verify who they are. I track issues with my code through Issues and cut releases for users to download through Releases. There's not a lot of git in this workflow, this is mostly placed on top of git. So here are the primary issues I see with modern forges that I would love to solve. Absolutely. There are a lot of tools that do parts of this. I want someone to take them, put them all together and fit them up. I want JJ as the VCS, I want this as the forge and I want the expectation that I as a user could live happily with a raspberry pi as a forge for a long time. I want those forges designed around modern concepts like object storage and shallow clones and getting constantly hammered by LLM bots. Now in a universe where GitHub was doing a good job, I wouldn't even bother writing this up. GitHub is the default and talking to people about overcoming the default is usually a waste of time. Heinz is the default ketchup, when I order a Coke I don't want a Pepsi and if I'm going to use a forge up until 2026 there would have to be an amazing reason for me to not choose the one that everyone uses. Up until recently other forges have been like sweet potato french fries, which is to say never the thing you actually want. But we live in a world where the monolithic forge is breaking down and nobody has built the replacement. The people with the money are busy with the rockets. The people with the taste are busy with their day jobs. And the rest of us are opening PRs titled 'asdfasdf' at midnight, waiting for a robot to check them, wondering when the tool we spend our whole working lives inside stopped being built for us. If I ever get the submarine money, I'll let you know. Stuff happens in the wrong order. You know the PR. Commit 1: 'Feature.' Commit 2: 'fix.' Commit 3: 'fix.' Commit 4: 'actually fix.' Commit 5: 'please.' Commit 6, made at 11:47 PM on a Thursday: 'asdfasdf'. This person has a family. This person has hobbies. This person is, at this moment, crying. You don't want the feedback loop after the commit you want it before. Let me do an enforced pre-commit hook to run the jobs remotely on the forge and provide the feedback to the user before they push. PR approval is too boolean. The PR is approved or it's not approved. Real code review, like real life, lives in the middle. 'Sure, fine, we'll deal with it later' is a legitimate human response and should be a legitimate button. Gerrit has a better model for this. If I weakly approve something as a maintainer, let me flag it for later. PRs are too inflexible. I don't need 4 eyes on every change, especially in a universe where LLMs exist. The global GDP lost annually to senior engineers staring at a four-line PR waiting for someone — anyone — to type 'LGTM' could fund a moon mission. A nice one. With legroom. Let me customize and more easily control this. If the person is a maintainer and the LLM says its low risk/no risk just let them go. Stacked PRs are just better. They're easier to review and understand. They have to be a first-class citizen not an add-on through a tool other than your VCS. A forge shouldn't do everything. Issue tracking yes. Kanban board, probably not. Wiki? I doubt it. Everything tools always turn into crap. You add features when its easy to add features and then pay the maintenance price for those features forever regardless of their rate of adoption because now someone, somewhere uses them and you are locked in. The standard unit of hosting is too large. Running Github Enterprise is a big task. Running GitLab is also a relatively big ask. These are complicated products with a lot of moving pieces. I want smaller individual units of hosting that I can link together to make an organization. It's fine if they're not globally federated and I need to make an account for each Organization, but an Organization should be flexible enough to let me say "these 12 Raspberry Pis are my org". I don't know how they communicate securely, I hire people for that problem. My local copy of the repo should be a representation of the entire repo, not just the code. I should be able to approve a PR from the same VCS I use to check in the code. I should be able to go through my issues by looking through local files. On the flip side, since I need to be online all the time to really work with a team, don't make me pay the storage price all the time. I want my VCS and my forge to work together. If I clone a repo, I want a pretty limited history for that repo when I clone. If I start to go back in time, spin up a worker to go fetch that stuff from the VCS when I need it. I don't need to hammer the forge with giant clone requests on the assumption that I might need to rebuild the forge at any moment with the entire history of the entire project. Actions need to be signed, SHA'd and usable offline. If I want, I should be able to get tarballs of all my actions, stick them in the repo and tell my system "don't go anywhere for checkout action, that's right here". If I say latest, have that work like Dependabot does now where it opens up a PR to put the latest tarballs in my repo. Actions are critical and they should be runnable on my local machine through the same VCS if I want to.

Open Source Git

▲ 0

0 views

matduggan.com 3 months ago

You can absolutely have an RSS dependent website in 2026

I write stuff here. Sometimes the stuff is good. Sometimes it reads like I wrote it at 2 AM after an argument with a YAML file, which is because I did. But one decision I made early on was that I didn't want to offer an email newsletter. Part of this was simple economics. At one point I did have a Subscribe button up, and enough people clicked it that the cost of actually sending those emails started to resemble a real bill. Sending thousands of emails when you have no ads, no sponsors, and no monetization strategy beyond "I guess people will just... read it?" doesn't make a lot of financial sense. But the bigger reason — the one I actually care about — is that I didn't want a database full of email addresses sitting under my control if I could possibly avoid it. There's a particular flavor of anxiety that comes with being the custodian of other people's personal data, a low-grade dread not unlike realizing you've been entrusted with someone's elderly cat for two weeks and the cat has a medical condition. I can't lose data I don't have. I never need to lie awake wondering whether some user is reusing their bank password to log into my website just to manage their subscription preferences. The best way I can safeguard user data is by never having any in the first place. It's not a security strategy you'll find in any textbook, but it is airtight. Now, when I explained this philosophy to people who run similar websites, the reaction was — and I'm being generous here — warm laughter . The kind of laughter you get when you ask if an apartment in Copenhagen is under $1,000,000. Email newsletters are the only way to run a site like this, they said. RSS is dead, they said. You might as well be distributing your writing via carrier pigeon or community bulletin board. One person looked at me the way you'd look at someone who just announced they were going to navigate cross-country using only a paper atlas. Not angry. Just sad. I'm lucky in that I'm not trying to get anyone to pay me to come here. If I were, the math would probably change. I'd be out there A/B testing subject lines and agonizing over open rates like everyone else, slowly losing pieces of my soul in a spreadsheet. But if your question is simply, "Can I make a hobbyist website that actual humans will find and read without an email newsletter?" — the answer is a resounding yes. And I have the logs to prove it. All of this is from Nginx access.log. These logs get rotated daily and don't include the majority of requests that hit the Cloudflare cache before they ever reach my server, so the real numbers are higher. But I think they're reasonably representative of the overall shape of things. About half my traffic is readers hitting or — people who have, of their own free will, pointed an RSS reader at my site and said yes, tell me when this person has opinions again . The other half are arriving via a specific link they stumbled across somewhere in the wild. If we do a deeper dive into that specific RSS traffic, we learn a few interesting things. The user-agent breakdown shows the usual suspects — the RSS readers you'd expect, the ones that have been around long enough to have their own Wikipedia articles. There are also some abusers in the metrics. I have no idea what "Daily-AI-Morning" is, but whatever it's doing, it's polling my feed with the frantic energy of someone refreshing a package tracking page on delivery day. The time distribution, though, is pretty good — spread out across the day in a way that suggests real humans checking their feeds at real human intervals, rather than a single bot hammering me every thirty seconds. My conclusion is this: if you want to run a website that relies primarily on RSS instead of email newsletters, you absolutely can. The list of RSS readers hasn't dramatically changed in a long time, which is actually reassuring — it means the ecosystem is stable, not dead. The people who use RSS really use RSS. They're not trend-chasers. They're the type who still have a working bookmark toolbar. They are, in the best possible sense, your people. Effectively, if you make your site RSS-friendly and you test it in NetNewsWire, you will — slowly, quietly, without a single "SUBSCRIBE FOR MORE" pop-up — build a real audience of people who actually want to read what you write. No email database required. No passwords to leak. No giant confusing subscription system.

Web Development Blogging Yaml

▲ 0

0 views

matduggan.com 3 months ago

I Can't See Apple's Vision

I don't typically write about Apple stuff. It's the most written-about company on earth. Every product launch gets the kind of forensic scrutiny normally reserved for plane crashes and celebrity divorces. Mostly though, I feel like a line cook at a Denny's talking trash about whether the French Laundry has lost their way. I'm back here microwaving a Grand Slam and opining about Thomas Keller's sauce work. The engineers I know personally at Apple are, on average, much more talented than me. They work harder, they do it for decades without a break, and none of them have ever shipped a feature while still wearing pajama pants at 2 PM. It seems insane for someone of my mediocre talent to critique them. It also feels a little dog-pile-y. Apple employees know Tahoe sucks. They know it the way you know your haircut is bad — they don't need strangers on the internet confirming it. And to be fair, there's genuinely great work buried inside Tahoe: the clipboard manager, the automation APIs, a much-improved Spotlight. But visually it's gross, and that matters when your entire brand identity is "we're the ones who care about design." Instead, I want to talk about a bigger problem and one that I do feel qualified to talk about because I am very guilty of committing this sin. I don't see a cohesive vision for MacOS and WatchOS. This, more than one bad release, seems far worse to me and dangerous for the company. Since this is already 2000 words as a draft I'll save WatchOS for another time. I'm verbose but even I have limits. Now to be clear this isn't across every product . iPadOS has a strong vision and have the strength of their convictions to change approaches. The different stabs at solving the window problem inside of the iPad and make it so that you still have an iPad experience while being able to do multiple things at the same time is proof of that. iOS has an incredibly strong vision for what the product is and isn't and how the software works with that. VisionOS and tvOS are less strong, but visionOS is still finding its footing in a brand new world. The Apple TV hardware and software is in a weirdly good position even though nothing has changed about it in what feels like geological time. I've purchased every version of the Apple TV, and with the exception of that black glass remote — the one that felt like it was designed by someone who had never held a remote, or possibly a physical object — everything has been pretty good. I'm still not clear how storage works on the Apple TV and I don't think anybody outside of Apple does either. I'm not even sure Apple knows. But somehow it's fine. But with watchOS and MacOS we have 2 software stacks that seem to be letting down the great hardware they are installed in. They seem to be evolving in random directions with no clear end goal in mind. I used to be able to see what OS X was aiming for, even if it didn't hit that goal. Now with two of Apple's platform I'm not able to see anything except a desire to come up with something to show as this years release. When I got my first Mac — an iBook G3 — the experience was like test-driving a Ferrari that someone had fitted with a lawnmower engine. You'd click on the hard drive icon and wait. And wait. And in those few seconds of waiting, you'd think: man, this would be incredible if the hardware could keep up. The software had somewhere it wanted to go. The hardware just couldn't get it there yet. This trend continued for a long time on OS X, where you'd see Apple really pushing the absolute limits of what it could get away with. After the rock solid stability of 10.4 Apple took a lot of swings with 10.5 and they didn't all land. The first time you opened the Time Machine UI and the entire thing crawled to an almost crash, you'd think boy maybe this wasn't quite ready for prime time . But this entire time there wasn't really a question, ever, that there was a vision for what this looked like. The progression of OS X from the beta onward was this: OS X tried to accommodate you, not the other way around. When you look at these screenshots I'm always surprised how light the touch is. There isn't a lot of OS here to the user. Almost everything is happening behind the scenes and the stuff you do see is pretty obvious. The first time I thought "oh man, they've lost the thread" was Notifications. On iOS, Notifications make sense — you've got apps buried in folders three screens deep, so a unified system for surfacing what's happening is genuinely useful. On macOS, this design makes absolutely no sense at all. You can see your applications. They're right there. In the Dock. Which is also right there. This is the beginning of this feeling of "we aren't sure what we're doing here with the Mac anymore". iOS users like Notifications so maybe you dorks will too? It consumes a huge amount of screen real estate, it was never (and still isn't) clear what should and shouldn't be a notification. Even opening up mine right now it's filled with garbage that doesn't make sense to notify me about. A thing has completed running the thing that I asked it to run? Why would I need to know that? There is also already a clear way to communicate this information to me. The application icon adds an exclamation point or bounces up and down in the dock. With Notifications you end up with just garbage noise taking up your screen for no reason. Maybe worse, it's not even garbage designed with the Mac in mind. It's just like random crap nobody cares about that looks exactly like iOS Notifications. The issue with copying everything from iOS is that it's like copying someone's homework — except they go to a different school, in a different country, studying a different subject. It's not just wrong in the way where you tried and failed. It's wrong in a way that makes everyone who encounters it deeply uncomfortable. The teacher doesn't even know where to begin. They just stare at it. For years afterwards it seemed like the purpose of MacOS was just to port iOS features to the Mac years after their launch on iOS. Often these didn't make much sense or hadn't had a lot of effort expended in making them very Mac-y. Like there was clearly a favorite child with iOS, then a sassy middle child with iPadOS and then, like a 1980s sitcom where there was a contract dispute, "another child" you saw every 5th episode run down the stairs in the background with no lines. Me at home would shout at my TV "I knew they didn't kill you off MacOS!". Now with Tahoe there's clearly some sort of struggle happening inside of the team. And here's what's maddening — buried inside this visual catastrophe, someone at Apple is doing incredible work. Clipboard management has been table stakes in the third-party ecosystem for years. Apple finally added a version that handles 90% of use cases. It's classic Sherlocking: Apple shows up ten years late to the party, brings a decent bottle of wine, and somehow half the guests leave with them. Same with Spotlight. Spotlight hasn't gotten a ton of love in years. Suddenly it's really competing with third-party tools. If you're searching for a file, you can filter it based on where the file is stored. Type "name of Directory" press the Tab key, and then type the name of the file before pressing Enter. This is great! We finally have keyword search for stuff like . Application shortcuts for opening stuff with things like for Firefox is nice. Assign a quick key like “se” to  Send Email . Type it in Spotlight, hit enter, and compose your message. This is all classic Apple thinking which is "how can we make the Mac as good as possible such that you, the user, don't need to download any third-party applications to get a nice experience". You don't need a word processor, you have a word processor and a spreadsheet application and presentation software and a PDF viewer and a clipboard manager and a system launcher and automation APIs etc etc etc. This is a vision that is consistent throughout the entire systems history, how can we help you do the things you need to do more easily. But the reason why I'm stressed as someone who is pretty invested in the ecosystem is that the visual stuff is so bad and not just bad, but negligent. We didn't test how it was gonna look under a bunch of situations so that's now someone else's problem. Whenever I get a finder sidebar covering folder contents so I had to resize the window every time, or the Dock freaks out and refuses to come back out, it feels like I installed one of those OS X skins for a Linux distro. I buy Apple stuff cause its nice to look at and this is horrible to look at. Why is this so big? Why did you cut off the word "Finder" from Force Quit? Everywhere you look there's a million of these papercuts. We have a resolution on our laptops screen that would have made people collapse in 2005 why must we waste all of it on UI elements? Also you can't grab window edges as shown by the best post ever written here: https://noheger.at/blog/2026/01/11/the-struggle-of-resizing-windows-on-macos-tahoe/ Why is there so much empty space between everything? Why are there six ways to do literally everything? Why did we copy the concept of Control Center from iOS at all if there's very little limit on screen real estate and we could already do this from the menu bar? So we're going to keep the Mac menu bar but we're going to add a full iPad control system and then we're going to use the iPad control system to manage the menu bar . I will say the "Start Screen Saver" makes me laugh because its a mistake I would make in CSS. The text is too long so the button is giant but we didn't resize the icon so it looks crazy. Now do we need the same text inside the button as outside of it? No, and that leads me to the other banger. It's pretty clear the two white boxes inside of "Scene or Accessory" were supposed to be text, Scene on the top and then Accessory on the bottom, but SwiftUI couldn't do that so they left the placeholder. Somewhere there is a Jira ticket to come back to this that got trashed. Also, complete aside. Has anyone in the entire fucking world ever run Shazam from a Mac? What scenario are we designing for here? I hear a banger at the coffee shop so I hold my MacBook Pro up over my head like John Cusack in Say Anything , hoping it catches enough audio before my arms give out? "Recognize Music" is in my menu bar, taking up space that could be used for literally anything else, on the off chance I need to identify a song using a device that weighs four pounds and has no microphone worth using in a noisy room. If you are going to copy ipadOS's homework you need to think about it for 30 seconds . So my hope is that the improvement camp wins. That the people who built the better Spotlight and the clipboard manager and the automation APIs are the ones who get to set the direction. Because right now it feels like the best work on macOS is being done in spite of the overall vision, not because of it. Like someone's sneaking vegetables into a toddler's mac and cheese. The good stuff is in there — you just have to eat around a lot of neon orange nonsense to find it. Steve Jobs talked about creative people having to persuade five layers of management to do what they know is right. I don't know how many layers there are now. But I know what it looks like when the creative people are losing that argument, and I know what it looks like when they're winning it. Right now, on macOS, it looks like both are happening at the same time, in the same release, on the same screen. And that's scarier than any one bad design choice. It's Unix, but you never need to know that. All the power, none of the beard. You get the stability of a server OS without ever having to type into anything. Everything annoying is abstracted away. Drivers? Gone. "Installing" an application? You drag it into a folder. That's it. That's the install. It felt like the computer was meeting you more than halfway — it was practically doing your job for you and then apologizing for not doing it sooner. If it seems like it should work, it works. Double-click a PDF, it opens. Put in a DVD, it plays. Drag an app to the Applications folder and it becomes an application. This sounds obvious now, but in 2003 this was like witchcraft if you were coming from Windows. But it was also serious. It wasn't cluttered with stupid bullshit. It was designed for people who made things — with real font management, color calibration, the works. The OS tried to stay out of your way. Your content was the show; everything else was stagecraft.

CSS Design

▲ 0

0 views

matduggan.com 3 months ago

Hosting a Snowflake Proxy

In the nightmarish world of 2026 it can be difficult to know how to help at all. There are too many horrors happening to quickly to know where one can inject even a small amount of assistance. However I wanted to quickly post about something I did that was easy, low impact and hopefully helps a tiny fraction of a fraction of a percent of people. So I was browsing Mastodon when someone posted a link asking for people to host Snowflake proxies. Snowflake is a lightweight proxy best explained by David Fifield below. Effectively it is a lightweight and easy to run way to bypass censorship that doesn't require running a VPN and involves almost zero technical knowledge. It's quite the design and one that I kept shaking my head thinking "man I never would have thought of this in a mission years" as I read more about how it works. So I have a box sitting on an internet connection where I'm lucky enough to have plenty of excess capacity. I figured "why not share it". I thought I'd post the process here in case people were curious but were worried about how much bandwidth it might use or how many resources. Setting it up on a Debian box took like 5 minutes. That's it. So this has been running for two weeks and in that two weeks I've served up the following amount of traffic: CPU usage is quite low, Memory is slightly higher than I would have thought but that's likely a function of running for so long. Remember you can modify the systemd service file to limit memory if you are interested in running this yourself but are concerned about crossing a gig of memory. All in all I haven't noticed I'm running this at all. Obviously its great to run the browser extension to increase the pool of IP addresses and keep them from becoming static and blockable, but if you have a dedicated box with a large amount of bandwidth and are looking for a quick 20 minute project to help out people trying to deal with internet censorship, this seems like a good one to me. Get the package from here: https://packages.debian.org/sid/snowflake-proxy with: Make sure it is enabled and running

Open Source

▲ 0

0 views

matduggan.com 3 months ago

Markdown Ate The World

I have always enjoyed the act of typing words and seeing them come up on screen. While my favorite word processor of all time might be WordPerfect ( here ), I've used almost all of them. These programs were what sold me on the entire value proposition of computers. They were like typewriters, which I had used in school, except easier in every single way. You could delete things. You could move paragraphs around. It felt like cheating, and I loved it. As time has gone up what makes up a "document" in word processing has increased in complexity. This grew as word processors moved on from being proxies for typewriters and into something closer to a publishing suite. In the beginning programs like WordPerfect, WordStar, MultiMate, etc had flat binary files with proprietary formatting codes embedded in there. When word processors were just proxies for typewriters, this made a lot of sense. But as Microsoft Word took off in popularity and quickly established itself as the dominant word processor, we saw the rise of the .doc file format. This was an exponential increase in complexity from what came before, which made sense because suddenly word processors were becoming "everything tools" — not just typing, but layout, images, revision tracking, embedded objects, and whatever else Microsoft could cram in there. At its base the is a Compound File Binary Format, which is effectively just a FAT file system with the file broken into sectors that are chained together with a File Allocation Table. It's an interesting design. A normal file system would end up with sort of a mess of files to try and contain everything that the has, but if you store all of that inside of a simplified file system contained within one file then you could optimize for performance and reduced the overhead that comes with storing separate objects in a flat file. It also optimizes writes, because you don't need to rewrite the entire file when you add an object and it keeps it simple to keep revision history. But from a user perspective, they're "just" dealing with a single file. ( Reference ) The .doc exploded and quickly became the default file format for humanity's written output. School papers, office memos, résumés, the Great American Novel your uncle was definitely going to finish — all .doc files. But there was a problem with these files. They would become corrupted all of the goddamn time. Remember, these were critical documents traveling from spinning rust drives on machines that crashed constantly compared to modern computers, often copied to floppy disks or later to cheap thumb drives you got from random vendor giveaways at conferences, and then carried to other computers in backpacks and coat pockets. The entire workflow had the structural integrity of a sandwich bag full of soup. So when Word was saving your critical file, it was actually doing a bunch of different operations. It was: These weren't atomic operations so it was super easy in an era when computers constantly crashed or had problems to end up in a situation where some structures were updated and others weren't. Compared to like a file where you would either get the old version or a truncated new version. You might lose content, but you almost never ended up with an unreadable file. With as someone doing like helpdesk IT, you constantly ended up with people that had just corrupted unreadable files. And here's the part that really twisted the knife: the longer you worked on the same file, the more important that file likely was. But Word didn't clean up after itself. As a .doc accumulated images, tracked changes, and revision history, the internal structure grew more complex and the file got larger. But even when you deleted content from the document, the data wasn't actually removed from the file. It was marked as free space internally but left sitting there, like furniture you moved to the curb that nobody ever picked up. The file bloated. The internal fragmentation worsened. And the probability of corruption increased in direct proportion to how much you cared about the contents. Users had to be trained both to save the file often (as AutoRecover wasn't reliable enough) and to periodically "Save As" a new file to force Word to write a clean version from scratch. This was the digital equivalent of being told that your car works fine, you just need to rebuild the engine every 500 miles as routine maintenance. The end result was that Microsoft Word quickly developed a reputation among technical people as horrible to work with. Not because it was a bad word processor — it was actually quite good at the word processing part — but because when a user showed up at the Help Desk with tears in their eyes, the tools I had to help them were mostly useless. I could scan the raw file for text patterns, which often pulled out the content, but without formatting it wasn't really a recovered file — it was more like finding your belongings scattered across a field after a tornado. Technically your stuff, but not in any useful arrangement. Sometimes you could rebuild the FAT or try alternative directory entries to recover slightly older versions. But in general, if the .doc encountered a structural error, the thing was toast and your work was gone forever. This led to a never-ending series of helpdesk sessions where I had to explain to people that yes, I understood they had worked on this file for months, but it was gone and nobody could help them. I became a grief counselor who happened to know about filesystems. Thankfully, people quickly learned to obsessively copy their files to multiple locations with different names — thesis_final.doc, thesis_final_v2.doc, thesis_FINAL_FINAL_REAL.doc — but this required getting burned at least once, which is sort of like saying you learned your car's brakes didn't work by driving into a bus. So around 2007 we see the shift from to , which introduces a lot of hard lessons from the problems of . First, it's just a bundle, specifically a ZIP archive. Now in theory, this is great. Your content is human-readable XML. Your images are just image files. If something goes wrong, you can rename the file to .zip, extract it, and at least recover your text by opening document.xml in Notepad. The days of staring at an opaque binary blob and praying were supposed to be over. However, in practice, something terrible happened. Microsoft somehow managed to produce the worst XML to ever exist in human history. Let me lay down the scope of this complexity, because I have never seen anything like it in my life. Here is the standards website for ECMA-376. Now you know you are in trouble when you see a 4 part download that looks like the following: If you download Part 1, you are given the following: Now if you open that PDF, get ready for it. It's a 5039 page PDF. I have never conceived of something this complicated. It's also functionally unreadable, and I say this as someone who has, on multiple occasions in his life, read a car repair manual cover to cover because I didn't have anything else to do. I once read the Haynes manual for a 1994 Honda Civic like it was a beach novel. This is not that. This is what happens when a standards committee gets a catering budget and no deadline. There was an accusation at the time that Microsoft was making OOXML deliberately more complicated than it needed to be — that the goal was to claim it was an "open standard" while making the standard so incomprehensibly vast that it would take a heroic effort for anyone else to implement it. I think this is unquestionably true. LibreOffice has a great blog post on it that includes this striking comparison: So the difference between ODF format and the OOXML format results in a exponentially less complicated XML file. Either you could do the incredible amount of work to become compatible with this nightmarish specification or you could effectively find yourself cut out of the entire word processing ecosystem. Now without question this was done by Microsoft in order to have their cake and eat it too. They would be able to tell regulators and customers that this wasn't a proprietary format and that nobody was locked into the Microsoft Office ecosystem for the production of documents, which had started to become a concern among non-US countries that now all of their government documents and records were effectively locked into using Microsoft. However the somewhat ironic thing is it ended up not mattering that much because soon the only desktop application that would matter is the browser. The file formats of word processors were their own problems, but more fundamentally the nature of how people consumed content was changing. Desktop based applications became less and less important post 2010 and users got increasingly more frustrated with the incredibly clunky way of working with Microsoft Word and all traditional files with emailing them back and forth endlessly or working with file shares. So while was a superior format from the perspective of "opening the file and it becoming corrupted", it also was fundamentally incompatible with the smartphone era. Even though you could open these files, soon the expectation was that whatever content you wanted people to consume should be viewable through a browser. As "working for a software company" went from being a niche profession to being something that seemingly everyone you met did, the defacto platform for issues, tracking progress, discussions, etc moved to GitHub. This was where I (and many others) first encountered Markdown and started using it on a regular basis. John Gruber, co-creator of Markdown, has a great breakdown of "standard" Markdown and then there are specific flavors that have branched off over time. You can see that here . The important part though is: it lets you very quickly generate webpages that work on every browser on the planet with almost no memorization and (for the most part) the same thing works in GitHub, on Slack, in Confluence, etc. You no longer had to ponder whether the person you were sending to had the right license to see the thing you were writing in the correct format. This combined with the rise of Google Workspace with Google Docs, Slides, etc meant your technical staff were having conversations through Markdown pages and your less technical staff were operating entirely in the cloud. Google was better than Microsoft at the sort of stuff Word had always been used for, which is tracking revisions, handling feedback, sharing securely, etc. It had a small subset of the total features but as we all learned, nobody knew about the more advanced features of Word anyway. By 2015 the writing was on the wall. Companies stopped giving me an Office license by default, switching them to "you can request a license". This, to anyone who has ever worked for a large company, is the kiss of death. If I cannot be certain that you can successfully open the file I'm working on, there is absolutely no point in writing it inside of that platform. Combine that with the corporate death of email and replacing it with Slack/Teams, the entire workflow died without a lot of fanfare. Then with the rise of LLMs and their use (perhaps overuse) of Markdown, we've reached peak . Markdown is the format of our help docs, many of our websites are generated exclusively from Markdown. It's now the most common format that I write anything in. This was originally written in Markdown inside of Vim. There's a lot of reasons why I think Markdown ended up winning, in no small part because it solved a real problem in an easy to understand way. Writing HTML is miserable and overkill for most tasks, this removed the need to do that and your output was consumable in a universal and highly performant way that required nothing of your users except access to a web browser. But I also think it demonstrates an interesting lesson about formats. and . along with ODF are pretty highly specialized things designed to handle the complexity of what modern word processing can do. LibreOffice lets you do some pretty incredible things that cover a huge range of possible needs. Markdown doesn't do most of what those formats do. You can't set margins. You can't do columns. You can't embed a pivot table or track changes or add a watermark that says DRAFT across every page in 45-degree gray Calibri. Markdown doesn't even have a native way to change the font color. And none of that mattered, because it turns out most writing isn't about any of those things. Most writing is about getting words down in a structure that makes sense, and then getting those words in front of other people. Markdown does that with less friction than anything else ever created. You can learn it in ten minutes, write it in any text editor on any device, read the source file without rendering it, diff it in version control, and convert it to virtually any output format. The files are plain text. They will outlive every application that currently renders them. They don't belong to any company. They can't become corrupted in any meaningful way — the worst thing that can happen to a Markdown file is you lose some characters, and even then the rest of the file is fine. After decades of nursing .doc files like they were delicate flowers that you had to transport home strapped to your car roof, the idea of a format that simply cannot structurally fail is not just convenient. It's a kind of liberation. I think about this sometimes when I'm writing in Vim at midnight, just me and a blinking cursor and a plain text file that will still be readable when I'm dead. No filesystem-within-a-filesystem. No sector allocation tables. No 5,039-page specification. Just words, a few hash marks, and never having to think about it again. Updating the document stream (your text) Updating the formatting tables Update the sector allocation tables Update the directory entries Update summary information Flush everything to disk Part 1 “Fundamentals And Markup Language Reference”, 5th edition, December 2016 Part 2 “Open Packaging Conventions”, 5th edition, December 2021 Part 3 “Markup Compatibility and Extensibility”, 5th edition, December 2015 Part 4 “Transitional Migration Features”, 5th edition, December 2016

Programming HTML Xml Rust

▲ 0

0 views

matduggan.com 4 months ago

Update to the Ghost theme that powers this site

I added a few modifications to the OSS Ghost theme that powers this site. You can get it here: https://gitlab.com/matdevdug/minimal-ghost-theme I tried to make it pretty easy to customize, but if you need something changed feel free to open an issue on the repo. Thanks for all the feedback! Added better image caption support. Added the cool Mastodon feature outlined here to attribute posts from your site back to your Mastodon username by following the instructions here.

Web Development Open Source

▲ 0

0 views

matduggan.com 4 months ago

Boy I was wrong about the Fediverse

I have never been an "online community first" person. The internet is how I stay in touch with people I met in real life. I'm not a "tweet comments at celebrities" guy. I was never funny enough to be the funniest person on Twitter. So when Twitter was accidentally purchased by a fascist high on ketamine, I moved to Mastodon mostly because it seemed to be “Twitter without the bullshit”. No recommended for you feed, no ads, it was broken in a way I find charming. Of course search was broken because all OSS social tools must have one glaring lack of functionality. In a nightmare world full of constant change it’s good to have a few constants to hold on to. A lot of the narrative at the time was “this is our flag in the ground in the fight against The Man”. It wasn’t clear in this context if they meant corporations or the media or the weird pseudo celebrity that had taken over social media where people would breathlessly tell me about shit like “Chris-Chan” and “Logan Paul bought a Pokemon card”. We all need pointless hobbies, but I care about YouTube stars like I care about distant stars dying. It’s interesting to someone somewhere but those people don’t talk to me. I mostly use social media as a place to waste time, not a platform to form para-social relationships to narcissists. I prefer my narcissism farm to table. I’d rather dig a grave with a rusty spoon than watch a Twitch “star”. Anyway, I watched mostly apathetically as the internet tried to rally itself to another cause. I read my news at the normal newspapers, watched my normal television and put social media off into its own silo. Then Trump effectively shut down the entire free press in the US in a series of bullshit lawsuits. See I had forgotten the one golden rule of capitalism. To thrive in capitalism one must be amoral. Now you can be wildly sickeningly successful with morals but you cannot reach that absolute zenith of shareholder value. Either you accept a lower share price and don’t commit atrocities or you become evil. There is no third option. So of course media corporations became bargaining chips for the oligarchs' actual businesses. Why fight a defamation suit when you can settle it by running favorable coverage and maybe bankrupting the media outlet you bought as a stocking stuffer? Suddenly I couldn’t find any reliable reporting about anything in the US. My beloved Washington Post became straight-up propaganda and desperate attempts to cope. "Best winter stews to make while you watch your neighbors get kidnapped at gunpoint." Twelve dollars a month for that. Threads was worthless because it’s the most boring social media website ever imagined. It’s a social media network designed by brands for brands, like if someone made a cable channel that was just advertisements and meta commentary about the advertisements you just saw. Billions of dollars at their disposal and Meta made a hot new social media network with the appeal of junk mail. Bluesky had a bunch of “stuff” but they’re trying to capture that 2008 Twitter lightning in a bottle which is a giant waste of time. We’re never going to go back to pretending that tweeting at politicians does anything and everyone there is desperately trying to build a “brand” as the funny one or whatever. I want news I don’t want your endless meta commentary on the news. People talk a lot about the protocols that power Bluesky vs. ActivityPub, because we're nerds and we believe deep in our hearts that the superior protocol will win. This is adorable. It flies in the face of literally all of human history, where the more convenient thing always wins regardless of technical merit. VHS beat Betamax. USB-C took twenty years. The protocol fight is interesting the way medieval siege warfare is interesting — I'm glad someone's into it, but it has no bearing on my life. There's no actual plan to self-host Bluesky. Their protocol makes it easier to scale their service. That's why it was written and that's what it does. End of story. Now EU news remained reliable, but sending European reporters into the madness of the US and trying to get a “report” out of it is an exercise in frustration. This became especially relevant for me when Trump threatened to invade Greenland and suddenly there was a distinct possibility that there might be an armed conflict between Denmark and the US. Danish reporters weren’t getting meetings with the right people and it was just endless rumors and Truth Social nonsense. If the American press had given me 20 minutes of airtime I could have convinced everyone they don’t want to get involved with Greenland. We’re not tough enough as a people to survive in Greenland, much less “take it over”. Greenlandic people shrug off horrific injuries hundreds of kilometers from medical help with a smile. I watched a Greenlandic toddler munch meat from the spine of a seal with its head very much intact. We aren’t equipped to fuck with these people, they are the real deal. So in this complete breakdown of the press came in the Fediverse. It became the only reliable source of information I had. People posted links with a minimal amount of commentary, picking and choosing the best content from other social media networks. They’re not doing it to “build a brand” because that’s not a thing in the Fediverse. It’s too disjointed to be a place to build a newsletter subscription base. Instead it became the only place consistently posting trustworthy information I could actually access. This became personally relevant when Trump threatened to invade Greenland, which is the kind of sentence I never expected to type and yet here we are. It would be funny if I wasn't a tiny bit concerned that my new home was going to get a CIA overnight regime change special in the middle of the night. It was somewhere in the middle of DMing with someone who had forgotten more about Greenland than I would ever know and someone who lived close to an RAF base in the UK that it clicked. This was what they had been talking about. Actual human beings were able to find each other and ask direct questions without this giant mountain of bullshit engagement piled on top of it. Meta or Oracle or whoever owns TikTok this week couldn't stop me. I never expected to find my news from strangers on a federated social network that half the internet has never heard of. I never expected a lot of things. But there's something quietly beautiful about a place where people just... share what they know. No brand deals, no engagement metrics, no algorithm nudging you toward rage. Just someone who spent twenty years studying Arctic policy posting a thread at 2 AM because they think you should understand what's happening. It's the internet I was promised in 1996. It only took thirty years and the complete collapse of American journalism to get here. Find me at: https://c.im/@matdevdug

Media Culture Journalism

▲ 0

0 views

matduggan.com 5 months ago

I Sold Out for $20 a Month and All I Got Was This Perfectly Generated Terraform

Until recently the LLM tools I’ve tried have been, to be frank, worthless. Copilot was best at writing extremely verbose comments. Gemini would turn a 200 line script into a 700 line collection of gibberish. It was easy for me to, more or less, ignore LLMs for being the same over-hyped nonsense as the Metaverse and NFTs. This is great for me because I understand that LLMs represent a massive shift in power from an already weakened worker class to an increasingly monarch-level wealthy class. By stealing all human knowledge and paying nothing for it, then selling the output of that knowledge, LLMs are an impossibly unethical tool. So if the energy wasting tool of the tech executive class is also a terrible tool, easy choice. Like boycotting Tesla for being owned by an evil person and also being crappy overpriced cars, or not shopping at Hobby Lobby and just buying directly from their Chinese suppliers, the best boycotts are ones where you aren’t really losing much. Google can continue to choke out independent websites with their AI results that aren’t very good and I get to feel superior doing what I was going to do anyway by not using Google search. This logic was all super straight forward right up until I tried Claude Code. Then it all got much more complicated. Let’s just get this out of the way right off the bat. I didn't want to like Claude Code. I got a subscription with the purpose of writing a review on it where I would find that it was just as terrible as Gemini and Copilot. Except that's not what happened. Instead it was like discovering the 2AM kebab place might actually make the best pizza in town. I kept asking Claude to do annoying tasks where it was easy for me to tell if it had made a mistake and it kept doing them correctly. It felt impossible but the proof was right in front of me. I’ve written tens of thousands of lines of Terraform in my life. It is a miserable chore to endlessly flip back and forth between the provider documentation and Vim, adding all the required parameters. I don’t learn anything by doing it, it’s just a grind I have to push through to get back to the meaningful work. The amount of time I have wasted on this precious time on Earth importing all of a companies DNS records into Terraform, then taking the autogenerated names and organizing them so that they make sense for the business is difficult to express. It's like if the only way I knew how to make a hamburger bun was to carefully put every sesame seed by hand on the top only to stumble upon an 8 pack of buns for $4 at the grocery store after years of using tiny tweezers to put the seeds in exactly the right spot. I feel the same way about writing robust READMEs, k8s YAML and reorganizing the file structure of projects. Setting up more GitHub Actions is as much fun as doing my taxes. If I never had to write another regex for the rest of my life, that would be a better life by every conceivable measure. These are tasks that sap my enthusiasm for this type of work, not feed it. I’m not sad to offload them and switch to mostly reviewing its PRs. But the tool being useful doesn’t remove what’s bad about it. This is where a lot of pro-LLM people start to delude themselves. In no particular order are the arguments I keep seeing about LLMs from people who want to keep using them for why their use is fine. This is the most common one I see and the worst. It can be condensed down to “because most things on the internet originally existed to find pornography and/or pirate movies, stealing all content on the internet is actually fine because programmers don’t care about copyright”. You also can’t have it both ways. OpenAI can’t decide to enforce NDAs and trademarks and then also declare law is meaningless. If I don’t get to launch a webmail service named Gmail+ then Google doesn’t get to steal all the books in human existence. The argument basically boils down to: because we all pirated music in 2004, intellectual property is a fiction when it stands in the way of technology. By this logic I shoplifted a Snickers bar when I was 12 so property rights don't exist and I should be allowed to live in your house. I have an internet friend I met years ago playing EVE Online that is a brutally pragmatic person. To someone like him, code craftsmanship is a joke. For those of you who are unaware, EVE Online is the spaceship videogame where sociopaths spend months plotting against each other. His approach to development is 80% refining requirements and getting feedback. He doesn’t care at all about DRY, he uses Node because then he can focus on just JavaScript, he doesn’t invest a second into optimization until the application hits a hard wall that absolutely requires it. His biggest source of clients? Creating fast full stacks because internal teams are missing deadlines. And he is booked up for at least 12 months out all the time because he hits deadlines. When he started freelancing I thought he was crazy. Who was going to hire this band of Eastern European programmers who chain smoke during calls and whose motto is basically "we never miss a deadline". As it turns out, a lot of people. Why doesn't he care? Why doesn't he care about these things? He believes that programmers fundamentally don't understand the business they are in. "Code is perishable" is something he says a lot and he means it. Most of the things we all associate with quality (full test coverage, dependency management, etc) are programmers not understanding the rate of churn a project undergoes over its lifespan. The job of a programmer, according to him, is delivering features that people will use. How pleasant and well-organized that code is to work with is not really a thing that matters in the long term. He doesn't see LLM-generated code as a problem because he's not building software with a vision that it will still be used in 10 years. Most of the stuff typically associated with quality he, more or less, throws in the trash. He built a pretty large stack for a automotive company and my jaw must have hit the table when he revealed they're deploying m6g.4xlarge for a NodeJS full-stack application. "That seems large to me for that type of application" was my response. He was like "yeah but all I care about are whether the user metrics show high success rate and high performance for the clients". It's $7000 a year for the servers, with two behind a load balancer. That's absolutely nothing when compared with the costs of what having a team of engineers tune it would cost and it means he can run laps around the internal teams who are, basically, his greatest competition. To be clear, he is very technically competent. He simply rejects a lot of the conventional wisdom out there about what one has to do in order to make stuff. He focuses on features, then securing endpoints and more or less gives up on the rest of it. For someone like this, LLMs are a logical choice for him. The annoying thing about my friend is that his bank account suggests he's right. But I can't get there. If I'm writing a simple script or something as a one-off, it can sometimes feel like we're all wasting the companies time when we have a long back and forth on the PR discussing comments or the linting or whatever. So it's not that this idea is entirely wrong . But the problem with programming is you never know what is going to be "the core" of your work life for the next 5 years. Sometimes I write a feature, we push it out, it explodes in popularity and then I'm a little bit in trouble because I built a MVP and now it's a load-bearing revenue generating thing that has to be retooled. I also just have trouble with the idea that this is my career and the thing I spend my limited time on earth doing and the quality of it doesn't matter. I delight in craftsmanship when I encounter it in almost any discipline. I love it when you walk into an old house and see all the hand crafted details everywhere that don't make economic sense but still look beautiful. I adore when someone has carefully selected the perfect font to match something. Every programmer has that library or tool that they aspire to. That code base where you delight at looking at it because it proves perfection is possible even if you have never come close to reaching that level. For me its always been looking through the source code of SQLite that restores my confidence. I might not know what I'm doing but it's good to be reminded that someone out there does. Not everything I make is that great, but the concept of "well great doesn't matter at all" effectively boils down to "don't take pride in your work" which is probably the better economic argument but feels super bad to me. In a world full of cheap crap, it feels bad to make more of it and then stick my name on it. The best argument for why programmers should be using LLMs is because it's going to be increasingly difficult to compete for jobs and promotions against people who are using them. In my experience Claude Code allows me to do two tasks at once. That's a pretty hard advantage to overcome. Last Tuesday I had Claude Code write a GitHub Action for me while I worked on something else. When it was done, I reviewed it, approved it, and merged it. It was fine. It was better than fine, actually — it was exactly what I would have written, minus the forty-five minutes of resentment. I sat there for a moment, staring at the merged PR, feeling the way I imagine people feel when they hire a cleaning service for the first time: relieved, and then immediately guilty about the relief, and then annoyed at myself for feeling guilty about something that is, by any rational measure, a completely reasonable thing to do. Except it isn't reasonable. Or maybe it is. I genuinely don't know anymore, and that's the part that bothers me the most — not that the tool works, but that I've lost the clean certainty that it shouldn't. So now I'm paying $20 a month to a company that scraped the collective knowledge of humanity without asking so that I can avoid writing Kubernetes YAML. I know what that makes me. I just haven't figured out a word for it yet that I can live with. When I asked my EVE friend about it on a recent TeamSpeak session, he was quiet for awhile. I thought that maybe my moral dilemma had shocked him into silence. Then he said, "You know what the difference is between you and me? I know I'm a mercenary. You thought you were an artist. We're both guys who type for money." I couldn't think of a clever response to that. I still can't.

JavaScript AI Programming Terraform Yaml

▲ 0

0 views

matduggan.com 5 months ago

The Small Web is Tricky to Find

One of the most common requests I've gotten from users of my little Firefox extension( https://timewasterpro.xyz ) has been more options around the categories of websites that you get returned. This required me to go through and parse the website information to attempt to put them into different categories. I tried a bunch of different approaches but ended up basically looking at the websites themselves seeing if there was anything that looked like a tag or a hint on each site. This is the end conclusion of my effort at putting stuff into categories. Unknown just means I wasn't able to get any sort of data about it. This is the result of me combining Ghost, Wordpress and Kagi Small Web data sources. Interestingly one of my most common requests is "I would like less technical content" which as it turns out is tricky to provide because it's pretty hard to find. They sorta exist but for less technical users they don't seem to have bought into the value of the small web own your own web domain (or if they have, I haven't been able to figure out a reliable way to find them). This is an interesting problem, especially because a lot of the tools I would have previously used to solve this problem are....basically broken. It's difficult for me to really use Google web search to find anything at this point even remotely like "give me all the small websites" because everything is weighted to steer me away from that towards Reddit. So anything that might be a little niche is tricky to figure out. So there's no point in building a web extension with a weighting algorithm to return less technical content if I cannot find a big enough pool of non-technical content to surface. It isn't that these sites don't exist its just that we never really figured out a way to reliably surface "what is a small website". So from a technical perspective I have a bunch of problems. I think I can solve....some of these, but the more I work on the problem the more I'm realizing that the entire concept of "the small web" had a series of pretty serious problems. First I need to reliably sort websites into a genre, which can be a challenge when we're talking about small websites because people typically write about whatever moves them that day. Most of the content on a site might be technical, but some of it might not be. Big sites tend to be more precise with their SEO settings but small sites that don't care don't do that, so I have fewer reliable signals to work with. Then I need to come up with a lot of different feeding systems for independent websites. The Kagi Small Web was a good starting point, but Wordpress and Ghost websites have a much higher ratio of non-technical content. I need those sites, but it's hard to find a big batch of them reliably. Once I have the type of website as a general genre and I have a series of locations, then I can start to reliably distribute the types of content you get. Google was the only place on Earth sending any traffic there Because Google was the only one who knew about it, there never needed to be another distribution system Now that Google is broken, it's almost impossible to recreate that magic of becoming the top of list for a specific subgenre without a ton more information than I can get from public records.

Web Development Data Web

▲ 0

0 views

matduggan.com 5 months ago

GitButler CLI Is Really Good

My workflow has remained mostly the same for over a decade. I write everything in Vim using the configuration found here . I run Vim from inside of tmux with a configuration found here . I write things on a git branch, made with the CLI, then I add them with to that branch, trying to run all of the possible linting and tests with before I waste my time on GitHub Actions. Then I run which is an alias to . Finally I successfully commit, then I copy paste the URL returned by GitHub to open a PR. Then I merge the PR and run to go back to the primary branch, which is an alias to . This workflow, I think, is pretty familiar for anyone working with GitHub a lot. Now you'll notice I'm not saying because almost nothing I'm doing has anything to do with . There's no advantage to my repo being local to my machine, because everything I need to actually merge and deploy code lives on GitHub. The CI runs there, the approval process runs there, the monitoring of the CI happens there, the injection of secrets happens there. If GitHub is down my local repo does, effectively, nothing. My source of truth is always remote, which means I pay the price for complexity locally but I don't benefit from it. At most jobs: This means the following is also true: Almost all the features of are wasted on me in this flow. Now because this tool serves a million purposes and is designed to operate in a way that almost nobody uses it for, we all pay the complexity price of and never reap any of the benefits. So instead I keep having to add more aliases to paper over the shortcomings of . These are all the aliases I use at least once a week. Git's offline-first design creates friction for online-first workflows, and GitButler CLI eliminates that friction by being honest about how we actually work. (Edit: I forgot to add this disclaimer. I am not, nor have ever been an employee/investor/best friends with anyone from GitButler. They don't care that I've written this and I didn't communicate with anyone from that team before I wrote this.) So let's take the most basic command as an example. This is my flow that I do 2-3 times a day without my aliases. I do this because can't make assumptions about the state of the world. However because GitButler is designed with the assumption that I'm working online, we can skip a lot of this nonsense. It's status command understands that there is always a remote main that I care about and that when I run a status that I need to understand my status relative to the remote main as it exists right now. Not how it existed the last time I remembered to pull. However this is far from the best trick it has up its sleeve. You're working on a feature, notice an unrelated bug, and now you have to stash, checkout, fix, commit, push, checkout back, stash pop. Context switching is expensive and error-prone. GitButler effectively hacks a solution into that fixes this with multiple branches applied simultaneously. Assign files to different branches without leaving your workspace. What do I mean by that. Let's start again with my status Great looks good. Alright so lets say I make 2 new branches. I'm working on a new feature for adding auth and while I'm working on that, I see a typo I need to fix in a YAML. I can work on both things at the same time: And easily commit to both at the same time without doing anything weird . Stacked PRs are the "right" way to break up large changes so people on your team don't throw up at being asked to review 2000 lines, but Git makes them miserable. When the base branch gets feedback, you have to rebase every dependent branch, resolve conflicts, force-push, and pray. Git doesn't understand branch dependencies. It treats every branch as independent, so you have to manually maintain the stack. GitButler solves this problem with First-class stacked branches. The dependency is explicit, and updates propagate automatically. So what do I mean. Let's say I make a new API endpoint in some Django app. First I make the branch. So let's say I'm working on the branch and get some good feedback on my PR. It's easy to resolve the comments there while leaving my branched off this as a stacked thing that understands the relationship back to the first branch as shown here. In practice this is just a much nicer way of dealing with a super common workflow. Maybe the most requested feature from new users I encounter is an easier undo. When you mess up in Git, recovery means diving into , understanding the cryptic output, and hoping you pick the right . One wrong move and you've made it worse. GitButlers is just easier to use. So the basic undo functionality is super simple to understand. rolls me back one operation. To me the mental model of a snapshot makes a lot more sense than the git history model. I do an action, I want to undo that action. This is better than the git option of: I've been using GitButler in my daily work since I got the email that the CLI was available and I've really loved it. I'm a huge fan of what this team is doing to effectively remodel and simplify Git operations in a world where almost nobody is using it in the way the tool was originally imagined to be used. I strongly encourage folks go check it out for free at: https://docs.gitbutler.com/cli-guides/cli-tutorial/tutorial-overview . It does a ton of things (like help you manage PRs) that I didn't even touch on here. Let me know if you find something cool that I forgot at: https://c.im/@matdevdug You can't merge without GitHub (PRs are the merge mechanism) You can't deploy without GitHub (Actions is the deployment trigger) You can't get approval without GitHub (code review lives there) Your commits are essentially "drafts" until they exist on GitHub You never work disconnected intentionally You don't use local branches as long-lived divergent histories You don't merge locally between branches (GitHub PRs handle this) You don't use for archaeology — you use GitHub's blame/history UI (I often use git log personally but I have determined I'm in the minority on this). Your local repo might be offline for days or weeks The "remote" might be someone else's laptop, not a central server Divergent histories are expected and merging is a deliberate, considered act

Python Yaml Git

▲ 0

0 views

matduggan.com 6 months ago

The Year of the 3D Printed Miniature (And Other Lies We Tell Ourselves)

One amusing thing about following tech news is how often the tech community makes a bold prediction or assertion, only to ultimately be completely wrong. This isn't amusing in a "ha ha, we all make mistakes" kind of way. It's amusing in the way that watching someone confidently stride into a glass door is amusing. You feel bad, but also, they really should have seen that coming. Be it VR headsets that would definitely replace reality by 2018, or self-driving cars in every driveway "within five years" (a prediction that has been made every five years since 2012), we have a remarkable talent for making assumptions about what consumers will like and value without having spent a single goddamn minute listening to those same consumers. It's like a restaurant critic reviewing a steakhouse based entirely on the menu font. So when a friend asked me what I thought about "insert new revolutionary technology that will change everything" this week, my brain immediately jumped to "it'll be like 3D printers and Warhammer." This comparison made sense in the moment, as we were currently playing a game of Warhammer 40,000, surrounded by tiny plastic soldiers and the faint musk of regret. But I think, after considering it later, it might make sense for more people as well—a useful exercise in tech enthusiasm versus real user wants and needs. Or, put another way: a cautionary tale about people who have never touched grass telling grass-touchers how grass will work in the future. One long-held belief among tech bros has been the absolute confidence that 3D printers would, at some point, disrupt . Exactly what they would disrupt wasn't 100% clear. Disruption, in Silicon Valley parlance, is less a specific outcome and more a vibe—a feeling that something old and profitable will soon be replaced by something new and unprofitable that will somehow make everyone rich. A common example trotted out was one of my favorite hobbies: tabletop wargaming. More specifically, the titan of the industry, Warhammer 40,000. Every time a new 3D printer startup graced the front page of Hacker News, this proclamation would echo from the comments section like a prophecy from a very boring oracle: "This will destroy Games Workshop." Reader, it has not destroyed Games Workshop. Games Workshop is doing fine. Games Workshop will be selling overpriced plastic crack to emotionally vulnerable adults long after the sun has consumed the Earth. For those who had friends in high school—and I'm not being glib here, this is a genuine demographic distinction—40k is a game where two or more players invest roughly $1,000 to build an army of small plastic figures. You then trim excess plastic with a craft knife (cutting yourself at least twice, this is mandatory), prime them, paint them over the course of several months, and then carefully transport them to an LGS (local game shop) in foam-lined cases that cost more than some people's luggage. Another fellow dork will then play you on a game board roughly the size of a door, covered in fake terrain that someone spent 40 hours making to look like a bombed-out cathedral. You will both have rulebooks with you containing as many pages as the Bible and roughly as open to interpretation. Wars have been started over less contentious texts. To put 40k in some sort of nerd hierarchy, imagine a game shop. At the ground level of this imaginary shop are Magic: The Gathering and Pokémon TCG games. Yes, these things are nerdy, but it's not that deep into the swamp. It's more of a gentle wade. You start with Pokémon at age 10, burn your first Tool CD at 14, and then sell your binder of 'mons to fund your Magic habit. This is the natural order of things. Deeper into the depths, maybe only playing at night like creatures who have evolved beyond the need for vitamin D, are your TTRPGs (tabletop RPGs). The titan of the industry is Dungeons & Dragons, but there is always some new hotness nipping at its heels, designed by someone who thought D&D wasn't quite complicated enough. TTRPGs are cheap to attempt to disrupt—you basically need "a book"—so there are always people trying. These are the folks with thick binders, sacks of fancy dice made from materials that should not be made into dice, and opinions about "narrative agency." Near the bottom, almost always in the literal basement of said shop, are the wargame community. We are the Morlocks of this particular H.G. Wells situation. I, like a lot of people, discovered 40k at a dark time in my life. My college girlfriend had cheated on me, and I had decided to have a complete mental breakdown over this failed relationship that was doomed well before this event. The cheating was less a cause and more a symptom, like finding mold on bread that was already stale. Honestly, in retrospect, hard to blame her. I was being difficult. I was the kind of difficult where your friends start sentences with "Look, I love you, but..." Late at night, I happened to be driving my lime green Ford Probe past my local game shop. The Ford Probe, for those unfamiliar, was a car designed by someone who had heard of cars but had never actually seen one. It was the automotive equivalent of a transitional fossil. I loved it the way you love something that confirms your worst suspicions about yourself. There, through the shop window, I saw people hauling some of the strangest items out of their trunks. Half-destroyed buildings. Thousands of tiny little figures. Giant robots the size of a small cat with skulls for heads. One man was carrying what appeared to be a ruined spaceship made entirely of foam and spite. I pulled over immediately. The owner, who knew me from playing Magic, seemed neither surprised nor pleased to see me. This was his default state. Running a game shop for 20 years will do that to a person. "They're in the basement," he said, in the mostly dark game shop, the way someone might say "the body's in the basement" in a very different kind of establishment. I descended the rickety wooden stairs to a large basement lit by three naked bulbs hanging from cords. The aesthetic was "serial killer's workspace" meets "your uncle's unfinished renovation project." It was perfect. Before me were maybe a dozen tables littered with plastic. Some armies had many bug-like things, chitinous and horrible. Others featured little skeletons or robots. There were tape measures everywhere and people throwing literal handfuls of small six-sided dice at the table with the intensity of gamblers who had nothing left to lose. Arguments broke out over millimeters. Someone was consulting a rulebook with the desperation of a lawyer looking for a loophole. I was hooked immediately. 40k is the monster of wargaming specifically because of a few genius decisions by Games Workshop, the creators—a British company that has somehow figured out how to print money by selling plastic and lore about a fascist theocracy in space. It's a remarkable business model. Since the beginning of the game, 40k casual games have allowed proxies. Proxies are stand-ins for specific units that you need for an army but don't have. Why don't you have them? Excellent question. Let me tell you about Games Workshop's relationship with its customers. Games Workshop has always played a lot of games with inventory. Often releases will have limited supply, or there are weird games with not fulfilling the entire order that a game shop might make. Even when they switched from metal to plastic miniatures, the issues persisted. This has been the source of conspiracy theories since the very beginning—whispers of artificial scarcity, of deliberate shortages designed to create FOMO among people who were already deeply susceptible to FOMO because they collect tiny plastic soldiers. Whether the conspiracy theories are true is almost beside the point. The feeling of scarcity is real, and feelings, as any therapist will tell you, are valid. Even the stupid ones. So players had proxies. Anything from a Coke can to another unit entirely. Basically, if it had the same size base and roughly the same height, most people would consider it allowable. "This empty Red Bull can is my Dreadnought." Sure. Fine. We've all been there. This is where I first started to see 3D-printed miniatures enter the scene. Similar to most early tech products, the first FDM 3D-printed miniatures I saw were horrible. The thick, rough edges and visible layer lines were not really comparable to the professional product, even from arm's length. They looked like someone had described a Space Marine to a printer that was also drunk. But they were totally usable as a proxy and better than a Coke can. The bar, as they say, was low. But the technology continued to get better and cheaper and, as predicted by tech people, I started to notice more and more interest in 3D printing among people at the game stores. When I first encountered a resin 3D-printed army at the table, I'll admit I was intrigued. This person had basically fabricated $3,000 worth of hard-to-get miniatures out of thin air and spite. This was supposed to be the big jumping-off point. The inflection moment. There were a lot of discussions at the table about how soon we wouldn't even have game shops with inventory! They'd be banks of 3D printers that we would all effortlessly use to make all the minis we wanted! The future was here, and it smelled like resin fumes! Printing a bunch of miniatures off a resin 3D printer quickly proved to have a lot of cracks in this utopian plan. Even a normal-sized mini took hours to print. That wouldn't be so bad, except these printers couldn't just live anywhere in your apartment. They're not like a Keurig. You can't just put them on your kitchen counter and forget about them. When I was invited to watch someone print off minis with a resin 3D printer, it reminded me a lot of the meth labs in my home state of Ohio. And I don't mean that as hyperbole. I mean there were chemicals, ventilation hoods, rubber gloves, and a general atmosphere of "if something goes wrong here, it's going to go very wrong." The guy giving me the tour had safety goggles pushed up on his forehead. He was wearing an apron. At one point, he said the phrase "you really don't want to get this on your skin" with the casual tone of someone who had definitely gotten it on his skin. In practice, the effort to get the STL files, add supports, wash off the models with isopropyl alcohol, remove supports without snapping off tiny arms, and finally cure the mini in UV lights was exponentially more effort than I'm willing to invest. And I say this as someone who has painted individual eyeballs on figures smaller than my thumb. I have a high tolerance for tedious bullshit. This exceeded it. Before I start, I first want to say I don't dislike the 3D printing community. I think it's great they're supporting smaller artists. I love that they found a hobby inside of a hobby, like those Russian nesting dolls but for people who were already too deep into something. I will gladly play against their proxy armies any day of the week. But people outside of the hobby proclaiming that this is the "future" are a classic example of how they don't understand why we're doing the activity in the first place. It's like watching someone who has never cooked explain how meal replacement shakes will eliminate restaurants. You're not wrong that it's technically more efficient. You're just missing the entire point of the experience. The reason why Games Workshop continues to have a great year after year—despite prices that would make a luxury goods executive blush, despite inventory issues, despite a rulebook that changes often enough to require a subscription service—is because of this fundamental misunderstanding. Players invest a lot of time and energy into an army. You paint them. You decorate the plastic bases with fake grass and tiny skulls. You learn their specific rules and how to use them. You develop opinions about which units are "good" and which are "trash" and you will defend these opinions with the fervor of a religious convert. Despite the eternal complaints about the availability of inventory, the practical reality is that most people can only keep a pipeline of one or maybe two armies going at once. The bottleneck isn't acquiring plastic. The bottleneck is everything else . So let's do the math on this. You buy a resin 3D printer. All the supplies. You get a spot in your house where you can safely operate it—which means either a garage, a well-ventilated spare room, or a relationship-ending negotiation with whoever you live with. You find or buy all the STLs you need. Let's say they all have supports in the files, so you just need to print them off. Best-case scenario. Let's say we break even around 50-75 infantry and a few larger models. This is over the raw cost of materials, but we need to factor in the space in your house it takes up, plus there's a learning curve with figuring out how to do it. You also need to invest a lot of time getting these files for printing and finding the good ones. For the sake of keeping this simple, let's just assume the actual printing process goes awesome. No failed prints. No supports that fuse to the model. No discovering that your file was corrupted after six hours of printing. Fantasy land. Here's the thing: getting the raw plastic minis is not the time-consuming part. First, you need to paint them. I take about two hours to paint each model, and I'm far from the best painter out there. I'm solidly in the "looks good from three feet away" category, which is also how I'd describe my general appearance. Vehicles take longer because they're bigger—maybe 10-20 hours for one of those. We're talking somewhere in the ballpark of 150 hours to paint everything that you need to paint for a standard army. Now don't get me wrong, I love painting. But I'm a 38-year-old with a child and a full-time job. Finding 150 hours for anything that isn't work, childcare, or sleep requires the kind of calendar Tetris that would make a project manager weep. It is a massive investment of time to get an army on the table, even if you remove the financial element of buying the minis entirely. Frankly, the money I pay to Games Workshop is the easiest part of the entire process. Often the box will be lovingly stacked on top of other sealed mini boxes—a pile of shame, we call it—until I can start the process of even hoping to catch up. I have boxes I bought during the Obama administration. They're still sealed. They judge me. But okay, let's say we get them all painted. What's next? Next comes "learn how the army works." There is a ton of flexibility to each army in 40k and how they work and operate. It takes a bit of research and time to figure out what they all do, which is something you are 100% expected to know cover to cover when you show up to play. It's not my job to know what your army can and cannot do. If you show up not knowing your own rules, you will be eaten alive, and you will deserve it. So what I saw with the 3D printing crowd felt a lot like the "Year of the Linux Desktop" crowd. Every year they would proclaim that soon we'd all get on board with their vision. They would print off an incredibly impressive army with all the hard-to-find minis that were sold once at a convention in 1997. They'd get the army "painted" to some definition of painted—and I'm using those quotation marks with malice—get on the table, and then play effectively that one army the same as the rest of us. The printer didn't give them more time. It didn't give them more skill. It just gave them more unpainted plastic, which, brother, I have plenty of already. For those in the 3D printing crowd who weren't big into playing, just painting, part of the point is showing off your incredible work to everyone else. Except nobody wants to see a 3D-printed forgery of an official model. It's like showing up to a car show with a kit car that looks like a Ferrari. Sure, it's impressive in its own way, but it's not really a Ferrari, and everyone knows it, and now we're all standing around pretending we don't know it, and it's uncomfortable for everyone. Once someone figured out one of your minis was 3D printed, shops generally wouldn't feature it in their display cases. So there was no reason for people who were going to put in 10+ hours per model to skip paying for the official real models. If you're going to invest that much time, you want the real thing. You want the little Games Workshop logo on the base. You want to be able to say "yes, I paid $60 for this single figure" with the quiet dignity of someone who has made peace with their choices. "Well then the shops can just sell the STLs and do the printing there!" This shows me you haven't spent a lot of time in these shops. Game shops need to carry a ton of inventory all the time, and a lot of their sales are impulse purchases. I see a mini I wouldn't typically be interested in, but it's done and ready, and I'm weak, and now I own it. That's the business model. They also operate on relatively thin margins—these aren't Apple Stores, they're labors of love run by people who got into this because they loved games and are now slowly being crushed by commercial rent and distributor minimums. It's just not feasible for them to print minis on demand and have enough staff to keep an eye on all the printing. Plus, tabletop wargaming isn't their major revenue generator anyway—it's card games like Pokémon and Magic. The wargamers in the basement are a bonus, not the main attraction. We're the weird cousins who show up to Thanksgiving and everyone tolerates us because we're family. At the end of the day, the 3D printing proclamation that it would disrupt my hobby ended up being a whole lot of nothing. A series of reasonable mistakes were made by people enthusiastic about the technology, resulting in the current situation where every year is the year that all of this will get disrupted. Any day now. Just you wait. They looked at the price of miniatures and saw inefficiency. They looked at the scarcity and saw opportunity. What they didn't see was that the price and the scarcity were almost beside the point. The hobby isn't about acquiring plastic. The hobby is about what you do with the plastic after you acquire it. The hobby is about the 150 hours of painting. The hobby is about the arguments over rules interpretations. The hobby is about descending into a basement lit by three naked bulbs and finding your people. You can't 3D print that. So the next time someone tells you that some new technology is going to "disrupt" something you love, ask yourself: do they actually understand why people love it? Do they understand the irrational, inefficient, deeply human reasons people engage with this thing? Or are they just looking at a spreadsheet and seeing numbers that don't make sense to them? Because if it's the latter, you can probably ignore them. They'll be wrong. They're almost always wrong. In the meantime, you can find me in the basement, losing match after match, surrounded by tiny plastic soldiers I've spent hundreds of hours painting, playing a game that makes no sense to anyone who hasn't given themselves over to it completely. It's not efficient. It's not optimized. It's not disrupting anything. The game looks more complicated to play than it is. Especially now, in the 10th edition, the core rules don't take long to learn. However, there is a lot of depth to the individual options available to each army that take a while to master. So it hits that sweet spot of being fast to onboard someone onto while still providing frightening amounts of depth if you're the kind of person who finds "frightening amounts of depth" appealing rather than exhausting. I am that kind of person. This explains a lot about my life. The community is incredible. When I moved from Chicago to Denmark, it took me less than three days to find a local 40k game. Same thing when I moved from Michigan to Chicago. The age and popularity of the game means it is a built-in community that follows you basically around the world. Few other properties have this kind of stickiness. It's like being a Deadhead, except instead of following a band, you're following a shared delusion that tiny plastic men matter. They do matter. Shut up. Cool miniatures. They look nice. They're fun to paint and put together. They're complicated without being too annoying. This is the part that 3D printers are supposed to help with.

Gaming Business

▲ 0

0 views

matduggan.com 7 months ago

SQLite for a REST API Database?

When I wrote the backend for my Firefox time-wasting extension ( here ), I assumed I was going to be setting up Postgres. My setup is boilerplate and pretty boring, with everything running in Docker Compose for personal projects and then persistence happening in volumes. However when I was working with it locally, I obviously used SQLite since that's always the local option that I use. It's very easy to work with, nice to back up and move around and in general is a pleasure to work with. As I was setting up the launch, I realized I really didn't want to set up a database. There's nothing wrong with having a Postgres container running, but I'd like to skip it if its possible. So my limited understanding of SQLite before I started this was "you can have one writer and many readers". I had vaguely heard of SQLite "WAL" but my understanding of WAL is more in the context of shipping WAL between database servers. You have one primary, many readers, you ship WAL to from the primary to the readers and then you can promote a reader to the primary position once it has caught up on WAL. My first attempt at setting up SQLite for a REST API died immediately in exactly this way. So by default SQLite: This seems to be caused by SQLite having a rollback journal and using strict locking. Which makes perfect sense for the use-case that SQLite is typically used for, but I want to abuse that setup for something it is not typically used for. So after doing some Googling I ended up with these as the sort of "best recommended" options. I'm 95% sure I copy/pasted the entire block. What is this configuration doing. However my results from load testing sucked. Now this is under heavy load (simulating 1000 active users making a lot of requests at the same time, which is more than I've seen), but still this is pretty bad. The cause of it was, of course, my fault. My "blacklist" is mostly just sites that publish a ton of dead links. However I had the setup wrong and was making a database query per website to see if it matched the black list. Stupid mistake. Once I fixed that. Great! Or at least "good enough from an unstable home internet connection with some artificial packet loss randomly inserted". So should you use SQLite as the backend database for a FastAPI setup? Well it depends on how many users you are planning on having. Right now I can handle between 1000 and 2000 requests per second if they're mostly reads, which is exponentially more than I will need for years of running the service. If at some point in the future that no longer works, it's thankfully very easy to migrate off of SQLite onto something else. So yeah overall I'm pretty happy with it as a design. Only one writer at a time Writers block readers during transactions Switches SQLite from rollback journal to Write-Ahead Logging (WAL) Default behavior is Write -> Copy original data to journal -> Modify database -> Delete journal. WAL mode is Write -> Append changes to WAL file -> Periodically checkpoint to main DB So here you have 4 options to toggle for how often SQLite syncs to disk. OFF is SQlite lets the OS handle it. NORMAL is the SQLite engine still syncs, but less often than FULL. WAL mode is safe from corruption with NORMAL typically. FULL uses the Xsync method of the VFS (don't feel bad I've never heard of it before either: https://sqlite.org/vfs.html ) to ensure everything is written to disk before moving forward. EXTRA: I'm not 100% sure what this exactly does but it sounds extra. "EXTRA synchronous is like FULL with the addition that the directory containing a rollback journal is synced after that journal is unlinked to commit a transaction in DELETE mode. EXTRA provides additional durability if the commit is followed closely by a power loss. Without EXTRA, depending on the underlying filesystem, it is possible that a single transaction that commits right before a power loss might get rolled back upon reboot. The database will not go corrupt. But the last transaction might go missing, thus violating durability, if EXTRA is not set." = please wait up to 60 seconds. this one threw me for a loop. Why is it a negative number? If you set it to a positive number, you mean pages. SQLite page size is 4kb by default, so 2000 = 8MB. A negative number means KB which is easier to reason about than pages. I don't really know what a "good" cache_size is here. 64MB feels right given the kind of data I'm throwing around and how small it is, but this is guess work. = write to memory, not disk. Makes sense for speed.

Backend HTML Database

▲ 0

0 views

matduggan.com 7 months ago

Making RSS More Fun

I don't like RSS readers. I know, this is blasphemous especially on a website where I'm actively encouraging you to subscribe through RSS. As someone writing stuff, RSS is great for me. I don't have to think about it, the requests are pretty light weight, I don't need to think about your personal data or what client you are using. So as a protocol RSS is great, no notes. However as something I'm going to consume, it's frankly a giant chore . I feel pressured by RSS readers, where there is this endlessly growing backlog of things I haven't read. I rarely want to read all of a websites content from beginning to end, instead I like to jump between them. I also don't really care if the content is chronological, like an old post about something interesting isn't less compelling to me than a newer post. What I want, as a user experience, is something akin to TikTok. The whole appeal of TikTok, for those who haven't wasted hours of their lives on it, is that I get served content based on an algorithm that determines what I might think is useful or fun. However what I would like is to go through content from random small websites. I want to sit somewhere and passively consume random small creators content, then upvote some of that content and the service should show that more often to other users. That's it. No advertising, no collecting tons of user data about me, just a very simple "I have 15 minutes to kill before the next meeting, show me some random stuff." In this case the "algorithm" is pretty simple: if more people like a thing, more people see it. But with Google on its way to replacing search results with LLM generated content, I just wanted to have something that let me play around with the small web the way that I used to. There actually used to be a service like this called StumbleUpon which was more focused on pushing users towards popular sites. It has been taken down, presumably because there was no money in a browser plugin that sent users to other websites whose advertising you didn't control. You can go download the Firefox extension now and try this out and skip the rest of this if you want. https://timewasterpro.xyz/ If you hate it or find problems, let me know on Mastodon. https://c.im/@matdevdug So I wanted to do something pretty basic. You hit a button, get served a new website. If you like the website, upvote it, otherwise downvote it. If you think it has objectionable content then hit report. You have to make an account (because I couldn't think of another way to do it) and then if you submit links and other people like it, you climb a Leaderboard. On the backend I want to (very slowly so I don't cost anyone a bunch of money) crawl a bunch of RSS feeds, stick the pages in a database and then serve them up to users. Then I want to track what sites get upvotes and return those more often to other users so that "high quality" content shows up more often. "High quality" would be defined by the community or just me if I'm the only user. It's pretty basic stuff, most of it copied from tutorials scattered around the Internet. However I really want to drive home to users that this is not a Serious Thing. I'm not a company, this isn't a new social media network, there are no plans to "grow" this concept beyond the original idea unless people smarter than me ping with me ideas. So I found this amazing CSS library: https://sakofchit.github.io/system.css/ The Apple's System OS design from the late-80s to the early 90s was one of my personal favorites and I think would send a strong signal to a user that this is not a professional, modern service. Great, the basic layout works. Let's move on! So I ended up doing FastAPI because it's very easy to write. I didn't want to spend a ton of time writing the API because I doubt I nailed the API design on the first round. I use sqlalchemy for the database. The basic API layout is as follows: The source for the RSS feeds came from the (very cool) Kagi small web Github. https://github.com/kagisearch/smallweb . Basically I assume that websites that have submitted their RSS feeds here are cool with me (very rarely) checking for new posts and adding them to my database. If you want the same thing as this does, but as an iFrame, that's the Kagi small web service. The scraping work is straightforward. We make a background worker, they grab 5 feeds every 600 seconds, they check for new content on each feed and then wait until the 600 seconds has elapsed to grab 5 more from the smallweb list of RSS feeds. Since we have a lot of feeds, this ends up look like we're checking for new content less than once a day which is the interval that I want. Then we write it out to a sqlite database and basically track "has this URL been reported", if so, put it into a review queue and then how many times this URL has been liked or disliked. I considered a "real" database but honestly sqlite is getting more and more scalable every day and its impossible to beat the immediate start up and functionality. Plus very easy to back up to encrypted object storage which is super nice for a hobby project where you might wipe the prod database at any moment. In terms of user onboarding I ended up doing the "make an account with an email, I send a link to verify the email". I actually hate this flow and I don't really want to know a users email. I never need to contact you and there's not a lot associated with your account, which makes this especially silly. I have a ton of email addresses and no real "purpose" in having them. I'd switch to Login with Apple, which is great from a security perspective but not everybody has an Apple ID. I also did a passkey version, which worked fine but the OSS passkey handling was pretty rough still and most people seem to be using a commercial service that handled the "do you have the passkey? Great, if not, fall back to email" flow. I don't really want to do a big commercial login service for a hobby application. Auth is a JWT, which actually was a pain and I regret doing it. I don't know why I keep reaching for JWTs, they're a bad user experience and I should stop. I'm more than happy to release the source code once I feel like the product is in a somewhat stable shape. I'm still ripping down and rewriting relatively large chunks of it as I find weird behavior I don't like or just decide to do things a different way. In the end it does seem to do whats on the label. We have over 600,000 individual pages indexed. Honestly I've been pretty pleased. But there are some problems. First I couldn't find a reliable way of switching the keyboard shortcuts to be Mac/Windows specific. I found some options for querying platform but they didn't seem to work, so I ended up just hardcoding them as Alt which is not great. The other issue is that when you are making an extension, you spend a long time working with these manifests.json. The specific part I really wasn't sure about was: I'm not entirely sure if that's all I'm doing? I think so from reading the docs. Anyway I built this mostly for me. I have no idea if anybody else will enjoy it. But if you are bored I encourage you to give it a try. It should be pretty light weight and straight-forward if you crack open the extension and look at it. I'm not loading any analytics into the extension so basically until people complain about it, I don't really know if its going well or not. admin - mostly just generating read-only reports of like "how many websites are there" leaderboard - So this is my first attempt at trying to get users involved. Submit a website that other people like? Get points, climb leaderboard. I need to sort stuff into categories so that you get more stuff in genres you like. I don't 100% know how to do that, maybe there is a way to scan a website to determine the "types" of content that is on there with machine learning? I'm still looking into it. There's a lot of junk in there. I think if we reach a certain number of downvotes I might put it into a special "queue". I want to ensure new users see the "best stuff" early on but there isn't enough data to determine "best vs worst". I wish there were more independent photography and science websites. Also more crafts. That's not really a "future thing", just me putting a hope out into the universe. Non-technical beta testers get overwhelmed by technical content.

Design CSS Web Development JSON

▲ 0

0 views

matduggan.com 9 months ago

I broke and fixed my Ghost blog

Once a month I will pull down the latest docker images for this server and update the site. The Ghost CMS team updates things at a pretty regular pace so I try to not let an update sit for too long. With this last round I suddenly found myself locked out of my Ghost admin panel. I was pretty confident that I hadn't forgotten my password and when I was looking at the logs, I saw this pretty spooky error. I was surprised by this sudden error, especially when I dumped out the database and confirmed that the hashed password for my Ghost user matched the password I was giving it. If you want to try that, this is the guide I followed: https://hostarmada.com/tutorials/blog-cms/ghost/how-to-change-the-admin-password-of-your-ghost-blog-if-you-get-locked-out/ So Ghost is a good CMS system, but it can be a little bit slow under load from automated scraping from RSS readers. I want to cache everything that I can with Nginx, so I use Nginx to store a lot of that junk. My configuration is not too terribly clever and has worked up to this point. The basic point is to get caching on the public content and then definitely NOT cache the ghost admin panel. After some testing, I confirmed this seemed to all work. But I was still locked out. Alright so I still couldn't figure out what was going on, so I went through the docs. Then I found this seemingly new addition. https://docs.ghost.org/config?_ga=2.92846045.1713439663.1760543217-1048546310.1760543217#security Now I have transactional email set up, but just looking at the error it seemed to feel related. So I added: to my docker-compose file to disable this new feature and then blamo, suddenly works fine. So if you are locked out of your Docker CMS admin panel, disable this (temporarily hopefully because it's a good feature) to let you continue to log in, debug your transactional email and then turn it back on. Hope that helps.

Tutorial DevOps

▲ 0

0 views

matduggan.com 9 months ago

Greenland is a beautiful nightmare

Greenland is a complicated topic here in Denmark. The former colony that is still treated a bit like a colony is something that inspires a lot of emotions. Greenland has been subjected to a lot of unethical experiments by Denmark, from taking their kids to wild experiments in criminal justice. But there is also a genuine pride a lot of people have here for the place and you run into Danes who grew up there more often than I would have guessed. When the idea of going to Greenland was introduced to me, I was curious. Having lived in Denmark for awhile, you hear a lot about the former colony and its 55,000 residents. We were invited by a family that my wife was close with growing up and is Danish. They wanted to take their father back to see the place he had spend some time in during his 20s and had left quite an impression. A few drinks in, I said "absolutely let's do it", not realizing we had already committed to going and I had missed the text message chain. A few weeks before I went, I realized "I don't know anything about Greenland" and started to watch some YouTube videos. It was about this time when I started to get a pit in my stomach, the "oh god I think I've made a huge mistake" feeling I'm painfully familiar with after a career in tech. Greenland appeared to have roughly 9 people living there and maybe 5 things to look at. Even professional travel personalities seemed to be scraping the bottom of the barrel. "There's the grocery store again!" they would point out as they slipped down the snowy roads. I couldn't tell any difference between different towns in the country. It reminded me a lot of driving through Indiana. For those not in the US, Indiana is a state in the US famous for being a state one must drive through in order to get somewhere better. If you live in Michigan, a good state and want to go to Illinois, another good state, one must pass through Indiana, a blank state. Because of this little strip here, you often found yourself passing through this place. Driving through Indiana isn't bad, it's just an empty void. It's like a time machine back to the 90s when people still smoke in restaurants but also there's nothing that sticks out about it. There is nothing distinct about Indiana, it's just a place full of people who got too tired on their way to somewhere better and decided "this is good enough". The difference is that Greenland is very hard to get to, as I was about to learn. Finally the day arrived. Me, my wife, daughter, 4 other children and 6 other adults all came to the Copenhagen Airport and held up a gate agent for what felt like an hour to slowly process all of our documents. Meanwhile, I nursed a creeping paranoia that I'd be treated as some sort of American spy, given my government's recent hobby of threatening to purchase entire countries like they're vintage motorcycles on Craigslist. The 5 hour flight is uneventful, the children are beautifully behaved and I begin to think "well this seems ok!" like the idiot I am. As I can look down and see the airport, the pilot comes on and informs us that there is too much fog to land safely. Surely fog cannot stop a modern aircraft full of all these dials and screens I think, foolishly. We are informed there is enough fuel to circle the airport for 5 hours to wait for the fog to lift. What followed was three hours of flying in lazy circles, like a very expensive, very slow merry-go-round. After the allotted time, we are informed that we must fly to Iceland to refuel and then we will be returning to Denmark . After a total of 15 hours in the air we will be going back to exactly where we started, to do the entire thing again. We were obviously upset at this turn of events, but I noticed the native Greenlandic folks seemed not surprised at this turn of events. As I later learned, this happens all the time . The native Greenlanders on board seemed utterly unsurprised by this development, displaying the kind of resigned familiarity that suggested this was Tuesday for them. I began wondering if I could just pretend Iceland was Greenland—surely my family wouldn't notice the difference? But the pilot, apparently reading my mind, announced that no one would be disembarking in Iceland. It felt oddly authoritarian, like being grounded by an airline, as if they knew we'd all just wander off into Reykjavik and call it close enough. We crash out in a airport hotel 20 minutes from our apartment after 15 hours in the air and tons of CO2 emissions only to wake up the next day to start again. This time, I notice that all of the people are asking for (and receiving) free beer from the crew that they are stashing in their bags. It turns out soda and beer, really anything that needs to be imported, is pretty expensive in Greenland. The complimentary drinks are there to be kept for later. Finally we land. The first thing you notice when you land in Greenland is there are no trees or grass. There is snow and then there is exposed rock. The exterior of the airport is metal but the inside is wood, which is strange because again there are no trees. This would end up being a theme, where buildings representing Denmark were made out of lots of wood, almost to ensure that you understood they weren't from here. We ended up piling all of our stuff into a bus and heading for the hotel in Nuuk. Nuuk is the capital of Greenland and your introduction to the incredible calm of the Greenlandic people. I have never met a less stressed out group of humans in my life. Nobody is really rushing anywhere, it's all pretty quiet and calm. The air is cold and crisp with lots of kids playing outside and just generally enjoying life. The city itself sits in a landscape so dramatically inhospitable it makes the surface of Mars look cozy. Walking through the local mall, half the shops sell gear designed to help you survive what appears to be the apocalypse. Yet somehow, there's traffic. Actual traffic jams in a place where you can walk from one end to the other in twenty minutes. It's like being stuck behind a school bus in your own driveway. To put this map into some perspective, it is only six kilometers from the sorta furthest tip to the airport. But riding the bus around Nuuk was a peaceful experience that lets you see pretty much the entire city without needing to book a tour or spend a lot of money. We went to Katuaq, a cultural center with a cafe and a movie theater that was absolutely delicious food. But again even riding the bus around it is impossible to escape the feeling that this is a fundamentally hostile to human life place. The sun is bright and during the summer its pretty hot, with my skin feeling like it was starting the burn pretty much the second it was exposed to the light. It's hard to even dress for, with layers of sunscreen, bug spray and then something warm on top if you suddenly got cold. The sun, meanwhile, has apparently forgotten how to set, turning our hotel rooms into solar ovens. You wake up in a pool of your own sweat, crack a window for relief, and immediately get hit with air so cold it feels personal. It's like being trapped in a meteorological mood swing. So after a night here, we went back to the airport again and flew to our final destination, Ilulissat. The flight to our final destination revealed Greenland's true nature: endless, empty hills stretching toward infinity, punctuated by ice formations that look like nature's sculpture garden. Landing in Ilulissat felt like victory—we'd made it to the actual destination, not just another waypoint in our Arctic odyssey. Walking through the tiny airport, past Danish military recruitment posters (apparently someone, somewhere, thought this place needed defending), I felt genuinely optimistic for the first time in days. Well you can sleep easy Danish military, because Ilulissat is completely protected from invasion. The second I stepped outside I was set upon by a flood of mosquitos like I have never experienced before. I have been to the jungles of Vietnam, the swamps of Florida and the Canadian countryside. This was beyond anything I've ever experienced. There are bugs in my mouth, ears, eyes and nose almost immediately. The photo below is not me being dramatic, it is actually what is required to keep them off of me. In fact what you need to purchase in order to walk around this area at all are basically bug nets for your face. They're effectively plastic mesh bags that you put on. Our hotel, charming in that "remote Arctic outpost" way, sat adjacent to what I can only describe as a canine correctional facility. Dozens of sled dogs were chained to rocks like some sort of prehistoric parking lot, each with a tiny house they could retreat to when the existential weight of their circumstances became too much. Now, I'd always imagined sled dogs living their best life—running through snow, tongues lolling, living the Disney version of Arctic life. I'd never really considered their downtime, assuming they frolicked in meadows or something equally wholesome. The reality was more "minimum security prison with a view." The dogs are visited roughly twice a day by the person who owns and feeds them, which was quite the party for the dogs that lost their minds whenever the car pulled up. Soon the kids really looked forward to dog feeding time. The fish scrapes the dogs lived on came out of a chest freezer that was left exposed up on the rock face without electricity and you could smell it from 50 yards away when it opened. During one such performance, a fellow parent leaned over and whispered with the casual tone of someone commenting on the weather, "I think that one is dead." Before I could process this information, the frozen canine was unceremoniously launched over a small cliff like a furry discus. A second doggy popsicle followed shortly after, right in front of our assembled children, who watched with the kind of wide-eyed fascination usually reserved for magic shows. We stopped making dog feeding time a group activity after that and had to distract the kids from ravens flying away with tufts of dog fur. Obviously a big part of Greenland is the nature, specifically the icebergs. Icebergs are incredible and during the week we spend up there, I enjoyed watching them every morning. It's like watching a mountain slowly moving while you sit still. The visual contrast of the ice and the exposed stone is beautiful and peaceful. Finding our tour operator proved to be an exercise in small-town efficiency. The man who gave me directions was the same person who picked us up from the airport, who was also our tour guide, who probably doubled as the mayor and local meteorologist. It was like a one-man civic operation disguised as multiple businesses—the ultimate small-town gig economy. The sea around Greenland is calmer than anything I've ever been on before, perfectly calm and serene. All around us whales emerged, thrilling my daughter. However the biggest hit of the entire tour, maybe the entire trip, was a member of the crew who handed each of the kids a giant rock of glacier ice to eat. I had to pull my daughter away to observe the natural beauty as she ate glacier ice like it was ice cream. "LOOK AT MY ICE" she was yelling as they slipped and slid around the deck of this boat. So if you've ever wonder "what is a glacier", let me tell you. Greenland has a lot of ice and it pushes out from the land that is covers into the sea. When that happens, a lot of it breaks off. This sounds more exciting than it is. On TV in 4K it looks incredible, giant mountains of ice falling into the ocean. Honestly you can go read the same thing I did here . However that doesn't happen very often. So in order for us tourists to be able to see anything, we had to go to a very productive glacier. This means there are constantly small chunks breaking off and falling into the sea. Practically though, it kinda looks like you are a boat in a slushee. It's beautiful and something to see, but also depressing to see along the rock face how much more ice there used to be. Back in town, we hopped on the "bus". Now the bus here is clearly a retrofitted party van, complete with blue LED lights. The payment system is zip tied to a desk chair that is, itself, wedged in the front. However the bus works well and does get you around. The confusing part is that you will, once again, sometimes encounter a lot of traffic. People are driving pretty quickly and really seem to have somewhere to go. You also see a lot of fancy cars parked outside of houses here. Which begs a pretty basic question. If there was almost nowhere to drive to in Nuuk, where in the hell are these people driving . The distance between the end of the road and the beginning of the road is less than 6 km. Also the process to make a road here is beyond anything you've ever seen. Everything requires a giant pile of explosives. Where did these vehicles even come from? Why does one ship a BMW to a place accessible only by plane and boat? More importantly, where was everyone going with such determination? It was like watching a very expensive version of bumper cars, except everyone was committed to the illusion that they had somewhere important to be. Everyone had dings and scrapes like crashes were common. Anyway, as I dodged speeding cars filled with people heading nowhere, I decided to hop off the bus and head to the grocery store. Inside was less a store and more the idea of a store. There was a lot of alcohol, chips, candy and shelf-stable foods, which all makes sense to me. What was strange was there wasn't a lot else, including meat. Locals couldn't be eating at the local restaurants, where the prices were as high as Berlin or Copenhagen for food. So what were they eating? When I asked one of my bus drivers, he told me that it was pretty unusual to buy meat. They purchased a lot of whale and seal meat. I had sorta heard this before, but when we stopped the bus he pointed out a group of men hauling guns out into a small boat to go shoot seals. The guns were held together with a surprising amount of duct tape, which is not something I associate with the wild. I had assumed, based on my casual reading of the news, that we were mostly done killing whales. As it turns out, I was wrong. They eat a lot of whale and it is, in fact, not hard to find. If you are curious, whale does not taste fishy. It tastes a little bit like if you cooked reindeer in a pot of seaweed. I wouldn't go out of your way for it, but it's not terrible. The argument I've always heard for why people still kill whales is because it's part of their culture and also because it's an important source of protein. When you hear the phrase "part of their culture" I always imagined like traditional boats going out with spears. What I didn't imagine was industrial fishing boats and an industrial crane that lifts the dead whale out of the water for "processing". Some of the illusion is broken when your boat tour guide points out the metal warehouse with the word "whale" on the side. "Yeah the water here was red with blood for a week" the guide said, counting the cigarettes left in a pack he had. It's a wild place unlike anywhere I've ever been. It is the closest I have ever felt to living a sci-fi type experience. The people of Greenland are amazing, tough, calm and kind. I have nothing but positive experiences to recount from the many people I met there, Danish and Greenlandic, who patiently sat through my millions of questions. However it is, by far, the least hospitable to human life place I've ever been to. The folks who live there have adapted to the situation in, frankly, genius ways. If that's your idea of a good time, Greenland is perfect for you. Maybe don't get emotionally attached to the sled dogs though. Or the whales.

Journalism Culture Go

▲ 0

0 views

matduggan.com 10 months ago

FYI: Broadcom is ruining Bitnami containers

For a long time Bitnami containers and Helm charts have been widely considered the easiest and fastest way to get reliable, latest versions of popular applications built following container best practices. They also have some of the better docs on the internet for figuring out how to configure all this stuff. However Broadcom, in their infinite capacity for short term gain over long term relationships, has decided to bring that to a close. On July 16th they informed their users that the platform was changing. Originally they were going to break a ton of workflows with only 43 days warning, but have expanded that out to a generous 75 days. It's impossible to read these timelines as anything other than Broadcom knows that enterprise customers won't be able to switch off in 43 or 75 days and is using this to extort people into paying them the rumored $50,000 a year to keep using the images. You can read the entire announcement here: https://github.com/bitnami/containers/issues/83267 Here is my summary though: TL;DR: Bitnami is significantly reducing their free container image offerings and moving most existing images to a legacy repository with no future updates. Free Community Tier (Severely Limited): Your Existing Images: Production Users: Before September 29th: Helm Charts: If you're using Bitnami for anything beyond basic development with latest tags, you'll need to either pay for Bitnami Secure Images or migrate to alternative container images before September 29th. Only a small subset of hardened images will remain free Available only with "latest" tags (no version pinning) Intended for development use only Find the limited selection at: https://hub.docker.com/u/bitnamisecure All current Bitnami images (including versioned tags) move to No updates, patches, or support for legacy images Use legacy repo only as temporary migration solution Need to subscribe to "Bitnami Secure Images" for continued support Includes security patches, LTS branches, and full version catalog Audit your deployments - Check which Bitnami images you're using Update CI/CD pipelines - Remove dependencies on deprecated images Choose your path: Development only: Migrate to the limited free tier (latest tags only) Production: Subscribe to Bitnami Secure Images or find alternatives Temporary fix: Update image references to (not recommended long-term) Source code remains open source on GitHub Existing OCI charts at won't receive updates Charts will fail unless you override image repositories

Open Source DevOps

▲ 0

0 views

matduggan.com 1 years ago

What Does a Post-Google Internet Look Like

With the rise of the internet came the need to find information more quickly. The concept of search engines came into this space to fill this need, with a relatively basic initial design. This is the basis of the giant megacorp Google, whose claim to fame was they made the best one of these. Into this stack they inject ads, both ads inside the sites themselves and then turning the search results themselves into ads. As time went on, what we understood to be "Google search" was actually a pretty sophisticated machine that effectively determined what websites lived or died. It was the only portal that niche websites had to get traffic. Google had the only userbase large enough for a website dedicated to retro gaming or VR headsets or whatever to get enough clicks to pay their bills. Despite the complexity, the basic premise remained. Google steers traffic towards your site, the user gets the answer from your site and then everyone is happy. Google showed some ads, you showed some ads, everyone showed everyone on Earth ads. This incredibly lucrative setup was not enough, however, to drive endless continous growth, which is now the new expectation of all tech companies. It is not enough to be fabulously profitable, you must become Weyland-Yutani. So now Google is going to break this long-standing agreement with the internet and move everything we understand to be "internet search" inside their silo. In March 2024 Google moved to embed LLM answers in their search results ( source ). The AI Overview takes the first 100 results from your search query, combines their answers and then returns what it thinks is the best answer. As expected, websites across the internet saw a drop in traffic from Google. You started to see a flood of smaller websites launch panic membership programs, sell off their sites, etc. It became clear that Google has decided to abandon the previous concept of how internet search worked, likely in the face of what it considers to be an existential threat from OpenAI. Maybe the plan was always to bring the entire search process in-house, maybe not, but OpenAI and its rise to fame seems to have forced Google's hand in this space. This is not a new thing, Google has been moving in this direction for years. It was a trend people noticed going back to 2019. It appears the future of Google Search is going to be a closed loop that looks like the following: This is all backed up by data from outside the Google ecosystem confirming that the ratio of scrapes to click is going up. Basically it's costing more for these services to make their content available to LLMs and they're getting less traffic from them. This new global strategy makes sense, especially in the context of the frequent Google layoffs. Previously it made strategic sense to hold onto all the talent they could, now it doesn't matter because the gates are closing. Even if you had all the ex-Google engineers money could buy, you can't make a better search engine because the concept is obsolete. Google has taken everything they need from the internet, it no longer requires the cooperation or goodwill of the people who produce that content. So the source of traffic for the internet is going to go away. My guess is there will be some effort to prevent this, some sort of alternative Google search either embraced or pushed by people. This is going to fail, because Google is an unregulated monopoly. Effectively because the US government is so bad at regulating companies and so corrupt with legalized bribery in the form of lobbying, you couldn't stop Google at this point even if you wanted to. While the US Department of Justice has finally decided to doing something, it's almost too late to make a difference. https://www.justice.gov/opa/pr/department-justice-prevails-landmark-antitrust-case-against-google Even if you wanted to and had a lot of money to throw at the problem, it's too late. If Apple made their own search engine and pointed iOS to it as the default and paid Firefox to make it the default, it still wouldn't matter. The AI Overview is a good enough answer for most questions and so convincing consumers to: I'm confident there will still be sites doing web searching, but I suspect given the explosion in AI generated slop it's going to be impossible to use them even if you wanted to. We're quickly reaching a point where it would be possible to generate a web page on demand, meaning the capacity of the slop-generation exceeds the capacity of humans to fight it. Because we didn't regulate the internet, we're going to end up with an unbreakable monopoly on all human knowledge held by Microsoft and Google. Then because we didn't learn anything we're going to end up with a system that can produce false data on demand and make it impossible to fact check anything that the LLM companies return. Paid services like Kogi will be the only search engines worth trying. So I think you are going to see a rush of shutdowns and paywalls like you've never seen before. In some respects, it is going to be a return to the pre-Google internet, where it will once again be important that consumers know your domain name and go directly to your site. It's going to be a massive consolidation of the internet down and I think the ad-based economy of the modern web will collapse. Google was the ad broker, but now they're going to operate like Meta and keep the entire cycle inside their system. My prediction is that this is going to basically destroy any small or medium sized business that attempts to survive with the model of "produce content, get paid per visitor through ads". Everything instead is going to get moved behind aggressive paywalls, blocking archive.org. You'll also see prices go way up for memberships. Access to raw, human produced information is going to be a premium product, not something for everyday people. Fake information will be free. Anyone attempting to make an online store is gonna get mob-style shakedown. You can either pay Amazon to let consumers see your product or you can pay Google to have their LLM recommend your product or you can (eventually) pay OpenAI/Microsoft to do it. I also think these companies will use this opportunity to dramatically reprice their advertising offerings. I don't think it'll be cheap to get the AI Summary to recommend your frying pan. I suspect there will be a brief spike in other forms of marketing spend, like podcasts, billboards, etc. When companies see the sticker shock from Google they're going to explore other avenues like social media spend, influencers, etc. But all those channels are going to be eaten by the LLM snake at the same time. If consumers are willing to engage with an LLM-generated influencer, that'll be the direction companies go in because they'll be cheaper and more reliable. Podcast search results are gonna be flooded with LLM-generated shows and my guess is that they're going to take more of the market share than anyone wants to admit. Twitch streaming has already moved from seeing the person to seeing an anime-style virtual overlay where you don't see the persons face. There won't be a reason for an actual human to be involved in that process. My prediction is that a lot of the places that employ technical people are going to disappear. FAANG isn't going to be hiring at anywhere near the same rate they were before, because they won't need to. I don't need 10,000 people maintaining relationships with ad sellers and ad buyers or any of the staff involved in the maintenance or improvement of those systems. The internet is going to return to more of its original roots, which are niche fan websites you largely find through social media or word of mouth. These sites aren't going to be ad driven, they'll be membership driven. Very few of them are going to survive. Subscription fatigue is a real thing and the math of "it costs a lot of money to pay people to write high quality content" isn't going to go away. In a relatively short period of time, it will go from "very difficult" to absolutely impossible to launch a new commercially viable website and have users organically discover that website. You'll have to block LLM scrapers and need a tremendous amount of money to get a new site bootstrapped. Welcome to the future, where asking a question costs $4.99 and you'll never be able to find out if the answer is right or not. Google LLM takes the information from the results it has already ingested to respond to most questions. Companies will at some point pay for their product or service to be "the answer" in different categories. Maybe this gets disclosed, maybe not, maybe there's just a little i in the corner that says "these answers may be influenced by marketing partners" or something. Google will attempt to reassure strategic partners that they aren't going to kill them, while at the same time turning to their relationship with Reddit to supply their "new data". Android is the dominant mobile platform on Earth Chrome is the dominant web browser Apple gets paid to make the other mobile platform default to Google Firefox gets paid to make the other web browser default to Google switch platforms and go back to a two/three/four step process compared to a one step process is a waste of time.

Business AI

▲ 0

0 views