Latest Posts (20 found)

Responsibility Is Taken Before It Is Given

To get promoted as a software engineer in Big Tech, you generally need to show “ownership” of a product or area. This means being responsible for it and deciding what should happen over time. I often hear more junior engineers ask: “How can I demonstrate ownership when my manager hasn’t given me anything to own?” This way of thinking is a trap: an easy one to fall into, but a trap nonetheless. The problem is that they are treating responsibility as a complete, self-contained opportunity that someone must give them first. In my experience, they’ve got it the wrong way round: responsibility is demonstrated before it is formally given. Put yourself in the shoes of your technical lead (TL): by “giving” you responsibility, they are implicitly accepting the consequences of your decisions. They need to trust you not only to get work done but to exercise judgement without constant supervision. The most important question they ask themselves is: “Does this person deal with problems the same way I would, or better ?” If the answer is yes, they can trust you with more responsibility. When I started working on Perfetto , I built its trace-analysis tooling mostly under my TL’s direction. As I began making implementation decisions myself, he would often spot major issues I had missed. For example, I built separate classes for three or four data structures I thought were unrelated. My TL insisted they were all tables, even though I couldn’t see how to unify them. Today, Perfetto’s trace processor has more than 100 tables, all declaratively defined and generated from that common abstraction. I always tried to understand his thought process: how had he even thought of something I had missed? What approach had he used to get there? Over time, I adopted his approaches myself, using them to pressure-test API changes or performance improvements before proposing them. More of my ideas started receiving a simple “yes, go ahead.” By then, I was setting the agenda myself: talking to users, understanding their problems, translating those into code changes and deciding what to improve next. Sometimes the dynamic even reversed: he would suggest something, and I would explain why it would not work. Eventually, my TL started calling me “the owner of the trace-analysis tools” instead of “the engineer who works on them.” To be clear, this does not mean grabbing projects or stepping on other people to demonstrate ownership. If you exercise sound judgement within your existing scope, a reasonable TL should gradually trust you with more responsibility. Of course, this does not work with a bad manager or in a bad environment, where earning responsibility may be more about politics than judgement. Formal responsibility is a trailing indicator, not a leading one. You’ll find that people begin trusting and deferring to your judgement before the responsibility becomes explicit. Responsibility is taken in small increments before it is given in full.

0 views
Dan Moore! Yesterday

What I Learned From Playing “Spelling Bee”

I’ve been playing Spelling Bee for a few weeks now, where you guess words from a selection of seven letters, one of which must be used in every word. The letters are arranged in a hexagon with that primary letter in the middle. If you want to learn more about how to play it, follow the directions here : For information on how to play, select the More in the top right corner of the game and then select How to Play . It has surprised me. I have learned that “oleo” is a word and that you can tell how hard the puzzle will be by looking at the rankings. But what I really wanted to talk about is the lessons I’ve learned from playing and how they can apply to your career. Switch Things Up There is a button that can flip around the arrangement of the letters around the hexagon. The primary letter remains in the center, but all the others are moved around randomly. When I first saw that, I thought it was silly. But I found it to be super useful over time. When I’m looking for words, I get stuck. Pressing the rotate button reveals new words because it changes my perspective. Just moving the letters around makes new words visible to me. In real life, when you feel stuck, you can sometimes make progress by flipping things around. I’ve done this in my career. I quit my first head of engineering job where I was trusted by the CEO, had loads of autonomy and worked on software that helped people find homes they loved. After, I enjoyed the freedom of lucrative contracting and learned a ton about other tech stacks. I took a quarter life sabbatical when I was around 25. After months of leisure, I learned that, yes, I really did like reading about parsing XML. I left the food-focused startup I’d co-founded (an area I’d dreamed of working in) which had just raised a seed round. Departing led me to an entirely different area of the software industry that I love and never would have explored at the startup. There are of course other ways to gain new perspective than quitting jobs: changing routines, asking friends for advice, or even taking a new route to work. Before you give up on something, flip the script and see what the results are. Just Try Something Sometimes you just have to try things. I will often put in words that I know are not valid, just mashing different combinations of vowels and consonants, like “manu”. This is not an English word, nor will it ever be. Spelling Bee also disallows proper nouns and three-letter words, but I’ve definitely put in plenty of those. Even though these are all invalid moves, they are movement. This helps me not feel like an idiot just staring at a phone screen. Even better, it can trigger other ideas. It’s a lot easier for me to visualize a full English word when I see part of it, rather than staring at those seven letters in a hexagon. After typing “manu” I found “manual”. There is value in just taking a step forward if you aren’t sure what to do. If you have some kind of goal in mind but aren’t sure how to get there, do something. Take a step towards that goal. If you are trying to get a job in the software industry, especially right now, it can feel overwhelming: so hard, so difficult. But you can take concrete steps towards that goal that are smaller. These steps, when you’re looking at them, may not make 100% sense and won’t make you money. But they are helping you on the path towards employment, and will lead to other steps to take that will get you closer to the goal. I’ve written before about how attending meetups is a fantastic career move . Doing so doesn’t have an immediate payoff, which is super frustrating when you’re trying to find a job, when your bills are piling up and your savings are draining. It’s super hard to think “I’m going to take an hour out of my day and go hang out with geeks talking about ruby “. But joining a meetup can open up other doors. Just like typing “manu” reveals “manual,” going to a meetup repeatedly can help you get a reputation in a community, offer contract opportunities, let you understand the market, and may help you get a job. This is just one example. There are a thousand different steps you can take towards your goal. It can be hard to determine which step is the right step, but the lucky thing is that there are many right steps. You just have to start. Double Down On What Works I like to double down when I have a pattern that is working. In Spelling Bee, if you see a pattern of “ull” being a portion of a word, you should double down on that while you’re thinking about it, and make all of the “ull” words you can. Same with suffixes like “ed” or “ing”. When I see that, I know that I can spell both the initial word like “mint” and get more points by adding “ing” to get “minting”. Even though I have 100% been guilty of a grass-is-greener outlook in my life, where I think, “oh, I’m frustrated with this company, or in this position, or with this situation, it’ll be better over <somewhere else>”. But after 25 years in software development, I know that every place has its issues. I was recently talking to a former colleague, and he was just hired at a super impressive top-tier company. He was talking about some of the issues they have and some of the burning fires. I would not have suspected it from the outside, because they look like they’re killing it. Sometimes, you should look for the good things in your current position and double down on them. This isn’t the same as staying stuck; it’s the opposite of that. Instead, this is about noticing when something is actually working and resisting the urge to abandon it just because it’s familiar. Switching helps when nothing is working; doubling down helps when something is. What does it look like? It might be working a bit extra, taking some free time to study for an employer-funded professional certificate, making small improvements to your team’s workflow, or just bringing your best self to work every day. All of these are taking what works and doing more of it. Double down on what is currently working for you, and it’ll make you happier and you’ll score more points. Three lessons I learned from the Spelling Bee game: switch things up, just try something, and double down on what works. Who knew that looking for words could be so educational?

0 views
Harper Reed Yesterday

Note #738

Sitting on the porch swing and I noticed some friends. Thank you for using RSS. I appreciate you. Email me

0 views
Pete Warden Yesterday

Why I ported Moonshine to Javascript

One of the most common requests I’ve heard from developers is an in-browser version of Moonshine that can run on a web page. In theory this should be straightforward – we already built MoonshineJS for the previous generation of models, and the core library is written in portable C++, so emscripten can compile it into WASM. There have even been some interesting community porting projects but I held off on official support until I had time to do it justice. I knew that porting the C++ core was just the beginning. Building something that would be straightforward for web developers to use required a lot more: After a lot of work, I finally have a version ready for feedback. The easiest way to try it is on the new moonshine.ai home page, where you can now see everything from a minimal transcription example to a full-blown Granola-style meeting note taker . As an open-source project, all the code for these is available and the simple examples include code snippets in-line too. Here’s one that shows how to run speech to text on a web page, to give you a flavor of the API: You may still be asking yourself why I made supporting Javascript in the browser such a priority? A lot of “X ported to WASM” stories end up being Hacker News bait without having any practical uses. The evidence that drove me was: I’m excited to get feedback on how to improve the initial version, and I’m looking forward to hearing about what people build with it, so please come by our Discord channel if you’d like to join our community. High-level APIs that were both idiomatic for browser Javascript and consistent with the other Moonshine language bindings. Infrastructure for testing from units to full web pages. Integration with the existing CI and deployment process. Examples that were interactive and showed the key capabilities of the library, with interactive inline code. Larger applications that demonstrated and tested how the framework runs in real-world conditions. Improved support for in-memory models and data files. This was involved a lot of changes to the core library, because while there had always been some methods that took memory buffers, coverage was patchy compared to loading from files. Clear developer demand . It came up frequently as a wishlist item when talking to users. Javascript’s dominance . Python rules machine learning, but JS is the most common language for applications, web and server-side. Advantages over alternatives . Voice interfaces are clearly only going to grow in importance over the next few years, but browser APIs are neglected and server-based alternatives are slow and costly compared to our on-client framework. Obvious applications . Dictation and meeting note taking are popular use cases for speech technology already, and talking with AI bots is becoming a lot more common too. Technical alignment . Deep in my bones I know that voice interfaces want to run on the client. The current status quo of streaming audio data to a server just to get text and intent back only exists because models used to be too large to run on consumer hardware. Today even household appliances have enough compute horsepower for local voice agents . It offends my engineering sensibilities to see old approaches linger on purely out of inertia. Speech wants to be free to use and local, just like all our other input devices like keyboards, mice, touchscreens, and cameras. Porting makes that possible on the web. Options . These days a lot of us have to frequently switch between languages and operating systems, and the power of AI coding assistants only increases the pressure to rapidly port applications. A library dependency is a big commitment, and knowing that it will be available anywhere you’re likely to run in the future makes the risk of betting on a framework much lower, even if you don’t need the option in the end.

0 views
Carlos Becker Yesterday

Old man goes to a Jiu Jitsu tournament

Seven months of training, 118kg, 2 bad knees, and 2 weeks of notice. Here’s how my first Jiu Jitsu tournament went.

0 views
xenodium Yesterday

agent-shell 0.73 updates

Another month, another agent-shell update. If you missed the last one, have a look at the 0.63 update . While this post showcases the latest highlights, the full list of changes is far chunkier than what we'll cover. agent-shell is a native Emacs mode to interact with AI agents powered by ACP ( Agent Client Protocol ). Since inception in September last year (yikes nearly a year), has featured a shell-like experience, powered by comint mode . There's also viewport mode (via ), if you prefer a more focused experience, and now we have . Chat mode fuses mode with a more traditional chat-like labelling experience. We're living on the edge here, so chat mode is now enabled by default. Ok not really that edgy, it's fairly safe (powered overlays ) and can be disabled entirely via . Chat mode itself is a minor mode, so you can always toggle it on and off via . In the last post, we talked about making less chatty with more grouping for the likes of tool calls and agent thinking, all collapsed by default. If that's far too quiet, you can expand by default via . If you found these two settings either too quiet or too chatty, we now have a third alternative via . When set, only the latest grouped activity is expanded by default, and automatically collapsed when the agent moves on to something else. I've become quite fond of this feature (thank you @nhojb for the PR ), so yes. It's also enabled by default. Queuing received some improvements. The related commands have been consolidated under . You can queue prompts while the agent is busy, then view, resume, or drop pending prompts via , , and . The pending queue is now shown after each new submission. opens a dedicated buffer for crafting a prompt, and it's now more independent of the shell. You can invoke it from any buffer (it resolves to the right shell), sends and returns you to whatever you were doing (fire and forget), and submits the prompt and immediately lets you craft another queued prompt. Shell initialization may take a second or two, depending on what agent you're using, which meant you had to wait for initialization before you could start typing into your new shell. Unnecessary, so that's no longer the case. Shell prompts are now offered as soon as possible, so one can get typing. While Markdown lists are easily digestible without special rendering, we can do better than that, so we now give them a better treatment with normalized padding, indentation, and of course, civilized bullets. TAB navigation made it into fairly early on. I love being able to TAB my way into any section in the buffer and press RET to toggle folding. That's great and all, but we can make the navigation experience richer by welcoming the likes of Markdown source blocks, links, and images to the navigation party. Why these in particular? They are all actionable by RET too, of course. While you'd rightly assume RET opens links to local text files in Emacs and delegates to browsers when needed, Markdown source blocks and images get a similar treatment. While their respective RET actions may not be as obvious, we can certainly make them much more discoverable, so we now add hints. Landing point on an actionable item now echoes a hint of what you can do and which key does it (say, "Press RET to copy" on a source block, "Press + to enlarge" on an image, and so on). Hints are also shown on mouse over events. While offers for customizing image sizes, it's fairly restrictive. Not all images are the same, so why force them all to fit within the same constraint? continues to offer a preferred default, but you can now scale images differently by getting the agent to annotate Markdown images with Pandoc-style link attributes . The attribute block goes right after the image, taking a and/or in pixels or percentages: I may be sweating the small stuff here, but this was really grinding my gears. We have lovely Markdown table rendering, which I'm glad we do as LLMs aren't always great at producing perfectly aligned tables. In the best of cases, the LLMs align the table perfectly, but it's just too wide for our Emacs window. Luckily, our lovely rendering also wraps cells to make them fit into our window. The thing is, all that lovely rendering goes out the door the moment you either resize your Emacs frame or merely split your window, resulting in a monstrosity like this: I know. I'm sweating the small stuff here, but hey we don't have to live like this. Emacs has all the hooks in the world, so let's track window changes and rejoin the civilized world. On a much smaller scale, I also wanted auto resize for images, so here you have it… While we can influence image size at render time, this can still generate undesirable image dimensions, so we can now rescale all images in buffer on demand. Sometimes a hammer really isn't the right tool, so we can now also rescale an image at point. Opening a local file (from a link, image, or mention) now routes through , a standard action you can customize. The default reuses a window already showing the file, or takes over the current one. If you'd like a different window arrangement, you could do something like: Following a local file link now pushes your origin onto xref 's marker stack, so ( ) brings you right back to where you were in your , just like other Emacs jumps. Foldable fragments now use and bind . If you're not a fan of the RET binding to toggle folding, you can now use your preferred binding. If is your jam, you can do something like: If you prefer styling agent thoughts differently, a new face lets you do just that. Streaming performance also received some love. Thanks to @suhail-singh for the profiling and improvements , and to @Scott-Guest and @claytharrison for the trace analysis and benchmarking in #757 . The "Available config options" section now displays possible values. can now be set to a function, letting you compute the available agent configurations dynamically rather than hard-coding a static list. The function is called on every access, so it stays current across code reloads. Maybe you'd like to list only available agents. Here's a rough snippet. now broadcasts an event, handy for external integrations that want to observe streamed output. now works correctly on remote hosts ( #742 by @CeleritasCelery ), smoothing out TRAMP-driven remote agents. agent-shell-hq joins the family, offering an interface for managing multiple sessions. If you peeked at the commit logs , you'll notice I've been working daily on , keeping up with project inflow. Since last month, 27 issues have been closed and 13 pull requests merged. As of today, the backlog sits at 11 open issues and 5 open PRs (versus 13 and 4 last time around). If there's something you'd like me to prioritize, feel free to ping. Vendor-neutral tooling matters more than ever, and there are a couple of ways to help keep going. Some cost money, others just a click. All are appreciated ;) is just me, an indie dev, while the tools it competes with have well-funded teams behind them. Time spent on is time away from work that pays the bills, so if it's useful to you, please consider sponsoring the project. And if your employer benefits from your use, nudge them to chip in too, they can typically contribute at a scale individuals can't. GitHub stars help with exposure, attracting new users and potential sponsors. Starring agent-shell costs nothing and can potentially help bring in more funding, so if you don't mind a couple of clicks, the project can really use another GitHub star . Thank you to all contributors for these improvements! Liking ? Would like to see it evolve? Consider sponsoring the effort. #730 : Queue requests sent during session/push and submit them when it ends ( @catern ) #737 : Add a workaround for goose ( @bergmannf ) #740 : Add a option to ( @nhojb ) #742 : Fix executable-find on remote host ( @CeleritasCelery ) #743 : Preserve "claimed" regions when rendering bold / italic / strikethrough ( @alberti42 ) #746 : Avoid repeated system-sleep load attempts ( @liaowang11 ) #748 : Preserve properties on escaped Markdown punctuation ( @Scott-Guest ) #752 : Guard group member walk against non-advancing block range ( @hamza-m-masood ) #756 : Preserve point during table rendering ( @Lenbok ) #762 : Document viewport workflow ( @KarimAziev ) #763 : Render raw tool output ( @mrychlik ) #765 : Prevent syntax highlighting from delaying other buffers' mode hooks ( @Scott-Guest ) #766 : Refresh viewport header even with nil agent-shell-prefer-viewport-interaction ( @catern )

0 views
Farid Zakaria Yesterday

nixpkgs-multiverse: fast mode

“The fastest evaluation is the one that never happens.” – Sun Tzu, The Art of Evaluation nixpkgs-multiverse gives you every version of every package that ever shipped in Nixpkgs from a single flake input. Note It continues to blow my mind that this is even possible. It feels like it suddenly unlocks a new dimension of Nixpkgs, and I am still trying to understand what it means. I think this capability is a fundamental change to the way we think about Nixpkgs, and it is not just a new feature. It is a new way of thinking about the entire ecosystem. There was always a penalty at the center of it. Asking for a specific version of , such as , meant fetching the whole ~378 MB Nixpkgs tree from 2021 and evaluating it to determine the . What if we could skip that evaluation? What if we could just ask for the path directly, and have Nix fetch it from the cache if it is there? This is a common idiom if you have ever used . That requires knowing the store path upfront. nixpkgs-multiverse now has a attribute that does exactly that: it gives you the store path for every indexed version of every package. This lets you skip the download and evaluation of Nixpkgs and get the store path straight from the cache. No Nixpkgs is fetched. Nothing is evaluated. No experimental features and no needed for this to work. The complete Nix API, except for releases, works with this fast path. If you want to learn more read the docs about the feature. Every channel bump published a listing of every path Hydra built for it: , or a for back in the pre-2017 era. These files are still available, and they are the source of the multiverse index. The listing is a map from derivation name to store path. The multiverse index is a map from to the revision that shipped it. By joining the two, every historical version gets a concrete address: Knowing the path is not enough, especially in the Nix language. We need to convince Nix that a string that looks like a store path actually is a store path. exists but it is an impure function and requires to work. How do we get around this? We attach “context” to the String. Context is the invisible baggage a String carries in Nix. When you interpolate a derivation into a String, the result remembers where it came from, and that is what makes realise the dependency instead of writing a dangling path into a script. lets you attach it by hand. The identifies that “this String names a store path that must exist,” which is exactly what produces for a path already in your store, except this works for a path that is not in your store yet and is not in this evaluation’s input closure either. Loopole! 👿 We then wrap that in an attrset that resembles like a derivation and the Nix CLI is satisfied: This is tomberek ’s trick from fastpkgs , and it is an amazing trick to circumvent needing . Everything about this remains pure evaluation, and the resulting graph is gauranteed to be bit-for-bit identical to what Nixpkgs would have produced if it had been evaluated. The only difference is that we skip the evaluation of Nixpkgs itself, and instead use the store path directly. The eval path derives the address, the fast path remembers it. A “fake” ( ) derivation has no , because there is no behind it. Nothing can build it and it can only be substituted. The CLI often wants a though when you hand it a derivation attrset, so we must make sure to append the output (i.e. ): and need a real derivation. Every fake derivation carries a lazy that is the real, revision-exact derivation: In the spirit of trying to keep my index small, is empty, so there is not additional information about the package. You can still get the from the real derivation by using as well. This scheme rests on cache.nixos.org still serving thirteen-year-old paths, which thankfully it does and with the same signing key. To demonstrate that the cache is offering nearly every path that Nixpkgs ever built, I ran a census of every indexed version of every package and asked the cache if it was still alive. As of August 14 2026, all 271,187 of them are alive . All of them, down to every NAR payload file. That is 14.8 TB of unpacked software from 2013 onward, one fast command away. 1 There’s some other data on nixmultiverse.com about the census, dependency graphs and additional features. Check it out! All of this is also available via the mvs command line tool as well for offline use. I guess now there is a caveat: there is now a trick in the multiverse. It remains mostly an index, some JSON, and a behind a memo table. The clever trick is tomberek ’s, and it is three important lines. The NixOS infrastructure has never garbage collected the binary cache. It is an S3 bucket that only grows, and the bill is paid by the NixOS Foundation and its sponsors.  ↩ The NixOS infrastructure has never garbage collected the binary cache. It is an S3 bucket that only grows, and the bill is paid by the NixOS Foundation and its sponsors.  ↩

0 views

Everything is about to “go dark”

I’m coming down from spending a few days at Usenix Security, right here in Baltimore. This means that my days have been taken up with two kinds of conversation: first, explaining to colleagues why Baltimore isn’t actually like The Wire. And second: trying not to talk about AI. Here I’m going to break both of those rules. I have many worries about what AI means for our field, for various definitions of “field”. But in this post I want to focus on just one thing I’ve started worrying about, and it’s a perverse thing: specifically, I’m worried that AI is going to make software much too secure. While that doesn’t sound so bad on the surface, there’s a consequence to this. I mean something very specific: I’m concerned that U.S. intelligence and law enforcement agencies are about to go dark, meaning lose a huge portion of their capability. And that this isn’t going to be simply a problem for those agencies, but also for those of us who value computer security and privacy in general. To explain how we got here, we need to talk about recent history. Here we have a real excuse to reference The Wire, which embeds a realistic snapshot of what electronic surveillance looked like in 2002. The cops in that show are after payphones and burners, all used for voice calls. While the mobile phones were new, nothing in here would have surprised a cop from 1989. Less than a decade later, everything was different. The change started in the late 2000s with the rise of smartphones and texting. In 2010, Apple began encrypting iPhone data using a key derived from the user’s passcode, and Google followed behind them. In 2011, Apple deployed end-to-end encrypted text messaging. By 2014, WhatsApp had 600 million users worldwide, and by 2016 nearly a billion — and they were all using end-to-end encrypted messaging. The chart below gives a snapshot of how quickly the world changed between The Wire era and 2016: The FBI and law enforcement agencies noticed the trend and took it very seriously. In 2014, Director Comey announced an initiative called G oing Dark , which would launch a “ national conversation” about what providers could do — or be compelled to do — to make these new communications media legible to law enforcement and counterintelligence. In 2016, the agency stopped talking. When a terrorist attack left the FBI with the shooter’s locked iPhone, the agency ordered Apple to give them access . The company refused . What broke the stalemate — and, to some extent, ended “Going Dark” itself — was something that neither the FBI nor Apple expected. An outside company announced that there was no need for Apple’s assistance: they could simply hack the phone . The Apple v. FBI case turned out to be microcosm of the whole debate. For the next decade, law enforcement and intelligence agencies continued to ask for exceptional access backdoors. But the urgency was gone: agencies and manufacturers knew that law enforcement could purchase targeted hacking tools if they needed them badly enough. Vendors like Apple and Google played a vigorous defense, closing vulnerabilities as soon as they learned about them. But commercial offensive vulnerability hunters consistently managed to keep the edge. Anyway, that’s the history. And now it’s about to be over. In April, Anthropic announced a new model called Mythos that was optimized for software vulnerability finding. The U.S. government temporarily blocked its export, restricting it to U.S. agencies. While the ban was dramatic and made for good PR, it was mostly pointless. OpenAI , along with Chinese open-weight model labs like Z.ai and Moonshot , have since demonstrated that vulnerability finding isn’t anything that a single model can hold a monopoly on. The list of serious vulnerabilities that these models have found is getting scarier (or more impressive) by the day. Initially this might seems like good news for the offense, and for hackers in general. But I doubt it will last. Defenders are now in the process of patching every bug they can find, often with AI helping them. Entire development toolchains are being rebuilt to incorporate powerful vulnerability scanning before software reaches the testing phase. This does not mean that every bug will be found: even calculating the number of bugs in a piece of code is probably uncomputable. In the real world, it does feel likely that we’re going to hit some sort of a ceiling on the number of useful bugs, and probably we’ll hit it soon. Thus: over the next two years, major pieces of software are likely to run out of remotely-exploitable bugs. While I think this is great, for law enforcement and offensive intelligence agencies, it’s going to be a nightmare. For the first time since 2010, law enforcement might experience what it looks like to really “go dark”, across a huge category of advanced (well-maintained) devices and pieces of software. The debate over “exceptional access” mechanisms never really went away. In some places, like the UK, it actually metastasized into something worse. Here in the US it mostly went into hibernation. Some of the slowdown can legitimately be attributed to expert pushback — academics and industry engineers pointing out the risk that backdoors might be abused by the very adversaries they’re designed to protect against. But I fear that the market was just pricing supply. The destruction of the low-hanging vulnerability fruit will make law enforcement (and intelligence) agencies’ need much more acute. The demand for constructed, intentional backdoors will begin in earnest. There will be enormous pressure on industry to re-architect their systems to make their systems friendly to exceptional access. In some cases, governments will ask for these capabilities in the expectation that they’ll be useful for spying on other governments — a strategy that might have been undetectable in the pre-AI era, but that probably will be detectable now. This might result in other governments curtailing their dependence on US software. In fact, the worst part about this dynamic is that these potential new backdoors will begin primarily useful for allowing the US to weaken its own systems, which will in turn allow foreign adversaries to find new ways to attack our communications. This deliberate self-sabotage will happen just at a moment when we’re finally learning how to defend our own infrastructure. I honestly have no idea. This is not a call to action for experts to rally behind a sophisticated plan. Like so many things about the AI revolution, it’s just occurring to me that we’re on a long greasy slide to a place that will look different than where we are today. Just realizing this doesn’t mean that I have a clever plan to avoid it. In this case, we’re just going to have to hope that this time we make the right choices, for no other reason than that they’re right.

0 views
Harper Reed Yesterday

Note #737

Denver has changed, man Thank you for using RSS. I appreciate you. Email me

0 views
Jim Nielsen Yesterday

Oh Hey, It’s Not Just Me

My whole life I’ve felt like I have great eyesight (and I still do). But lately I’ve been noticing what feels like…some slippage. Nothing big. Just this feeling in the morning like, as my brain and body wake up, my vision needs to as well. The strange novelty of this phenomenon was operating in some small part of my subconscious the other day when I was scrolling the internet and come across this post from Chris Coyier : you ever get to your mid fourties and have your eyesight drop off a friggin cliff And I’m all like: And then even more people weighed in on Chris’ thread, like “Hey me too!” This is why I love the web. I continually get that feeling of, “Oh hey, it’s not just me!” On both a personal and a professional level. And for that I’m grateful. I love being able to connect with people in a way that makes me not feel so alone (or weird) in my experience of life. In a polarized digital world, it’s good to remember that the web can still draw you to folks rather than push them away. We’ve got a lot in common. Reply via: Email · Mastodon · Bluesky

0 views
Stratechery Yesterday

2026.33: The CapEx Train Keeps Rolling

Welcome back to This Week in Stratechery! As a reminder, each week, every Friday, we’re sending out this overview of content in the Stratechery bundle; highlighted links are free for everyone . Additionally, you have complete control over what we send to you. If you don’t want to receive This Week in Stratechery emails (there is no podcast), please uncheck the box in your delivery settings . On that note, here were a few of our favorites this week. This week’s Sharp Tech video is on why everyone but OpenAI and Anthropic wants open weight models. The Capital Constraint . Everyone knows we are short of AI compute. Everyone knows that we may very soon be short on power. What happens, however, if we are short on capital? If AI is as valuable as it seems, then it should pay for itself, but that hasn’t happened yet. This week Nvidia announced a new funding mechanism to tap long-duration capital; this sort of financial engineering, along with Google leading the way in tapping equity , may build the bridge to sustainable AI revenue. It also expands the blast radius of a bubble in the service of Nvidia’s threatened margins. I covered this in Nvidia’s Risky Business , as well as this week’s episode of Sharp Tech. — Ben Thompson What to Do About AI Writing? Anyone who’s been online the past few years has found themselves wondering “was written by a human or AI?” and some version of that question will probably persist in perpetuity for all of us. To that end, the EU has mandated that providers of AI systems mark all their outputs, including text, as AI-generated. Wednesday’s Update explored Anthropic’s response to that European regulation and why Ben thinks all of this is a terrible idea. We talked more about the issue on Friday’s episode of Sharp Tech , including the history of idea propagation across centuries , and the likelihood that my son will likely feel differently about AI-generated output than I do.  — Andrew S harp A Tale of Two Cites. In an article that was written 100% by a human (though proofread by ChatGPT!), this week’s on Sharp Text explored why caustic rhetoric from Mayor Zohran Mamdani is probably not enough to convince finance execs to leave New York City (even if they live in Connecticut), while Hollywood has unfortunately seen much of its filmmaking industry outsourced to neighboring states and foreign countries (as David Ellison threatens to relocate Paramount’s operations). In brief: New York is a case study in the power of network effects, while Hollywood offers a lesson in the conditions that allow those effects to be broken (and Mamdani exemplifies an irony of the DSA movement, generally).  — AS Apple Earnings, More on Amazon’s Earnings — Apple’s earnings (and stock) are limited not by memory but rather chip shortages; then, more on Amazon’s earnings and Andy Jassy’s market analysis. Nvidia’s Risky Business — Nvidia is finding new ways for its customers to raise money, and it’s expanding the risk of the AI buildout significantly. Anthropic’s Watermarking, How It (Probably) Works, Worse Than It Seems — Anthropic is adding watermarking in response to the E.U.’s AI law. It’s a terrible idea, first and foremost for philosophical reasons. New York and the Power of the Network — New York City exemplifies the power of network effects, while modern Hollywood is a reminder that they do have limits . Immersive Baseball and Frontier Models The Subsea Cables Are Listening SK hynix and the HBM Revolution Summer Top Fives: Changing Our Mind, Post-Apocalyptic Crops and Players Nvidia’s Answer to Capital Constraints, Google’s Attrition and Direction, Q&A on AI Writing, Vision Pro, Vibe Coding

0 views
Unsung Yesterday

“What actually happened was the birth of a genre in digital graphics”

A really well-produced 27-minute video from Super Splash Wave about 8-bit and 16-bit pixel art: = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/what-actually-happened-was-the-birth-of-a-genre-in-digital-graphics/yt1-play.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/what-actually-happened-was-the-birth-of-a-genre-in-digital-graphics/yt1-play.1600w.avif" type="image/avif"> What I liked about the video is that it doesn’t just revel in nostalgia, but goes deeper into some techniques and trends and details in the videogame graphics of the 1980s and the 1990s: palette limitations, palette animation , sizes of objects, dithering, parallax, small ventures into 3D, etc. It was also great to see specific changes and evolution of what from the distance of 2026 might seem like one monolithic era. The very first games used rigid grids and the assets were generally done by engineers themselves. Then, the work was split between artists doing the visuals, and engineers implementing then. Only after that, closer collaboration between engineering and artists allowed truly spectacular effects to take place, mirroring deep collaboration between UX and front-end elsewhere. #games #graphics #history #youtube

0 views

Premium: How Much Money Does AI Need?

I’ve heard from people in the past that my articles are too long, and I wanted to start by saying that, for the most part, they’re going to stay long, because I feel like the only way for me to make my arguments is to be as specific and detailed as possible about the things I’m talking about.  Then again, sometimes it’s just because I imagine arguments against my work in my head and want to pre-empt them. Something about the AI bubble has made the boosters genuinely insane. They see these otherworldly declarations — hundreds of billions or trillions of dollars — and assume that nobody would say them in bad faith, and that the tech industry would never fail to live up to them, even though we’re barely a few years divorced from when Mark Zuckerberg burned $80 billion on the metaverse , what will one day be known as “the second-worst misallocation of capital in corporate history.” When the boosters  hear that OpenAI plans to spend $750 billion on compute costs through the end of 2030 , they shrug their shoulders and say “it’ll work it out.” When they hear that hyperscalers have $1.65 trillion in off-balance-sheet obligations and debt , they nod approvingly, saying that “these are some of the richest and most-profitable companies in the world,” and that they will “simply keep raising debt.” It’s somewhere between number-blindness and make-believe — these are such unfathomably-large sums that it’s hard for the average person to assume anything other than that nobody would sign contracts agreeing to pay them without the confidence they’d be able to do so, even though it’s all very silly. In any case, readers, I hear you , and today’s premium newsletter is going to be a shorter one, because it’s been an incredibly long week for me, including a day that started at 5AM with four different interviews — including my appearance on CNBC, which I encourage you to watch — that ended roughly 14 hours later, which means I’m a little depleted but nevertheless dedicated to you, the reader, and giving you value for your subscription. So today I’m going to be pithier, and focus on hard numbers and harder truths about the AI industry, and specifically seek to answer a question: how much does the AI industry actually need by 2030?  To be specific, I’m going to be focusing on the next three fiscal years for the companies that matter — the lead hyperscalers (Meta, Google, Microsoft, Amazon, Oracle), the two leading semiconductor firms making AI chips (NVIDIA, Broadcom), the main neoclouds (CoreWeave, Nebius, IREN, which I’ll cover in short), the main AI labs (OpenAI, Anthropic, and SpaceX) and the overall AI compute industry.  I’ve spent a great deal of time in the last few years explaining in detail why I think this will all collapse, but today’s goal is to show you, in hard numbers, exactly how much money the main players in AI need, based on consensus analyst estimates and my own research. And god damn, do they need a lot.

0 views

Who’s Tracking You? Use This New Service to Find Out

It can be daunting to determine who’s responsible for showing ads on the websites we visit, or who’s harvesting data from the mobile apps we use every day. That information is already semi-public, but it is not easily parsed and traditionally much of it has remained walled away in the hands of large advertising platforms. Not anymore: A powerful and free new service called DecryptAds scrapes and correlates this adtech data and makes it simple to quickly learn a great deal about the entities that are tracking you. A Decryptads summary of the advertising partnerships declared by espn.com. The newly launched decryptads.com says it is constantly scraping the files that websites and apps make publicly available to disclose the companies that are permitted to run ads or collect user data. These files include: – ads.txt : all of the adtech companies and data brokers that may run ads or harvest data from the site; – app-ads.txt : entities that can harvest data from or display ads on mobile and smart TV apps; – buyers.json/sellers.json : the entities buying, selling or reselling ad inventory for a given site or app. Zach Edwards is chief research officer for DecryptAds and a threat researcher at the security company Infoblox . Edwards said he and two other founders decided the service was needed because the adtech data in these files is generally only useful when it can be cross-referenced to build a more complete picture of the advertising ecosystem for each website or app. “It’s an adtech tool but we’re trying to approach adtech from a security perspective,” Edwards said. “It’s really built for a lot of privacy and security use cases that have been dramatically underserved.” Those use cases, he said, include tracking down the source of malicious ads that try to foist malware on targeted users, identifying ad networks located in adversarial nations, and detecting the fast growing swarms of AI-generated slop websites and apps. And as decryptads.com demonstrates, these potential security and privacy threats are near impossible to detect just by viewing a single apps.txt or app-ads.txt file. “Supply-chain integrity issues rarely live in a single file,” the site explains . “They show up as broken cross-references between ads.txt, app-ads.txt, and sellers.json files; as cloned declaration sets across unrelated domains; as seller removals that only make sense when viewed across exchanges; and even as supply paths in bid logs that never actually appear in any given publisher’s authorized-seller list.” A search in DecryptAds for the hugely popular sports network espn.com reveals 143 ad partners and 19 registered data broker domains are listed within its ads.txt and app-ads.txt files. That data broker information is gradually becoming available because four states — California, Oregon, Texas and Vermont — have recently passed laws requiring data brokers to register if they buy or sell data on consumers from those states. DecryptAds reports that almost half of those data brokers are collecting geolocation data from espn.com visitors who aren’t blocking ads, while another three disclose that they collect device fingerprints and sensitive personal information. A visual representation of the complex ad supply chain declared by espn.com. Image: decryptads.com. DecryptAds also makes it easy to learn the beneficiaries and national origins of the advertising firms lurking in apps and websites, displaying a conspicuous warning when adtech partners of an app or website are based in “geo-risk” areas like China and Russia, or in countries with strong financial and political ties to both — such as Cyprus and the United Arab Emirates (UAE). According to DecryptAds, espn.com works with four different advertising entities that are based in either Russia, China or the UAE, including the adtech firm Between Digital , which lists a New York address. However, the dossier on Between Digital flags them as a Russian firm, showing that their publisher offers (PDF) are processed through Alfa Bank , Russia’s largest private commercial bank and one of several financial institutions placed under U.S. sanctions in 2022 after Russia invaded Ukraine. KrebsOnSecurity sought comment from both Between Digital and the company’s founder, and will update this story in the event that either replies. A search for several top U.S. military news websites — including armytimes.com , airforcetimes.com , defensenews.com , navytimes.com , marinecorpstimes.com and federaltimes.com — shows they all allow Between Digital to serve ads and track users, as well as two entities in the UAE and another in the ownership secrecy haven of Panama. DecryptAds reports that Between Digital is collecting ad data on approximately 55,000 partner websites. The “Geo Risk” section of decryptads.com. Pivoting on Between Digital’s app-ads.txt file reveals hundreds of domains featuring simple web-based games that are frequently interrupted by ads. Edwards said Between Digital’s own declarations show the company is listed as both a publisher and a reseller on approximately two-thirds of their portfolio. “It means they are basically playing both sides of the bidding equation, which creates opportunities to direct client spend at your owned and operated properties or client infrastructure, essentially creating opportunities for conflicts of interest,” Edwards told KrebsOnSecurity. “The problem we have right now is that for years we’ve had almost no one policing these ads.txt and app-ads.txt files.” The Opera Web browser remains quite popular, and probably many users are unaware that since 2016 it has been majority owned and controlled by the Chinese company Kunlun Tech (the operational headquarters of Opera remain in Oslo, Norway). Opera.com’s profile at DecryptAds identifies 27 registered data brokers collecting information, including 15 adtech partners in the UAE, six in China, three in Cyprus, two in Russia and one each in Hong Kong and Ukraine. DecryptAds makes clear, however, that these companies represent just seven percent of the adtech partners specified in Opera.com’s ads.txt and app-ads.txt files. One feature of DecryptAds that sent this author down multiple hours-long research rabbit holes is its Legal Dossier lookup , which takes several minutes for each search but eventually churns out oodles of useful information about who owns a particular domain or app, when it was registered, and any aliases or relationships it may have to adtech companies and other websites or apps. For example, last month KrebsOnSecurity wrote about researchers from Bitsight who found that an extremely popular line of TV streaming sticks called H96 quietly rent out each user’s Internet connection to strangers. Bitsight also discovered that when these devices aren’t being used to stream pirated video content, they are spoofing themselves as mobile phones clicking ads on AI-generated slop websites . Bitsight concluded that the same Chinese company that made several of the malicious apps common to all of these H96 streaming sticks — the Fengwo Group — also also ran the network of ads and AI slop websites being clicked on by tens of thousands of these devices that are pretending to be mobile phones. Examples of ad landing pages linked to the Fengwo Group. These sites were designed to show ads only to H96 devices that were spoofing their device type as mobile phones. Image: Bitsight. A DecryptAds legal dossier on the (now dormant) Fengwo Group domain name for the AI slop website pictured on the left in the screenshot above ( medicalbeautyhub dot com ) shows it shares a seller ID ( 1674071 ) with a gaming website — giacoloredstones[.]com — which features yet another seller ID ( 103488000 ). Pivoting on that latter seller ID reveals hundreds of active websites within Russia’s Yandex ad system featuring extremely low-quality games or simple utilities that pepper visitors with ads. Edwards said that when advertising networks suspect a given advertiser is engaged in unauthentic clicks or displaying malicious ads, very often those networks will quietly remove the offender from their list of approved partners without letting anyone else know about their suspicions. This practice, he said, makes it easier for dodgy adtech firms to avoid accountability and continue victimizing others. To address that visibility gap, DecryptAds features a quiet removals feed that records and correlates all of the sellers.json removals across ad exchanges for the same seller domain or name. A screenshot of the Quiet Removals Feed at decryptads.com. “The way the adtech industry works, someone will write a report about ad fraud and only share it with their own clients and they won’t make it public,” Edwards said. “The ban is just removing them from the sellers.json file, but they told nobody. One day it was there, the next it was gone. So if you’re trying to navigate who is suspicious, that’s usually tough to do because there are a lot of adtech companies removing things all at once.” Malvertising, the term given to the practice of inserting malicious ads that foist malware or redirect visitors to phishing pages, remains an all-too-frequent occurrence in the modern adtech industry. But Edwards said these malicious ads are far more commonly found now on newly generated AI slop websites than on high traffic destinations that typically employ a variety of technologies and third party tools to quickly flag bad ads. “None of these slop AI content farms are paying for that kind of protection,” he said. “They’re just signing up the lowest quality partners, and it essentially becomes a greased rail to target the users of those sites with malicious ads. Most malvertising attacks don’t happen on espn.com or huffpost.com, but rather [on] some lower quality content farm and someone just went there because it came up in a search.” Edwards said the AI slop websites are populated with machine-generated blog posts and images, and cover a wide array of themes from home improvement and decorating to food recipes, hunting, cars and consumer technology. He said organizations that get hit with malicious ads are often at a loss for what to do next, unaware that in most cases the answer is one of the entities listed inside the website’s ads.txt or app-ads.txt file. “A lot of serious organizations are starting to understand that if we’re not breaking down this ad data, we’re not going to know who’s targeting government people with zero-click payloads on an almost daily basis,” he said. Edwards maintains that truly getting a handle on the malvertising and AI slop problems will require more data-sharing by the major ad networks. Specifically, he says those platforms do not broadly share what’s known as the “supply chain object” or SCO, structured data attached to each advertising bid request that lets buyers see every seller, reseller and intermediary involved in passing an ad impression from the publisher to the final buyer. “That SCO tells you who sold it or resold it, and who was the final entity that bought the impression that served that malware payload,” Edwards explained. “You may see the malicious zero-click redirection, but without the supply chain object — which is only served server side — you won’t know who targeted your people with malware and won’t have a way to try and prevent it properly. But if we can encourage the adtech industry to expose that SCO, it will get easier to find the culprit behind any one bad ad.” DecryptAds also offers an application programming interface (API) that allows researchers to automate queries and integrate the site’s functionality into popular AI platforms. The only sane reaction to the examples described above is to block all online ads outright. This approach is broadly endorsed by security experts because it also makes it more difficult for adtech firms and data brokers to build detailed profiles on you and track your movements around the web and in the real world. However, much depends on how you normally prefer to browse the Internet, and how much trust you place in third party browser plugins and extensions. For those primarily surfing via a regular desktop or laptop Web browser, uBlock Origin Lite is an excellent free and well-maintained open source option. uBlock Origin also should work with mobile browsers like Firefox, but apparently only on Android-based devices. Adblock Plus is a decent option for iPhone and iPad users. For power users, Adblock and uBlock Origin both support custom blocking rules from easylist.to , which publishes a frequently updated list that removes most advertisements from webpages. The well established browser extension NoScript blocks all non-approved Javascript code, and it generally does a fine job blocking most ads from loading. However, script blockers like NoScript may not be suitable for average users who don’t enjoy constantly having to referee which scripts should be allowed to load so that each site displays properly. More technically inclined/adventuresome readers should strongly consider a hardware approach to blocking ads at the local network level, because that is easily the cheapest, most secure and scalable way to do it. A tiny, low-cost and broadly available computer known as a Raspberry Pi can be turned into a powerful ad blocker for all devices on a local network when fitted with a microSD memory card and a free program called Pi-hole . Once you’ve set it up properly and changed your router’s network settings to use the Pi-hole’s DNS sinkhole and DHCP servers, it should prevent ads from displaying on any devices connected to that network. Bear in mind that ad blockers often do little to block ads and/or tracking that occurs from within mobile apps that users have chosen to install on their devices. Many websites now push users to install a mobile app, supposedly in order to more fully access and enjoy the site’s services and content. But in my experience, they’re not doing this because the user experience is somehow way better on the app (as LinkedIn tries to convince us non-app users several times a week via email). On the contrary, I find most mobile apps to be horribly designed, annoying, and/or completely unnecessary, and when given the option I will almost always choose to interact with a website or service directly in a Web browser. No, the cold truth is that big web destinations tend to get pushy with their apps because they make it easier for these companies to keep you on their platforms longer and to collect (and in many cases resell) far more precise data about who, what and where their users are. Also, companies pushing customers the hardest to install mobile apps always seem to liberally opt everyone in to having their data used to train large language models these days. So be cautious about the apps you install on your mobile devices ( including any smart TVs! ), and poke around their listings at DecryptAds if you want to learn more about their privacy practices and any relationships they may have to adtech firms.

0 views

curl performance

tldr: the live version is here: https://curl.se/perf/ How fast is “fast” and is it good enough? Does it run as fast now as it did before or was there a regression? What exactly needs to be fast? How fast is it? These are questions that many projects and products face, and in curl we are no different. Yet, performance testing and comparisons are hard and full of landmines and time-wasting efforts. For many years we have occasionally brought up the idea of a performance test suite for curl only to shut it down again because the challenges seemed hard and no one was volunteering to do this. This week it changed. I started out trying to find existing projects that host performance results for Open Source projects so that we could just feed our results something else and get great visualizations and data management. I did not find any such. I then took a look at what existing tools there are for this purpose, and most pointers seemed to suggest that Grafana is a popular and maybe even a good solution to build something like this with. But man, that is a complicated machine and it felt more than a little overwhelming just figure out where or how to start with it. I decided to postpone that take as well. I decided that instead of trying to do this the best and optimal way – I shouldn’t let perfect be the enemy of good – I would start out by doing the things I know how to do and take it as far as I can one step at a time. Something should be better than nothing . Performance testing needs decently stable system conditions so that repeated runs produce reasonably similar results, when all involved factors remain identical. This is basically impossibly to accomplish using most cloud infrastructure since those are almost always shared with countless other users. At least on the cheap and free tiers we use. We probably need our own dedicated hardware for this, but instead of trying to figure out where to get that and arrange for that, I would start by running performance tests on my own local development machine. I am a single user on this and it has many cores and runs decently fast. It should be good enough to get this going on. I created a first shell script that updates the curl source code from git, it configures and builds it. Then it runs a bunch of tests, outputs a bunch of data and logs all the output in a single log file. I started out with a few simple tests. How fast does curl download a 100 GB file from localhost, how many allocations and how big allocations does it need for a single HTTP download? My second script parses all the test log files from the previous builds and generates summaries and graphs for them. To make it possible for humans to see how the performance changes between builds and ideally to automatically detect when something changes more than what should be tolerated. As I am a graph addict already since before , and that journey has taught me a little gnuplot , I decided that even while there probably are much better tools and fancy JavaScript things that could be used, I don’t know them and learning them now is an endeavor I rather avoid. So I stick to what I know and can get results with quickly. A third script is invoked from a crontab every twenty minutes, sets up some variables and invokes the runner script. Once the basics started to work, I showed my curl friends the early versions and I soon created a new git repository for the code . After a little more poking, I soon made my locally produced performance test summary get packaged and automatically transferred to the curl website after each build, and voila, the first public curl performance tests were live and public. Getting this data available immediate triggered curl developers. It only took hours until we had the first proposed changes to improve some numbers, and soon we had a few merges to that affect. Visibility really helps! The performance numbers we get are still varying to a certain degree, partially of course because I still use my machine for my daily development things, but also because most of them do real (localhost) networking and that is by its nature a little… varying . The system builds and runs a new round every twenty minutes and it does that using the latest commits from git. This setup makes it sometimes run many rounds on the same commit and it might also mean that it sometimes updates and get several new commits at once, so it might skip a round for some commits. I might reconsider this design later, but since it is still a twenty minute time window, the number of commits is still limited. When the script makes multiple build rounds on the same commit, it accumulates the numbers and for the graph it stores the maximum, the median and the minimum value. It helps show the variation per commit and allows us to cram more into the graphs. It is still early days, but there will be a maximum limit to how many commits that can be displayed in a single graph and still be helpful. HTTP/2 parallel download speed through 261 builds spread over 31 build rounds Distribution To help visualize the distribution and data spread per test, I created a separate illustration that shows the minimum, maximum, P25, P75, medium and mean values in a Box-and-Whisker Plot . A Box-and-Whisker Plot showing the HTTP/2 parallel download speed data distribution. Changing conditions An obvious downside with me just storing build logs in files, is that it will not scale up to the millions. I did however decide that I’m not designing this system for that. At least not now. Performance tests are highly specific and dependent on the exact machine it runs on, the exact third party libraries and their versions that are used, the other components involved in the tests, such as the servers, and more. I expect that we will change conditions for the tests every once in a while that makes it hard to compare the current numbers with past numbers. Therefore I think the performance test numbers and values are primarily useful in the short term. To help us spot if we land something that subtly and unintentionally degrades something. To detect extremely slow and long-term changes in performance and even making sure we can better survive wiping all the existing build logs etc, I introduced a concept I call stakes . As in a stake pole. A marker. An arbitrary threshold set manually for each specific test. This value can be used to measure performance test results against, now and later. As conditions change and maybe something makes the results go up or down and we are fine with those changes because they are motivated and expected, then we just change the stakes. If it works out, I might try to have the system automatically detect and maybe highlight tests that deviate too much from its set stake (at least if done in the wrong direction) . It could be a signal that something bad was merged. As with everything in life, things are often balanced out. We already ran into this when we eagerly merged several changes to reduce the number of allocations done for a single HTTP download, only to realize that one of the optimizations we did had the side-effect that it expanded the size one of the main structs maybe a little too much… Improvements in one area might come at an expense in another. With sufficient tests and data we can improve curl for users, and at the same time make sure that our changes don’t come with a cost we are not prepared to pay. Exactly how to make the balance is of course a question we need to deal with, discuss and decide. Possibly for every change we do! As I write this, we have 24 tests and a full test round completes in about six minutes on my machine. We can of course do multiple builds using different hardware, different operating systems, different build options, different third party libraries and different test servers to check more angles of performance, and I am certainly open for and prepared to do that going forward. I will however first let this single-flavor run for a while so that we get more data, get a change to tweak it and make it as usable as possible for curl developers. As with everything there is no end to what we can make this do. This is a start. I sure we can take it further as we move along. In particular if people join in and help out. Both with ideas and proposals for visualizations, graphs and new tests to add, but also with actual pull-requests and code. Over the last year, we have merged, on average, about 10 commits per day. If we keep this pace up and this performance test setup can show 100 build rounds conveniently into a single graph, that is just ten days of development. Probably not enough. Once we reach one hundred builds or so in the first graphs I need to consider adding separate long term graphs that use select data-points to display data development over a longer time. Some googling told me the Largest-Triangle-Three-Buckets, or LTTB for short, is a fine algorithm to use for this. I now do a separate “long term” graph that “downsamples” the full range down to something that can be shown in a reasonable way. I suppose we will see properly in the future how this works. The stake thing I mentioned is one way to help us spot gradual performance changes over time. Another googling told me that there’s a Mann-Kendall Test + Sen’s Slope algorithm to use to identify trends in graphs like this and it can be used to plot a trend. It might work as a helper to better identify… yeah, the data trend for each test. The HTTP/2 parallel download speed trend at a specific moment Developing This setup has only existed for a few days. There is lots to do, lots to learn and much more to experiment with. Your comments, help and pull-requests will be appreciated!

0 views
マリウス Yesterday

Recovering BIOS Firmware on the Star Labs StarBook

As I described in my latest quarterly update , a perfectly routine firmware update managed to turn my Star Labs StarBook Mk VI (AMD) into an expensive paperweight. I had simply copy-pasted the one-liner from Star Labs ' official documentation , the script did its thing for about half a minute, shut the device down, and from that point on the StarBook refused to boot. Black screen, keyboard backlight on, the power LED lit, and the speakers occasionally producing a clacking sound. Sadly no amount of the usual turn-it-off-and-on-again rituals or battery disconnects brought it back. The only way out of this situation is to re-flash the BIOS chip externally using an SPI programmer. Star Labs do document this , however their guide assumes you’re using their programming kit together with a dedicated debug board and an FPC cable. That kit is significantly more expensive than a generic programmer, doesn’t list any make or model information, and, at the time of writing, has been permanently out of stock on their web shop. Not exactly helpful when you’re stranded somewhere with a dead laptop that happens to be the only computer you have with you. The good news is that you don’t need any of that, at least for this specific model of the StarBook . As Star Labs ’ own Sean pointed out in the GitHub issue I opened while debugging this mess, this specific StarBook uses a SOIC-8 flash chip, which means you can recover it with a cheap, generic CH341A programmer and an ordinary SPI clip, as long as you respect its voltage. Warning: Flashing a BIOS chip externally can permanently destroy your device if you do it wrong. The flash chip on the AMD StarBook runs at 1.8V and you must use a 1.8V adapter. Driving it at the CH341A ’s default 3.3V risks damaging the chip, and won’t read it correctly anyway. Everything below is what worked for me, documented to the best of my knowledge, but you’re doing this entirely at your own risk. The flash chip on my StarBook Mk VI (AMD) , which I could read off the silicon once I had the backplate off, is a Winbond 25R128JWSQ , a SPI NOR flash in a SOIC-8 package. The suffix on Winbond parts apparently denotes the 1.8V variants. The ubiquitous, three-dollar CH341A “black” programmers that you’ll find on AliExpress , Amazon , and pretty much everywhere else operate their SPI lines at 3.3V (and the parallel header at 5V). Clamp one of those directly onto a 1.8V chip and, best case, reads garbage. Worst case, however, you cook the flash or something downstream of it. The fix is a small 1.8V adapter board (essentially a level shifter with a voltage regulator) that sits between the CH341A and your SOIC-8 clip. These are sold as kits, e.g. the KOOBOOK CH341A Programmer + 1.8V Adapter combo that Sean linked in the issue. Make sure whatever you buy explicitly mentions 1.8V. You will need a CH341A programmer with a 1.8V SOIC-8 adapter, a SOIC-8 test clip (the spring-loaded “Pomona-style” clips, or the cheaper ribbon-cable variety, both work), a second computer running Linux (can be via a live medium, e.g. a USB stick) to drive the programmer from, e.g. a department store laptop and a Fedora live USB will do, if you’re eloquent enough to explain to the staff that you’re definitely not building what almost certainly looks to them like a bomb. You will also need the correct firmware image for your model (more on that below), a small Phillips screwdriver and, ideally, a plastic spudger. Power everything off and unplug the charger before you start. Flip the laptop over and remove the backplate by undoing the two long Phillips screws in the top corners first, and then the eight shorter screws around the edges. Lift the plate off carefully. Then, remove the five screws holding the battery in place (one of the screw positions is intentionally left empty) and gently unplug the battery connector. Last but not least, locate the SOIC-8 flash chip on the mainboard. It’s the little eight-legged Winbond chip described above. Note: While I had the StarBook open, I noticed that my (barely two year old) battery had started to visibly bulge, so do take a moment to inspect yours. A swollen lithium battery is a fire hazard and should be replaced. SOIC-8 flash chips have a defined pin 1, and the clip’s pin 1 (usually the wire on the red edge of the ribbon) has to line up with it. Get the orientation wrong and the chip simply won’t show up. For reference, the pinout of the Winbond SOIC-8 flash is: You don’t have to wire any of this up by hand, though, as the clip and the 1.8V adapter carry all eight lines for you. The only thing you need to get right is aligning pin 1 of the clip with pin 1 of the chip. Note: On my chip there is a gray dot painted onto the package, on the corner opposite to pin 1. Pin 1 is instead marked by the small indented (etched) dot, on the exact opposite side from the painted one. I have no idea why the gray dot is there, but if you align to it you’ll have the clip on backwards. Look for the indentation, not for the gray spot if yours has one too. With the clip attached, plug the CH341A into your second machine. A quick look at should confirm it enumerated: Install if you haven’t already: Before writing anything, make sure can actually talk to the flash over your clip: If everything is seated correctly, will identify the Winbond chip (detected as something like ). If instead you get: …then don’t panic. In my experience this is almost always poor clip contact rather than a real problem. I had to wiggle and reseat the clamp a few times before the chip showed up reliably, because those cheap clips are fiddly. Only proceed once the chip is detected consistently across a couple of runs. Even if the firmware is bricked, it’s good practice to take a backup before you overwrite anything. Read the chip twice and compare the dumps to be sure your contact is solid: If the two reads differ, your clip contact is flaky and you should reseat it and try again. As for the firmware image, Star Labs publish their firmware in a public GitHub repository . For external programming you want a full SPI image, not the EFI/ updater files. For my StarBook Mk VI (AMD) (product SKU ) that’s the image. The full-image files also live under the model’s directory . Pick the one that matches your model and rename it to something convenient, e.g. . Note: Star Labs ’ firmware versioning is, to put it mildly, a mess. As of writing, the last AMI (the original “BIOS”) release for the AMD StarBook is , while onwards is Coreboot . Whichever you decide to flash, just make sure it’s a full image for your exact model. Last but not least, write the downloaded image using the command: By default will erase, write, and then verify the chip. Star Labs ' official command appends (i.e. and ) to skip those verification passes, but I’d recommend leaving them off so confirms the write actually stuck. Either way, do not disconnect or disturb the programmer while it’s working. Once it finishes successfully, remove the clip, reconnect the battery, screw the backplate back on, and try to boot. When I powered mine back on, the StarBook came to life again, only to stop at a screen complaining about a missing boot entry, since flashing a fresh image also wipes the EFI boot variables. That’s nothing dramatic and you just need to point the firmware back at your bootloader. You can either use the boot menu and pick your SSD, which usually re-adds the boot entry, or boot a recovery/live system and run (this is what I did), or drop into the EFI shell and launch your bootloader manually: Note: On newer Coreboot releases Star Labs are enabling Rom Armor and anti-rollback. On the AMD board external flashing and downgrading still worked for me on , but this is expected to be locked down from onwards. What frustrates me most about this whole ordeal isn’t that a firmware update can go wrong, because that’s always a risk when you flash something. It’s that Star Labs ’ documented recovery path depends on a proprietary kit that nobody can actually buy, when a generic CH341A with a 1.8V adapter seemingly does the job just fine. However, this info is nowhere to be found in Star Labs ’ official documentation, which is why I decided to publish this write-up to begin with. Hopefully it spares the next person the day (and the stress) it cost me.

0 views
Kev Quirk Yesterday

On Publishing Your Salary

One of my favourite things to do online is perusing the IndieWeb . It's fun, and I've been able to discover a tonne of interesting people along the way. But every now and then, I come across someone who has published their salary history on their site. At first, I thought this was a great idea. It feels transparent, and it theoretically gives people an idea of what they could or should be earning in a particular field. But as I've discovered more people doing this (for the record, I've only seen it a handful of times), a pattern seems to have emerged that makes me think it might actually have the opposite effect. Before we go any further, I want to state for the record that I have nothing against people publishing their salary. If you want to do it, that's entirely your call. Personally, I've decided not to post it here, but I am open about it - if you want to know what I earn, email me and I'll gladly tell you. Discussing pay here in the UK isn't as taboo as in other parts of the world, either. We actually have legislation in place that allows us to talk about our salaries with co-workers. So what's my concern? Of the handful of times I've seen people sharing their salary history, they are always very high. For context, a six-figure (£100,000) salary in the UK puts you in the top 4% of earners in the country . This is where my concern starts - we only ever see these high outliers, so the sample size is incredibly skewed. Nobody is rushing to publish their £25,000 salary history, which means the curmudgeon in me can't help but feel that these pages can sometimes read more like a humble brag than genuine advocacy, even if the author's intent was entirely pure. For the reader, this can be deflating, rather than empowering. A developer earning a perfectly respectable £50k (which already puts them in the top 15% of UK earners) might look at Jane Smith earning £105k and think, "Jesus, I'm doing terribly." But salary is highly subjective. It depends on where you live, the company you work for, the unique responsibilities of your role, what your peers earn, how hard you negotiate, and how supportive your manager is. When you strip all that context away and just publish a high number, what is the reader supposed to do with it? If our developer goes to their manager and asks for a 100% pay rise because they saw Jane Smith online earning £105k, they're going to be laughed out of the room. Because, without context, the raw numbers are useless. I'm all for paying people the maximum amount possible for good work, but unfortunately, we live in a world where many organisations want to pay their staff the minimum they can get away with. I don't know what the perfect answer is, but I don't think it's telling people to stop sharing. Transparency is important, and it's severely lacking in the corporate world. Perhaps the solution is context? If you feel comfortable publishing your salary, maybe add a disclaimer explaining the unique circumstances behind it, stressing that it's a record of your personal journey, not an industry target or a standard benchmark. Or, maybe just ignore everything I've said in this post, as who am I to tell you what to do with your site. 🙃 I've thought about adding a salary page to this site many times, but I've always held off. Honestly, I think I'm grossly overpaid for what I do, especially compared to roles like nurses, police officers, or members of the armed forces. I'm incredibly grateful for it, and I've worked hard to get here, but I don't want to add to the noise. If you publish your salary on your site, more power to you. But it's worth considering how that number might be perceived by the people reading it, and whether it's truly helping them or just making them feel left behind. Thanks for reading this post via RSS. RSS is ace, and so are you. ❤️ You can reply to this post by email , or leave a comment .

0 views
matklad 2 days ago

Printing Lists

To print a comma-separated list, a concise idiom is to optionally print the comma first, before the element:

0 views
NULL on error 2 days ago

As gambiarras mais impiedosas que já fiz: parte 2

Dando continuidade à parte 1 . Antes da gambiarra da parte 1, eu trabalhava, em meu segundo emprego, com mídia indoor. Para quem não sabe, mídia indoor são aquelas televisões informativas instaladas em elevadores, supermercados e outros locais com média ou grande circulação de pessoas, que também exibem propagandas de vez em quando. Quando cheguei lá, eles já tinham um sistema que funcionava muito bem. Era um player de vídeo que usava o MPlayer ou, às vezes, uma distribuição Linux chamada MoviX . O único problema era o acesso remoto e o gerenciamento das playlists, que às vezes era feito por SSH ou exigia que alguém fosse até o local. Porém, além do gerenciamento remoto, que seria relativamente fácil de resolver, havia outro problema: A banda de internet. Naquela época, não existiam CDNs ou, caso existissem, custavam uma nota preta e eram de difícil acesso. A Amazon Web Services até existia, pois surgiu em 2006, e estamos falando de aproximadamente 2008, mas ainda era desconhecida, cara e não tinha infraestrutura no Brasil. Essa ideia de cadastrar um cartão em um serviço cujo valor oscilava estava completamente fora da realidade da minha bolha naquela época. Talvez uma VPS servisse, mas, de qualquer forma, esse não é o ponto. Nós tínhamos alguns “servidores”, também conhecidos como computadores pessoais antigos, lentos e que já não estavam sendo utilizados, além de uma conexão de 1 megabit compartilhada com todo o escritório. Naquela época, eu já era um grande fã da Valve, da Blizzard e de outras empresas que, por baixo dos panos, usavam BitTorrent para distribuir seus jogos. No instalador da Blizzard, inclusive, era possível ver detalhes como a quantidade de seeders e leechers durante o download, além da velocidade da transferência. Então, munido desse conhecimento, decidi criar um player de vídeo em C++ com uma biblioteca de torrent embutida. O player fazia polling constantemente, pois WebSocket ainda não existia. Ao receber uma playlist, ele calculava a diferença para descobrir quais arquivos precisavam ser baixados e quais deveriam ser excluídos do disco. Depois de identificar as mídias necessárias, iniciava os downloads usando torrent. Para deixar as coisas ainda mais interessantes, eu usava um HTTP seed , por meio do qual alguns dos primeiros pedaços eram baixados por HTTP, de forma aleatória. Na época, tínhamos algumas dezenas de máquinas. Como todas compartilhavam os arquivos entre si, a carga sobre a conexão do escritório diminuía bastante, enquanto os downloads se tornavam muito mais rápidos. Seria possível até saturar a conexão dos locais onde as máquinas estavam instaladas, mas, por precaução, o próprio cliente torrent limitava o uso da banda. Bem, talvez isso não seja exatamente uma gambiarra , mas foi um feito memorável da minha carreira. Espero que gostem e compartilhem as gambiarras de vocês nos comentários.

0 views
flowtwo.io 2 days ago

Building a New Blog Pt. 3

Long story short, I use a Cloudflare tunnel and self-host it on an old laptop. When I decided to rebuild this blog, one of my goals was to make it as "cloud-native" as possible. As a (mostly) static website, I concede it's sort of unnecessary since there are simpler ways of hosting a blog that don't require maintaining a backend at all. e.g. using Github Pages. But as soon as you want to do anything other than serve static content, it's easier if you have a backend you control. For example, I host some other projects on this domain and I have a custom publishing workflow for my posts—both of which require server-side logic. All that to say, I needed a backend somewhere and AWS is the cloud provider I'm most familiar with through work. But at work, it's not too often you get to try out new cloud services and stand up infrastructure from scratch. So, as with all my side projects, I took this as an opportunity to learn by doing. my odyssey through the clouds and back home The first question was where to store the data. My existing blog used MongoDB, so I stuck with that to make the migration easy. The most cloud-native version of MongoDB is AWS DocumentDB —it's a managed database service with full MongoDB compatibility. The pricing page says: With Amazon DocumentDB, you only pay for what you use, and there are no upfront costs. Sounded good to me. Did I actually do the math on what an instance costs per-month? No. But after 1 month I found out it was lot. That's the purple bar on the stacked bar graph above. Paying $100 a month for a blog database was not in the budget. This is when I learned that using a managed database service for my blog posts is like buying a yacht for a canoe trip. Then I found out that Mongo Cloud offers a free tier with up to 512MB of storage, so I switched to that. That's plenty for blog post storage. I guess that's still a managed database service, but it's free...so I took the free yacht. Next I had to decide how to host the backend. Building the backend as a Docker image would make it the most portable and easy to migrate, so that was a requirement from the start. Naturally, the first thing I tried was AWS Elastic Container Service (ECS) , a managed container orchestration service. Just upload your Docker image and click deploy, basically. I tried that for a couple months, but just like DocumentDB it is prohibitively expensive—even at the smallest scale of deployment. It's the teal bar on the graph above. I stopped using ECS and instead just installed the Docker engine on the smallest EC2 instance available ( ). My deployment workflow is just pushing an updated image to DockerHub and then pulling and running it on the server via a Docker compose file. For awhile, I also proxied all the traffic to my blog through AWS Elastic Load Balancing (ELB) , but not because I actually needed load balancing. I wanted to use ELB to handle SSL termination and automate the SSL cert renewal for my domain name. The alternative, which I had done before, was to route directly to my EC2 instance and handle SSL traffic directly via the app. I didn't want to do this again because using certbot + Java keystores (I run a JVM backend) was a huge headache, especially when it came to automated cert renewal. But again, using a managed service like ELB is not cheap, even if you have very little traffic. It's the orange bar on the chart above. Paying an extra $10 a month just for SSL management...also not in the budget. The better solution was to install an Nginx sidecar directly on my EC2 node which can handle SSL and proxies requests to my app. There's other benefits to using Nginx too, like response caching. After adding some aggressive cache expiration to the Nginx config, very little traffic actually ends up hitting my app and database. All these optimizations dropped my hosting costs down to about $15 a month. Not bad... an acceptable cost for me to have a space to host side projects and share content. But after running this for about a year, I looked at an old laptop I had sitting under my desk and thought, why not just run it there. For a laptop running linux, all you need to do is run: And now your computer never goes to sleep. Boom—you have a server. From the networking side, I originally thought it would require paying for a static IP from my ISP, but turns out there are several good options for proxying inbound traffic to your home LAN now. I chose to use a Cloudflare Tunnel , which is completely free, and it works great. Now I don't pay anything at all to host my blog*, and I still have the flexibility to run and deploy anything I want on it. It's kinda funny since the laptop is where I did most of the development for this blog. Now it's left the cloud and come back to the same machine. you shall return from whence you came Even though the premise of AWS is "elastic" services that scale from 0 to infinity, there's generally a minimum level of traffic needed to make the operational benefits of their managed services worthwhile, financially. For some of the services, the pricing doesn't actually scale down to the level of a personal blog in terms of resource and traffic requirements. Pro-tip: utilize the free tier from cloud providers for small projects! There are tons of ways of hosting something like this for free. And a million ways to do it the wrong way, as I showcased above. For example, DocumentDB was costing me $100/month. DynamoDB would've been free for the same use case. Self-hosting in 2026 is much more viable now thanks to cloud networking tools like Cloudflare Tunnel, Tailscale, Wireguard, or free VPS providers. You don't need to open up ports on your home network to the internet or pay for a static IP. * Alright, technically the laptop consumes about 6kWh of electricity a month, which costs me ~$0.80. Free-ish. Even though the premise of AWS is "elastic" services that scale from 0 to infinity, there's generally a minimum level of traffic needed to make the operational benefits of their managed services worthwhile, financially. For some of the services, the pricing doesn't actually scale down to the level of a personal blog in terms of resource and traffic requirements. Pro-tip: utilize the free tier from cloud providers for small projects! There are tons of ways of hosting something like this for free. And a million ways to do it the wrong way, as I showcased above. For example, DocumentDB was costing me $100/month. DynamoDB would've been free for the same use case. Self-hosting in 2026 is much more viable now thanks to cloud networking tools like Cloudflare Tunnel, Tailscale, Wireguard, or free VPS providers. You don't need to open up ports on your home network to the internet or pay for a static IP.

0 views