Posts in Cloud (20 found)
ptrchm 2 days ago

Postgres Backups to S3 with WAL-G and Kamal

The Kamal setup guides I found online focus on S3 backups using . You don’t want that for a production database. A better solution is to set up your Postgres database for Point-In-Time Recovery (PITR) using WAL-G or pgBackRest. This means your database is continuously archiving WAL segments to an S3 bucket (roughly every 60 seconds), so you can restore to any point in time. With LLMs, it’s not that hard to set up. This quick guide focuses on WAL-G, because I’ve found it to be a lot easier to set up than pgBackRest.

0 views

Lessons Learned from CISA’s Recent GitHub Leak

The Cybersecurity and Infrastructure Security Agency (CISA) has issued a postmortem on a recent data leak in which a contractor published dozens of internal CISA credentials — including AWS Govcloud keys — in a public GitHub repository for almost six months before being notified by KrebsOnSecurity. Experts say the gaps identified in the agency’s initial response provide important lessons that all security teams should absorb. On May 15, 2026, the security firm GitGuardian asked for help in notifying CISA about the existence of a public GitHub repository called “Private CISA” that included 844 MB of sensitive CISA-related data. One of the exposed files, titled “importantAWStokens,” included the administrative credentials to three Amazon AWS GovCloud servers. Another file — “AWS-Workspace-Firefox-Passwords.csv” — listed plaintext usernames and passwords for dozens of internal CISA systems. CISA quickly acknowledged our initial alert, but took more than 48 hours to invalidate the AWS keys and many other important secrets leaked in the GitHub repo. In its report on the data leak , CISA said the complexities of the agency’s systems and interconnections with federal and industry partners caused its key rotation to take longer than anticipated. “Drawing on this experience, CISA encourages others to maintain mature and well-tested key management capabilities,” the report notes. CISA also admitted it can do better when it comes to responding to security incident notifications from external parties. The postmortem stresses that clear and distinct reporting channels are essential to ensure that incidents affecting the organization itself are handled differently from those involving its products or customers. “In CISA’s case, these channels were not well defined, leading the security researcher to try multiple avenues – including emailing the contractor, submitting through CISA’s vulnerability disclosure platform (which is intended for vulnerabilities impacting the broader cybersecurity community), and ultimately involving a reporter,” reads the analysis written by Preston Werntz and Brad Libbey , the acting chief information officer and acting chief information security officer at CISA, respectively. CISA said it is refining its reporting channels to make them easier and faster for researchers. “Additionally, while many researchers rely on the security.txt file, organizations can ensure clarity by publishing reporting instructions in multiple prominent locations,” the CISA authors wrote. Guillaume Valadon , the GitGuardian researcher who first contacted KrebsOnSecurity about the exposed CISA credentials, said CISA ignored nine automated alerts about the exposed credentials prior to our notification on May 15. Valadon’s company constantly scans public code repositories at GitHub and elsewhere for exposed secrets, automatically alerting the offending accounts of any apparent sensitive data exposures. “Letting nine notification emails go unanswered is how a one-day incident becomes a six-month exposure,” Valadon wrote in an analysis of CISA’s report. “Make it trivial to report a leak about you, not just about your products. The person reporting a leak to you is not the threat. Publish a security.txt , but do not stop there. Put reporting instructions in several prominent places, and make sure a report about your own infrastructure does not land in a product-bug queue.” The report’s authors also emphasized the importance of continuously scanning public code repositories like GitHub for exposed secrets, and said CISA has since rotated all secrets and created an action plan to improve management of developer secrets and to better monitor for them going forward. The report notes that while CISA had developed a playbook for responding to cybersecurity incidents, that playbook somehow didn’t include what to do in situations involving GitHub or other cloud services. Valadon said the report validates the need to scan continuously — not just quarterly — for exposed secrets. “The Private-CISA repository sat public for six months,” Valadon wrote. “Continuous monitoring of public GitHub surfaced it. Comprehensive internal scanning could have caught the plaintext passwords and committed backups long before they left the building.” CISA gave itself passing grades on several areas of security preparedness that it said helped the agency gauge the scope and impact of the exposed secrets, including enhanced logging capabilities, and the adoption of zero-trust principles in both its production and development systems. CISA said those detailed logs allowed it to show that no customer or mission data was exposed, and that the leaked credentials were not used outside of CISA’s environments. The agency said the contractor who exposed the secrets had their system access revoked. Valadon reckons the biggest takeaway is the CISA postmortem itself, and praised the agency for being transparent about what worked and what didn’t. “To my knowledge, it is also the first time a national cybersecurity agency has publicly advocated for secrets scanning and for simplifying relations with security researchers,” Valadon wrote. “That is exactly the incident communication we should expect from every organization.”

0 views
Unsung 1 weeks ago

“Nothing at all like the bloated app that Dropbox’s Mac client has grown into”

John Gruber at Daring Fireball penned a short eulogy for Maestral: As of today Maestral continues to work just fine. I don’t know when these certificates are expiring. And I don’t know what I’m going to do when they do. Dropbox enshittified its app – my friend joked once that Dropbox is a rare example of a company that pivoted away from a product-market fit – and it seems Apple’s API changes didn’t really help, either. Maestral stepped in to help restore the minimalistic, functional core of Dropbox – I believe Doctorow terms this “disenshittification” – but it was helmed by one person, Sam Schott, who has every right to move on to other things. #enshittification #software eulogies #third party fixes

0 views

Enabling Packet Spraying over Commodity RNICs with In-Network Support

Enabling Packet Spraying over Commodity RNICs with In-Network Support Xiangzhou Liu, Wenxue Li, Zihao Wang, and Kai Chen EUROSYS'26 This paper proposes changes to top-of-rack (ToR) switch hardware to enable packets from a single flow to utilize many network paths ( Falcon offers a similar benefit via changes to the NIC rather than the switch). The Falcon approach is robust but more invasive. The switch-based approach from this paper is a more incremental change. The sweet spot for packet spraying is a data center that has a large number of network paths compared to network flows (i.e., connections). In such an environment, there is an incentive to spray the packets associated with one flow across multiple paths. The trouble with packet spraying is that packets will commonly arrive out-of-order. The system has to be able to distinguish the out-of-order case from genuine packet loss. Section 2.2. of the paper describes three techniques for handling packet loss: PFC - the trouble is that this has scalability limits, and only addresses packet loss due to buffer overflows (not angels flying down and flipping your bits) Timeouts - the trouble is that practical timeout values have to be large Selective Repeat Selective repeat is a feature of modern RDMA NICs which is similar to the bitmaps tracked by Falcon hardware. The idea is that a receiving NIC tracks an expected sequence number (ePSN) for each flow. If a packet arrives with a sequence number greater than (but not too much greater than) the ePSN, the NIC accepts it and records this fact in a per-flow bitmap. The receiving NIC then sends a NACK to the sender, requesting that the sender resend the packet corresponding to the ePSN. In the out-of-order case, this NACK is unnecessary as the expected packet will arrive soon enough. The question asked by this paper is: can one easily modify switch hardware to filter the unnecessary NACKs? The core idea proposed by this paper is that the NICs and switches agree on the number of paths for a particular flow. Sending NICs use a packet’s sequence number to determine which path to use (e.g., . The switch can then track information for each path associated with each flow. When a NACK arrives at a switch, the switch can drop the NACK as necessary (thus avoiding unnecessary retransmissions). For example, say there is a single flow mapped to 4 paths. Packets with PSNs [0, 4, 8, 12, …] will travel over path 0. Packets with PSNs [1, 5, 9, 13, …] will travel over path 1. If packets arrive at the receiving NIC in this order: [0, 1, 5 , 4] the NIC will send a NACK when it receives packet 5. However, the switch will drop that NACK because it “knows” that no packet has been received out of order with respect to its flow. Fig. 12 has simulated performance results for collective operations common in AI workloads. is the work described in this paper. CCT is a measure of how long the collective operation took. Source: https://dl.acm.org/doi/10.1145/3767295.3803588 Dangling Pointers This feels like an engineering solution to a business problem of how to get many NIC vendors to align on a packet spraying solution. I suspect there are many applications where it would not be too difficult to introduce multiple flows. For example, in a machine learning workload, the weights/deltas associated with layer could be assigned to flow . This would increase network path utilization without any hardware changes. Thanks for reading Dangling Pointers! Subscribe for free to receive new posts. PFC - the trouble is that this has scalability limits, and only addresses packet loss due to buffer overflows (not angels flying down and flipping your bits) Timeouts - the trouble is that practical timeout values have to be large Selective Repeat

0 views
Phil Eaton 1 months ago

This week in infrastructure

This is an external post of mine. Click here if you are not redirected.

0 views

The future of Siri, or: why private inference isn’t private enough

Yesterday Apple announced a big step towards deploying real AI in their Siri ecosystem. In most ways this is good and inevitable: Siri is one of the world’s most widely-used voice agents, and it would be good if it didn’t suck. The idea that Apple would boost its capabilities with frontier models wasn’t so much a matter of if , but a question of when and who . The who turns out to be Google: Apple looks like it will use some combination of Google Gemini models, combined with Google’s Confidential Inference and Apple’s own Private Cloud Compute for private hosting. These systems will process both your queries and evaluate private data from your devices. Apple’s marketing pitches the advantages as follows: There’s some tension between these goals. Apple has addressed this by marketing a service it calls Private Cloud Compute , or PCC. PCC was introduced in 2024 as a private model inference system that ran entirely on Apple Silicon, using a set of “trusted” hardware security modules running in Apple’s datacenters. The goal of this system is to ensure that your data never leaves Apple’s hardware: it’s encrypted from your phone to a dedicated server, and then it disappears once a response reaches your phone. The stateless design of PCC ensures (in theory) that your data doesn’t linger, and the design of the hardware prevents even Apple from seeing the inputs. Apple has since “expanded” PCC to encompass Google’s hardware as well. I will confess that I find the details of the new “expanded” PCC just a bit vague. It sounds a lot like Apple is primarily going to rely on Google’s existing confidential compute (running in Google datacenters) to process this data, but they’re bolting on a new layer of technical security to control which models are actually running. In any case: security experts can argue about whether this is good enough to keep Cozy Bear away from your data. What I will grant is that it’s probably good enough to keep Google and Apple from accessing your stuff, which is what most people are worried about in the first place. So why am I so nervous? To illustrate how agents might work, it’s helpful to consider an example use case. Let’s imagine that you’re planning a business dinner for six people. This involves several subtasks: In the past, this type of scheduling required a significant amount of human effort. The beauty of AI agents is that, in theory, this is exactly the sort of project that can be automated. The agent can first scan your recent conversations to answer the questions needed for steps (1) & (2), then it can conduct the searches described in step (3). With a nod from you, it can even author the calendar invites and text messages required to complete step (4). So what’s the problem here? The first and unsurprising observation is that being useful on these tasks requires your agent to have context , which means: relatively unrestricted access to your private data . You know about your invitees’ availability because they texted it to you. You know about Mike’s allergy because you’ve talked about it with him or jotted it down somewhere. (This could mean iMessages, email, contacts, or personal notes.) Re-entering all of this data into an agent would be annoying and time consuming and the whole point of an agent is to save you time. The winning personal assistant doesn’t win just because it’s smart: it wins because it “already knows” the things you need it to know, like a personal assistant who sits next to your desk. Allow me to dig into the details just a bit deeper. The agent might scan your messages database to learn the parameters needed to schedule your dinner. Or, in a more token-efficient system, it might read your messages continuously and store a “memory” that distills useful facts that it might need later. Both can be functionally equivalent, but one produces an artifact that may be highly sensitive. And keep in mind that the set of facts that might be useful is very broad. For example, Mike’s allergy is one of those facts. But there are many others. For example, the private conversation you had where you discovered that Mike was having an affair is potentially another fact that could be stored or accessed by a system. Memory or not, this data will all be within the agent’s view, and you’ll have to hope that it knows which one to operate on. With this data at its fingertips, your agent (which is really an LLM running on a server in a data center somewhere, combined with a bunch of local state and prompting) will need to perform inference over this data, either to summarize it, or to respond to the query itself. This is where Private Cloud Compute and Confidential Inference are designed to protect you. The purpose of these technologies is to ensure that this data, and any inference results, are restricted to you alone. The inputs and outputs should be wiped as soon as inference is done, and the only remaining copy of any of it should exist on your phone. So far I find this to be a compelling story, as long as you never plan to do anything else beyond inference. An AI that performs only inference is like a human assistant that can read your private files, but is otherwise locked in a windowless room, with no Internet access and no outbound phone. Your data is perfectly safe, but your assistant is worthless for all but the simplest tasks: for example, summarizing inbound messages for your consumption, or helping draft text messages. (In short, what Apple Intelligence does today.) Now imagine a personal assistant who can actually get things done. This assistant will need Internet access: at minimum the ability to query search engines, or in the future, search LLMs like Gemini or ChatGPT. To accomplish the later steps of our task, you’ll need it to schedule public calendar invites and draft messages to share with your contacts. This assistant is now useful, but the wonderful PCC guarantees of “no private data is accessible to others” are no longer so applicable. The privacy of your data no longer depends on the design of some silicon, but rather, on your assistant’s discretion and judgement. Let’s move back to our hypothetical business dinner. To accomplish step (3) your agent will need to visit a search engine or non-private LLM, perhaps asking it several queries, each of which leaks some information about your specific requirements. The nature of the data leakage really depends on how cautious the “private” agent is in authoring its queries. A perfectly reasonable case would be for the model to simply collect a series of useful facts, and upload them all to a more powerful “open” search LLM like Gemini, ChatGPT or Claude, as follows: “ Hey, LLM search engine, here is a list of thirty detailed facts about my attendees and the purpose of this meeting, find me a restaurant that works for everyone. “ This would be an incredibly efficient (and somewhat natural) design, since the non-private LLM is most likely going to be more powerful and capable than the private one. Unfortunately, it will also reveal an absurd amount of information about your private data, including some that may not be strictly necessary to get it done. (Is Mike’s affair relevant to the seating chart?) Put differently: private inference can work perfectly, and yet valuable ( monetizable ) data can still flow outward to a public search engine or LLM, simply because the agent was programmed to do its job in a slightly non-privacy-preserving way. You probably don’t care very much if a search engine learns that Mike is allergic to Szechuan food. But there are things you really should care about. In security parlance, they both have to do with different adversaries. Let’s begin with the most obvious “adversary”. Imagine you’re Mark Zuckerberg or Sundar Pichai, or whoever runs Apple’s advertising business. You have billions of users with piles of deeply useful data stored on their phones. This data is extremely valuable for targeted advertising, something that is about to become wildly more lucrative thanks to generative AI. At the same time, a big chunk of this data is inaccessible, simply because users don’t love the idea of you scanning their private conversations. And so while you might have access to some public data (like web browsing) you can’t read those years worth of intimate private conversations that many users store on their devices. Now imagine deploying an agent to users’ phones. That agent will have access to all that data. It’ll have access to everything the user does. To do its job, it will literally need to divine each user’s preferences and then operationalize them into queries that will repeatedly hit your search engine or “search LLM”. Whoever operates this search engine will learn a vast amount of useful information about the users’ desires, some of which will come from the most intimate private conversations — even conversations that happened years ago, and that you’ve forgotten about. If the person who operates the search engine is also the person who designs the model and its prompting, then you really have a best-case scenario for data monetization. It’s hard for me to believe that the major tech CEOs are unaware of this. Some folks will shrug at the threat of Google learning more about them. I don’t subscribe to this viewpoint, but I understand it. From the outside, at least, Google has been a reasonably good steward of users’ data. To my knowledge, there have been no major data breaches where our most intimate Google searches were dumped all over the Internet (in the style of AOL ‘s search breach.) The company deserves a lot of credit for this. So while I object to the idea that Google or Meta or Apple may learn even more about us from our private data, it’s at least possible that our most intimate secrets won’t be revealed to the entire world. But this does not mean your private data won’t become public: and that’s why we need to talk about a second adversary. This adversary isn’t a search engine that your agent talks to, it’s all the other people who will talk to your agent. Simon Willison describes a condition that he calls the lethal trifecta . This occurs when you have a combination of (a) access to private data, (b) untrusted content an LLM must parse, and (c) the ability to send external communications. These together create the perfect storm for data-exfiltration attacks, where a remote attacker simply “tricks” the LLM by sending it instructions to ship out confidential data. Although LLM technology is getting better, it’s still quite common for even frontier LLMs to fall for simple prompt injection attacks in which a malicious user includes text (as part of a website or a piece of data) that causes your LLM to reveal things it should not . This problem is very much alive. Just today, OpenAI recently unveiled a “lockdown mode” feature, where ChatGPT is restricted from making web searches due to the risk that it might upload your sensitive documents. Agents like the one Apple is building (whether they use confidential inference or not) are a nightmare case of the lethal trifecta. These systems will need to ingest a vast amount of data, much of which will come from highly untrustworthy sources: think incoming emails and text messages. They will have access to everything on your system, like your encrypted messages and documents. And, to be useful, they will need to handle all sorts of actions that have visible external effects, like scheduling calendar invites and sending text messages. The result is that your private data isn’t just vulnerable to the person who controls the agent, it’s potentially vulnerable to anyone who can cause your agent to misbehave. This problem exists regardless of how well-designed the private inference engines are. And OpenAI’s recent example illustrates that it’s far from solved. It’s possible that we’ll be able to solve these problems technically, or through some careful element of human observation — read all your outbound calendar invitations carefully — but right now we have not. Or let me put this differently: if you think spam directed at humans is bad, wait until it’s spam directed at agents. So far we’ve discussed two adversaries: the misaligned designers of private agents (such as search operators), and the possibility of remote prompt injection attacks. But of course, in any discussion of technical privacy systems we need to talk about the last elephant in the room: your government . We live in a society, and that society has laws. If an agent has access to all of your data, messages and actions, then technically speaking it has the ability to detect criminal activity. That criminal activity might include sharing of CSAM, or terrorism-related activity, or it could include tax fraud or any other form of crime. These agents make a perfect one-stop shop for crime detection, since they can identify patterns of bad behavior and also report them. Is this farfetched? Well, as I’m fond of repeating on this blog, this is more or less what existing rules published by the UK’s OFCOM require for encrypted messengers, and there have been proposals in the EU Commission to do similar things. The UK also maintains a vigorous regime of Technical Capability Notices (TCNs) that allow it to demand that providers make changes to their systems, changes that could potentially affect devices worldwide. Apple is in the midst of a battle with the UK over its other encrypted services. Traditionally in the United States we’ve shied away from this sort of thing, partly because it’s creepy and mostly because it seems like a direct attack on the Fourth Amendment. With that said, the Fourth Amendment applies only to governments: in theory a private company like Apple or Google could configure their agents to report crimes to them, and then pass along the serious ones to the government. This is more or less what Apple proposed to do in 2021, when they designed a system to monitor photos for CSAM. At the risk of saying more obvious things, the difference between a helpful private agent, a corporate advertising bot, and a government spy comes down mainly to a matter of prompting, and maybe a bit of model fine-tuning. Once you combine private data access and the ability to send messages, there is essentially no technical protection that private inference alone can offer. For decades the point of cryptography has been to remove trust: to replace “I promise not to look” with “I can’t.” Private inference is the most ambitious version of that promise. Against the adversary it was designed for — the provider who performs the inference itself — I believe that it probably does what it says. All I’m trying to say in this post is that this adversary is a very small piece of any agentic system. The adversaries we care about are the ones that deal with the model directly, or even the ones who designed the model or specified its technical requirements. There is no cryptographic primitive that protects you from “upload your search facts to Google” or “report anything suspicious to the government because I programmed you that way.” That protection, if it exists at all, lives in law and politics and corporate incentives: the exact messy human institutions that cryptography was invented to let us stop trusting. First, since your phone already has context about you — meaning, your private information, schedules, email, text messages — an AI-enabled Siri can potentially offer more useful answers to your practical requests than external LLMs. Want to schedule a reservation for next week’s birthday party? In theory, a future Siri-AI might already know who’s coming, and what kind of cake they like. Of course, what Apple calls “ context ” is also the raw data of your life. This is deeply private data from all of your apps, and that data can’t just be shipped to random adtech companies (or Sam Altman) for processing. Your context needs to be protected, and Apple bills itself as a privacy company. You need to juggle the participants’ schedules, know when they’re in town and available to meet. You need to choose the appropriate restaurant based on menu and location. This might depend on what you know about the participants’ preferences: Mike is wildly allergic to szechuan peppercorn, for example, which rules out quite a few options. With these time/cuisine/location constraints in place, you’ll need to search for a restaurant that actually has a table for six in the right place. Finally, you’ll need to book the reservation, mark your calendar, and alert your attendees.

0 views
Stratechery 1 months ago

The iPhone’s Last Stand

Listen to this post : Apple fans would, for years and years, sneer at Microsoft’s penchant for talking about products that may or may not ship, deriding them as vaporware. After Apple’s bungled 2024 launch of Apple Intelligence and new Siri , however, vaporware is fair game, and just in time for this Article. Last week, at its annual Build developer conference, Microsoft put forth a vision for a new ecosystem of hardware devices under the banner of Project Solara : The concept — which isn’t entirely clear from that video, but was more fully explained on stage — is that in the future you will be surrounded by an ecosystem of devices, none of which stand alone, but are more like portals to interact with your agents, which live in the cloud. In other words, as I wrote in February, Thin Is In : This is even clearer when you consider the next big wave of AI: agents. The point of an agent is not to use the computer for you; it’s to accomplish a specific task. Everything between the request and the result, at least in theory, should be invisible to the user. This is the concept of a thin client taken to the absolute extreme: it’s not just that you don’t need any local compute to get an answer from a chatbot; you don’t need any local compute to accomplish real work. The AI on the server does it all. I made the case in that Article that server-side inference would dominate AI workloads, thanks in particular to increasingly high memory demands for agents. What I found intriguing about Microsoft’s vaporware, however, is that it showcased a use case wherein this thin client approach was compelling for reasons beyond KV cache. Specifically, for most of tech history computing has been indistinguishable from interacting ; that’s why we place so much value on new input methods, as they often set off new paradigm shifts. By the same token, the problem with wearables as the paradigm beyond the iPhone is that interacting with them generally sucks. Sure, you can imagine a future where voice interaction is completely seamless or where a device can “see” what you see, but anything longer than a few seconds is much less convenient than simply swiping on your phone. Agents, however, compute on your behalf, without any interaction necessary: a few seconds is all you need to get work done for hours — at least in theory. Apple, a company that can actually make devices, was under heavy scrutiny going into yesterday’s WWDC keynote for a different concern: can the company make AI? And, if your standards are the state of the art in AI circa June 2024, when Apple took their first crack at answering the question, they did quite well. The company’s pre-recorded keynote took great pains to show actual demos — spinning indicators and all — and they worked! Here was the first one of what Apple is calling “Siri AI”: What’s fascinating about this specific demo is that it also showed just how far behind Apple is. New head of Siri Mike Rockwell successfully used Siri to set a reminder to enter a lottery for concert tickets, demonstrating context awareness and the ability to interact with the Reminders app through Apple’s App Intents framework; what would have been state of the art would have been asking Siri to enter the lottery on his behalf when the time came. In other words, to act outside of the interaction paradigm that has traditionally defined computing, and which Apple has dominated. At the same time, the fact that Apple is behind the state of the art might not matter that much given Apple’s market and opportunity in that market. To start with the former, Apple is targeting consumers, for whom traditional chatbot functionality is probably sufficient for the vast majority of their AI needs. Siri will be able to give you recipes, tips on do-it-yourself projects, or generate images. Moreover, the fact that Siri will have access to your iPhone gives it all of the same advantages that made me optimistic about Apple Intelligence in the first place. From an Update after that initial June 2024 launch : The key part here is the “understanding personal context” bit: Apple Intelligence will know more about you than any other AI, because your phone knows more about you than any other device (and knows what you are looking at whenever you invoke Apple Intelligence); this, by extension, explains why the infrastructure and privacy parts are so important. What this means is that Apple Intelligence is by-and-large focused on specific use cases where that knowledge is useful; that means the problem space that Apple Intelligence is trying to solve is constrained and grounded — both figuratively and literally — in areas where it is much less likely that the AI screws up. In other words, Apple is addressing a space that is very useful, that only they can address, and which also happens to be “safe” in terms of reputation risk. Honestly, it almost seems unfair — or, to put it another way, it speaks to what a massive advantage there is for a trusted platform. Apple gets to solve real problems in meaningful ways with low risk, and that’s exactly what they are doing. Apple actually made this version of Siri much more capable in terms of accessing world knowledge and image generation, which should make the experience much more seamless, but the real differentiation will clearly be that access to your personal information. You can ask Siri about something you received in messages — or was it email, or a voicemail? — and it will actually find what you’re looking for; it can also “see” what you are looking at on your screen, and act on the information. And, to the extent that third-party apps offer up their data to the Spotlight semantic index, and make actions available via App Intents, Siri can actually operate across different services in a way other AIs can not, at least without making massive sacrifices in security on a local Mac or PC. These capabilities are genuinely useful, and there’s a good chance they’re enough, at least for now, and that’s because there is another aspect of the consumer market that is worth considering — beyond the fact that billions of consumers already have iPhones. Specifically, consumers don’t want to work, and don’t really care about being productive. This reality about the consumer market is a lesson that Silicon Valley has to re-learn every decade or so. Consider Dropbox, whose founder, Drew Houston, is in the process of stepping down . Dropbox was a category-defining product that had a viral hook — if someone signed up with your referral code, you got more storage — and grew extremely fast amongst consumers; the company then spent too long trying to actually build a business in the consumer space, before finally realizing that the only way to make money with what was ultimately a productivity product was by selling to enterprise. The reason is obvious when you think about it: enterprises are paying for their employees’ time, so of course they are willing to pay for tools that make those employees more productive; consumers, on the other hand, are mostly looking to waste time, which is why attention-harvesting advertising is the only software business model that works at scale for consumer services. The fact that Silicon Valley forgets this is downstream from Silicon Valley being a bubble; normal people aren’t looking for agents to buy them tickets to a concert. Still, the bubble was strong enough to convince OpenAI to make the exact same mistake Dropbox did: the company somehow convinced itself that it could make enough money selling subscriptions to consumers; Anthropic, meanwhile, realized that it was enterprises who were willing to pay for AI’s massive productivity benefits, even as OpenAI failed to capitalize on their consumer market penetration by refusing to build an advertising product . This is a long-winded way of saying that I don’t think that Apple’s agentic shortcomings are a big deal, at least for now. Agents help you do work and be more productive, and consumers don’t want to work or care about being productive. What they do want to do is watch short-form video, and an iPhone is simply much better at that than any other device ever will be; in that context, Siri being good enough is enough, and it appears that Apple crossed that bar. There are actually a lot of interesting technical details about how Apple rebuilt Siri, including expanding Private Cloud Compute to include Nvidia chips running in Google data centers, as well as a 20 billion parameter on-device mixture-of-experts model that selects the expert on a per-query basis (as opposed to on a per-token basis) so that it can run in an iPhone’s limited memory. The key strategic takeaway of these implementation details, however, is the centrality of the iPhone. Microsoft’s Project Solara obviously makes sense for Microsoft given the fact that the company missed out on mobile, but it also fits with the infrastructure of AI, which is in the cloud, and increasingly about compute happening without a human in the loop. Apple, in contrast, is heavily incentivized to preserve the iPhone’s importance, and by extension, to focus on use cases organized around human interaction. However, it’s too simplistic to reduce these approaches to a cynical analysis of incentives; both make sense in their own right. What makes me intrigued about Project Solara is the fact that Microsoft is positioning it as purely an enterprise play, which is important because an enterprise has context about the work being done, making it more viable to build long-running agents — which the enterprise is willing to pay for. That context would be far more difficult to build for consumers, given the need to tie together a huge number of services to get a coherent set of data over which to operate. Indeed, the only entities that can probably pull that off are Google and Apple via Android and iOS, respectively — and Google is always going to be focused more on its cloud services as the point of integration instead of the device. That leaves Apple as the only company truly — dare I say it? — thinking differently. And yes, the iPhone as the true core of Siri (which will work across your devices, but get its differentiated context first-and-foremost from your iPhone) just so happens to perfectly align with Apple’s business model and desire to not spend billions in capex, but that doesn’t mean it’s the wrong approach. You’ll be able to access all of that capex that other companies are building on your phone, you’ll just have to use an app; if you need to find something personal, or work across apps, Siri will be the only one who can pull it off — as long as it’s not vaporware (and it appears the second time is the charm).

0 views
Xe Iaso 1 months ago

Giving your Go apps Tigris superpowers

Tigris is S3-compatible, which means you can point the AWS SDK at it and most things just work. The catch is that the Tigris-exclusive features—bucket forking, snapshots, object renaming, and the like—need verbose workarounds because the AWS SDK doesn't know they exist. So we wrote a Go SDK that does. It comes in two flavors: the package is a drop-in replacement for the standard S3 client with first-class methods for the Tigris-specific operations, and is a higher-level client for the common single-bucket case that infers its configuration from the environment so you stop passing the same parameters over and over. You can adopt the Tigris features incrementally without refactoring your existing S3 code, and the simpler API still works against other S3-compatible providers. I wrote up how it works and why we built it over on the Tigris blog.

0 views
Martin Alderson 1 months ago

xAI is looking more like a datacentre REIT than a frontier lab

An unexpected development over the past few weeks is xAI's new partnerships with Anthropic and Google, providing them with a huge amount of capacity. It's worth remembering that xAI is now part of SpaceX, after the two merged back in February - so the revenue from these deals flows straight into the entity about to go public. While much has been made of the potential financial engineering given SpaceX's upcoming IPO, I think there's a bit more to this than just pure accounting tricks. If you use Claude products much, you'll be (very, probably) aware that Anthropic has had serious capacity problems, especially early afternoon onwards in Europe and in the mornings in the US (this is when demand seems to be highest as both European users and the Americas are both at work, fighting for capacity). I've written about this compute crunch before a few times - the coming crunch , whether it's here yet , and what comes next . This resulted in Anthropic having to introduce new peak hour restrictions on their subscriptions, with usage between 5am–11am PT / 1pm–7pm GMT using more of your usage limit - with the aim of smoothing demand between peak hours and off peak hours where they had more capacity available. However, there is only so much demand shifting you can do when demand is growing as fast as Anthropic's. At some point you end up having to ration users further, which definitely is far from ideal when you have both Google and OpenAI breathing down your neck for customers. At the start of May, xAI announced a partnership with Anthropic to provide access to their (older) Colossus 1 datacentre in Memphis. This allowed Anthropic to reverse the usage limit restrictions on their subscriptions, and in general while stability of Anthropic services still leaves a lot to be desired, the peak time crunch has abated (for now, at least). The fees involved are enormous, ramping to $1.25bn/month for 300MW of capacity - approximately 220k GPUs. Last week, Google announced a similar partnership - $920mn/month for 110k GPUs [1] . It's important to note that both agreements have cancellation clauses - allowing either party to cancel with 90 days' notice after an initial lock-in period. If you take this on face value, this is a ludicrously profitable deal for xAI: While this doesn't include opex [2] and depreciation, if the deals continue for 18 months, xAI recoups all the capex they spent and still has many hundreds of MW of GPUs available. With the giant compute shortages likely to persist into the medium term, even older H100s are likely to be extremely useful even 18 months out. It's important to note there are certainly some red flags with the deal. Firstly, Elon Musk and OpenAI were/are locked in a bitter legal battle, and the Anthropic deal could be motivated to add pressure to OpenAI more than commercial reality. And Google is a major shareholder in SpaceX, so they certainly have incentive to juice the valuation of the IPO. While I'm sure there is some degree (potentially a lot!) of truth in these viewpoints, it's important to note that huge volumes of GPUs are in enormously short supply. One of the untold stories of this capex boom in datacentres is just how behind all of them are. Even OpenAI's flagship Stargate UAE datacentre - being built in a jurisdiction that is renowned for a laissez-faire attitude to building regulations - is now under direct threat from the current Iran conflict, with Iranian drones having already hit other UAE datacentres . In comparison, SpaceX/xAI are incredible at building datacentres on time. The original Colossus 1 datacentre was built in 122 days. Musk's empire does have a huge advantage in really understanding how to plan, build and execute enormous infrastructure projects quickly. While the hyperscalers no doubt have the experience to do this, they were built with far less urgency - with typical project execution taking many years. Given the capex only really started to ramp up in the last couple of years, many of these projects are still years away. This gives xAI a serious competitive advantage that shouldn't in my opinion just be hand waved away. There is no doubt this leaves Grok in an odd spot, with a lot of the datacentre capacity that was destined for Grok training and inference now being leased to a direct competitor. While it's foolish to write off any model provider, it certainly looks like a serious retreat from Grok vying to be a frontier class lab. But, perhaps, they over-specified their datacentre capacity - there is no doubt that inference demand for Grok models is likely to be seriously behind projections, leaving a bunch of spare capacity which might as well be monetised while the training lottery continues? It's hard to say and the xAI & Cursor deal muddies the water even further. As such, I think all three things are true to some degree. There's no doubt some level of financial engineering going on. There's also an enormous compute shortage. And it seems to me SpaceX/xAI does have a real competitive advantage in datacentre buildout. It's just the magnitude of how true each of these are is going to define the success or failure of the biggest IPO in North American history. Either way, the more I look at it, the more xAI is starting to resemble a datacentre REIT with a frontier lab attached, rather than the other way around. I suspect that these are likely to be GB200s given the pricing, vs the mostly H100/H200 for Anthropic, but this is speculation on my part. ↩︎ Power is the obvious big opex line, but at this scale it's almost a rounding error. 300MW running flat out is roughly 300,000 kW × 8,760 hours, or about 2.6 billion kWh a year. Tennessee has some of the cheapest industrial electricity in the US at around 6 cents/kWh , so buying it off the grid would cost somewhere around $160mn a year. Colossus actually runs largely on its own on-site gas turbines, which comes out even cheaper: at a simple-cycle heat rate of ~10,000 Btu/kWh and Henry Hub gas at ~$3.50/MMBtu , the fuel bill is only around $90mn a year. Either way, set against the ~$15bn a year Anthropic is paying for that 300MW, power is no more than about 1% of revenue. The deal value utterly dwarfs the running costs. ↩︎ I suspect that these are likely to be GB200s given the pricing, vs the mostly H100/H200 for Anthropic, but this is speculation on my part. ↩︎ Power is the obvious big opex line, but at this scale it's almost a rounding error. 300MW running flat out is roughly 300,000 kW × 8,760 hours, or about 2.6 billion kWh a year. Tennessee has some of the cheapest industrial electricity in the US at around 6 cents/kWh , so buying it off the grid would cost somewhere around $160mn a year. Colossus actually runs largely on its own on-site gas turbines, which comes out even cheaper: at a simple-cycle heat rate of ~10,000 Btu/kWh and Henry Hub gas at ~$3.50/MMBtu , the fuel bill is only around $90mn a year. Either way, set against the ~$15bn a year Anthropic is paying for that 300MW, power is no more than about 1% of revenue. The deal value utterly dwarfs the running costs. ↩︎

0 views
Stratechery 1 months ago

An Interview with Microsoft CEO Satya Nadella About Finding Core Competencies

Listen to this post: Good morning, This week’s Stratechery Interview is with Microsoft CEO Satya Nadella . I have previously interviewed Nadella in May 2024 , October 2022 , April 2020 , and May 2019 . As I noted yesterday , I spoke to Nadella shortly after the conclusion of his keynote at Build , Microsoft’s annual developer conference . One notable thing about the keynote was the fact that Nadella was — outside of product demos — the sole presenter; one gets the sense he has shifted into a much more hands-on role at Microsoft over the last year. The reasons why are clear: my first question to Nadella was if he was happy about where Microsoft was currently positioned as a company. We talk about the reasons for that question, the status of the company’s partnership with OpenAI, and whether Microsoft has invested sufficiently in AI infrastructure. Then we talk about the future of software, Microsoft’s business model in the age of AI, and if they can operate independently from the leading edge models. At the end we talk about Project Solara and whether Microsoft will ever pay residents to build data centers. One note, with regards to a misunderstanding towards the end of the interview: there is no documentation I could find about being able to use Copilot Cowork with non-Anthropic models; Microsoft’s own documentation fits my understanding. As a reminder, all Stratechery content, including interviews, is available as a podcast; click the link at the top of this email to add Stratechery to your podcast player. On to the Interview: This interview is lightly edited for clarity. Satya Nadella, welcome back to Stratechery. SN: It’s great to be with you, Ben. So first off, I don’t know if you realize this, but at least according to my daughter, the defining word for the real grinders in Gen Z — first off, LinkedIn is like the social network. SN: That’s great! Number two, the word they all use is “build”, “I’m building, I’m building”, so who knew when I was at the first Build, I think, in 2010? Or was it 2011? Who knew you were such a trendsetter? SN: (laughing) There you go, I’m thrilled that your daughter is building and is on LinkedIn. Yeah, well, I’m not sure if she’s on there, she’s more making fun of people, so we’ll see how it works. We last talked the summer of 2024 after Build, this was up in Seattle. To say a lot has changed since then is an understatement. I had a bunch of questions I wanted to ask you about the business as a whole, things going on, I’m going to start with those, then I have questions about the presentation at the end. But relative to that, I want to ask you one simple question: Are you happy with Microsoft’s current competitive position? SN: You know, always this is the trickiest thing, you can sit here and say, “I’m happy” — that means you’re not ambitious enough and when you say, “If you’re not competitive, what the heck are you doing?”. And plus you have like 57 different product lines. SN: I’d say the thing in these platform shifts in particular is to, one, get the conceptual model of, “Where is the opportunity for us as a company?” — most people measure competitive position as if it’s a complete zero-sum game, and it’s never been the case. Which is, it is not the case with the cloud, it is not the case in client-server, and so to me, “What is Microsoft uniquely capable of doing in this new world” — that’s the key thing that we have to answer before we even get to the competitive position. In that context, “What is it that we really have a shot at?”, which is we can be a trusted purveyor of a platform, which is what we’ve always done, that allows people to create more value on top of a platform, which is again the DNA we have. Even in a world where these frontier models seem to have no limit— A very large appetite. SN: They have large appetite. That is what I feel even this Build , this conference, we are at that state where we can now really turn this from any one frontier model to saying, “Hey, there is actually a way for a frontier ecosystem to emerge where there are many stakeholders who all actually are operating with their own frontier intelligence”, that is a place where I think we have a unique shot, a unique competitive angle, and most importantly, brand permission. This is the other thing I’ve learned, Ben, which is every company thinks they can do everything, and then they realize that the world doesn’t need them to, the world wants them to do the one thing. Is that a lesson that you had to learn? SN: Yeah, absolutely. I’ve always said this, at Microsoft we are at our best when we do what the world expects us to do, we are at our worst when we do things out of envy, which is just because somebody else had some cool hit, somewhere, doesn’t mean we should go do that. But enough about the Zune, right? SN: (laughing) Yeah, Zune was a great device, but the world didn’t need Zune from us, and so that was the end of it. This identification of your unique capabilities, is that one of the changes over the last two years where that has emerged? SN: Yeah, in fact, it has emerged and also the world’s kind of gotten to it. Has it been forced on you to an extent? SN: Yeah, even my own conceptual understanding, I started by thinking of, “What are models?”, models are kind of like some stateless APIs, then I adjusted and said, “Oh, maybe there’ll be like databases” — they’re really more than that. I don’t remember talking about this with you, but last time I talked to [Microsoft CTO] Kevin [Scott], we analogized it to processors at some point, and you actually did make a comparison in terms of the partnership to your partnership with Intel. SN: Exactly. So the question now is, it’s a better conceptual model to think of what we’re doing is you have to really build a learning machine, and any company has to build a learning machine, so what I want to build is essentially a multi-tenant learning system that allows everybody to have their own hill-climbing machine . So that conceptual idea, now I’ve turned what is essentially frontier is not about any frontier model — I want to build whatever you did with M365 or with Azure into a platform which allows everybody to basically build their own hill-climbing machine right because the future of a firm at a foundational level they’ll have human capital they’ll have token capital and for the token capital they need their own hill-climbing machine. All right, so I’ll jump to the end, you released seven new models, you emphasize the work you’ve done to build these models from scratch, not with distilling, not with using other models as teachers — so did you just articulate what the ambitions are with these models? SN: Yeah, there are two sets of things. One is we wanted to build from ground up with clean lineage, the models that we will have that we can license and allow enterprises to continuously hill-climb, so that’s why we want that model. By the way you talked about distillation — the point is to not use distillation during any of our own hill-climbing but at the very end, in fact some of the things that we are doing is, after all, we have all the OpenAI IP, in fact some of the performance gains we get is by doing RKLD, which is reverse knowledge distillation , and RL on top of it. So we have effectively two frontiers, we have our own, we have the OpenAI, and we’re going to use these things to eval match. And the clock is ticking to get to the right state you need to be while you still have that access . SN: Yeah, and there’s five years of it. But the bottom line is at any given point in time, I want to make sure that I’m using the best, most efficient model for whether it’s in coding, whether it’s in security, making sure also in our case, we’ll have a harness that’s independent of these models, we have the GitHub Copilot harness that’s used everywhere across Microsoft. Our goal is to make sure we have a model lineage, which we control end-to-end, we then use OpenAI IP, even with all of the capability it has — ultimately, the tests are going to be the evals for us and our customers. In the long run, the way it was framed today, and I thought it was very compelling, and it speaks to what you just said, was this idea of enterprises being able to take these models and in their own RL environments incorporate their data at a much deeper level than sort of a slap-on RAG implementation or basic post-training. Is that the end goal, though? SN: Yeah, the end goal for me is the following, which is I go back and say, let’s say that they’re a generalist model — if you go back even, Windows could have a release, then another release, and Adobe and Autodesk could keep building and keep going up, what’s the moral equivalent of that? That is the thing. And then in the first time, we said fine-tuning, it kind of didn’t work because we didn’t have the tools, we didn’t have the data collection regime, none of that. But now we have it. So let’s say the generalist models keep getting better, MAI models, let’s say, or OpenAI models, then you have this RLE. Right, but this deep customization of the models you’re talking about is only possible with MAI models. SN: That’s correct, but the thing that we want to start getting everyone on is this multi-tenant hill-climbing system — so if you think about it, we literally turned your use of M365, which already is a multi-tenant system, into a hill-climbing system for you. Okay, I’m gonna have to stop you, I’m going to give you an ELI5 opportunity, explain hill-climbing to the audience. SN: Hill-climbing is basically when you think about, “What does AI do?” — AI is all about taking an objective and continuously learning how to go predict and create that output that is the representation of that objective, and do so continuously. So that’s why a metaphor of hill-climbing is the best way to describe learning. And you want everybody to do this individually on their own hill. SN: Individually on their own. As opposed to like, hitching along. SN: What is your moat as a company? Your moat as a company is your tacit knowledge. In a world where AI exists, and network effects of AI exist, you need your own hill-climbing machine in which the models are learning. So the first thing we want you to do is, people don’t talk enough about this, but the private outputs, the evals, as I think about as, maybe the most important IP a firm creates are these private benchmarks and the private evals where you are tastefully recognizing what’s the output, the quality. And by the way, today’s failure cases are informing you to change the benchmark continuously, it’s not a static thing, that’s kind of how the evals work. And so if you have your private evals, then you have your own reinforcement learning environment that you’ve created, then you invite all the models to show up, and then you say, “Model A, generate the output that is maxing this eval using my environment and my trajectories and model B…”, and I can switch. In that context, the MAI models is one more lineage that you can put into,c and what we proved today was even a very efficiently trained reasoning model or a coding model can hill-climb using your traces and that will be more token-efficient and it will be fundamentally a great advantage. Exclusive to you the customer. SN: Yeah, that’s right. But is that just for now? If you fast-forward, is your vision that actually MAI models are fully competitive on the frontier with the other general models? SN: They are. Even today, when you start saying that — the world will keep getting better in general.** Well, I guess this goes back to, is this about how you need to do what you’re good at? SN: Correct. One, what we’re good at and also what’s the equilibrium of the world? Which is, if you believe there are only going to be two firms in the world, then of course, they only need two frontier models, but if you fundamentally believe that there are going to be as many firms as there are today and more, then what is the firm in the age of AI? It’s going to have human capital and token capital, how did that token capital get created? It’s not a bunch of API calls, it’s actually some set of weights even they have. Right. And so do you want to accrue that advantage or do you want to give it to OpenAI and Anthropic? Well, speaking of the OpenAI partnership, I mentioned you referred to it like the Microsoft-Intel partnership, and sometimes partnerships are the only way to get ahead. How do you think about that partnership now? SN: I still think that it’s — I’m very proud of the fact that we came together, you remember the circumstances in which we came together were very different and the fact that there is a company now that may go public and be a trillion-dollar company— This is my question — how long were the knockdown, drag out fights between in this corner, there’s Satya Nadella, the operator, and in this corner, there’s Satya Nadella, the investor, tussling over what to do? SN: (laughing) At the end of the day, we are an operating company, investment is just more of an accident. Yeah, but the shareholders are ultimately those investors! SN: I’m glad and it’s a fantastic outcome for our shareholders too and what have you. But I think the way I came at this, Ben, is to say genuinely I’ve always approached it as, if there’s a partner that we can partner with and ourselves innovate, and they’re also successful, that’s fantastic. I always go back to the story of having built SQL Server with SAP. SAP was successful, we were successful, we also then went on to do other things. And so therefore, I think OpenAI, I’m glad we worked with them, we’re working with them, they continue to be a premier partner. As I said, until 2032, we still have a lot as a customer of theirs, them as a customer of ours, as an IP partner. So every day OpenAI does well, Microsoft does well. Is there a bit where everyone thought you were so far ahead because of your partnership with OpenAI, and now when we talk about things like your MAI models, it’s like actually “We got a little bit lulled to sleep because we offloaded too much to them, and now we’re having to recalibrate”? SN: Lots of things, one is, like all things, there’s a lot more competition, there is OpenAI, there is Anthropic, there’s Google, there is tons of folks who are in there. And so I think for us, the beginning, it was great that we got started with OpenAI. Think about where we were in 2018 to where we are in 2026, here we are competing with Google and a bunch of people whose names I wouldn’t have known in 2018, and so that itself proves that to your very first question, “How competitive is Microsoft?” — I’m glad Microsoft took that shot. Here we are competing with a bunch of new people, a bunch of old people, and we have our own game. So we already talked about Satya Nadella, the operator, and Satya Nadella, the investor. What about Satya Nadella, the capital allocator ? There were a lot of reports in about early 2025 about Microsoft pausing and a reconsidering some data center investments, you guys have sort of spun that as, “Lots of speculative stuff”, “We’re streamlining”, etc. — but at the same time, your percentage of free cash flow committed to CapEx lags fairly significantly behind your peers. Four months ago, that was a compliment. Now, is it a diss? How are you feeling about that? SN: The last time I checked, my free cash flow is getting allocated pretty well to capital return that makes sense. Is there a case that you’ve underinvested? SN: Not really. I think the key thing that at least we wanted to make sure is we were not upside down on building — we have a hyperscale busines, we have our own application business, and we have our own research compute to allocate, there are three buckets, we wanted to allocate with great discipline on all three. So take the hyperscale business. Hyperscale businesses are about having a few big customers, but also having a massive long tail, so you can’t have a book of business that is just a few model companies — in fact, one model company — that was the fundamental decision. And you wanted to get out of that business. SN: Not just get out. They’re still there, they’re a major tenant. SN: They’re a major tenant. But, let’s face it, Anthropic over time or OpenAI over time will build their own, it makes sense. They would use — I’m not saying that they won’t use other cloud providers. So to me, it was clear as day that, what I wanted to do was not allocate all my compute only to one player and so that was the adjustment. And once you make that adjustment, you can’t build 10 gigawatts in Texas and say, “That’s it”, you’ve got to build a plant that is spread around the world, around the United States, and that adjustment is what we want to do on hyperscale. The other thing that I have to do is make sure we’re doing also the long-term thing for our investors, which is, “Let’s invest in ourselves”, which is inference compute has exploded, whether it’s in GitHub or whether it’s in M365 and we needed to make sure we fund our own applications. And then our own research compute, these MAI models. So I just took the approach of putting these three, we will definitely want to allocate as we see progress on all this and we’ll see how it all shakes out. But to me, I’m not literally matching quarter-to-quarter. By the way, the other interesting thing is the catch-up, we started early. You were early, and you got a lot of the good spots, a lot of the good power generation. SN: Yeah, and also two years of cash flow. Yeah, for sure. Well, speaking of the balance between the three, in January 2026 , you missed Azure earnings by like 0.1%, so it was very small, and you said on the call , you allocated more compute to internal R&D and applications. Setting aside the earlier question about whether or not you erred by the total amount of capacity, you talked in that call about having a portfolio approach in terms of investment, balancing Azure, and those two other businesses. That’s all well and good, but if there is a constraint, you do have to choose, do you think you made the right choice then? And is that the choice you’ll make going forward? Where you are at the end of the day, you have a higher lifetime value, higher margin on your own businesses, and that’s going to be number one. SN: Yeah, and also research compute. Ben, I think that for all of us, quite frankly, we have to really, at the end of the day, that’s why I think quarterly earnings are interesting, which is, of course, The Street should hold every one of us very accountable for “What did you do for me lately?”. But was that a very particular, annoying, being held accountable for the wrong thing? SN: It’s their job, everyone’s got to do their job, and so I can’t accuse them of them asking, “Hey, what did you do for me this quarter?”, that’s the question they rightfully should ask. And the right answer for me is, “I’ve done enough for you this quarter, and we’re also making sure that 10 quarters from now, Microsoft’s continuing to thrive”, and that’s the job, and sometimes there’s a little bit of disconnect on it. But when I look at the three things, you just have to be disciplined that you’re doing what you can add value, it can’t be, “Oh, I’m misallocated”. To your point, you get punished if you do things where you’re not producing. So that’s why research compute, here is now an MAI model output. Today, it’s just not a model output as an academic thing, that’s now in differentiating our Foundry where we now are able to license it, it’s going to grow Foundry revenue. And so as long as I’ve felt that as long as Microsoft can continue to invest in ways that show results, then we will have the ability to do the right thing in the long run and in the short run deliver results. For the last quarter, was there a bit of, “Let’s give a little bit more compute to Azure?” SN: Last quarter, no. In fact, that one was just a little more of the compute — we are supply-constrained. I know, but that’s what makes it so interesting. SN: We are not at all, like at this point, if anything, the thing that we do not want to do is to disappoint especially our enterprise customers on Azure. That was the question, right? Because if they look at that quarter and they’re like, “Hmm, Microsoft’s saying we’re supply-constrained and also we’re prioritizing our higher margin, higher lifetime value businesses, where does that leave me? I’m competing against my supplier”. SN: That’s one of the reasons why we had to make some very hard choices around, for example, raw GPUs. We’re not selling raw GPUs to a bunch of Neolabs, for example. I wish I could add more Neolabs on Azure, we just cannot. And so therefore, we are being very disciplined on some business that we turn away. Were those some of the conversations you had to have? SN: Yeah, and so to me, in a world where you have constraints, you want to basically make sure you’re building for both what the world expects and the customers who have trusted you in the longest and so we will definitely make sure that Azure has capacity, it’s just that we are not going to go for what I’ll call in this context, “easy money”. Which is, you can always, in today’s day and age, if you want to have short term Azure revenue, it’s pretty easy. Oh yeah, we’ve seen that , to say the least. SN: Yeah, all you gotta do is turn up, you know, and go sell it to a Neolab. So when it comes to AI infrastructure specifically, as you look out in the long run, you mentioned it may very well be rational for the frontier labs to build their own hardware, for example. You have all these Neolabs, you have whatever controls [Nvidia CEO] Jensen [Haung]’s allocation of GPUs, you have different ASICs, what is your true differentiation as a hyperscaler? Is it just lower cost of capital? SN: First of all, think of our hyperscale business as this portfolio, everything from what we are trying to get done is build a system which we have to be competitive in when it comes to tokens-per-dollar-per-watt, that’s one side of it. We can unpack that and what our thesis is there. Well, I just noticed when you were talking about some of your chips, sometimes it was tokens-per-watt, sometimes it was tokens-per-dollar. SN: Yeah, I think of all three, right? It’s like tokens as a function of both power and dollars and so that’s a systems thing that we have to be world class at and be competitive at. And I would be able to claim, and that’s where I think [Microsoft AI CEO] Mustafa [Suleyman] talked about it, like unless and until you build your own model, you can’t, there’s no point. I believe that you don’t want to build accelerators without building a model, you kind of have to co-design. In the long run, the only way to be super efficient on that is to think about, the network is a great example, which is you want the network, the model, all to come together in ways that make sense, so therefore that’s one side. Then the other side for us is the differentiation has to come from, “If I’m building agents on top of this infrastructure, what agents does Microsoft produce?”. I have three domains in which we are going to try and major on: coding, security, and knowledge work. Luckily these are three massive domains where tokens make sense — I’m not saying there won’t be others, science is another one we will enable but I think there will be others who will do great work in there. But to me the three primary domains in which all this is going to be exercised use. So when I think about the portfolio of building a system plus model plus these three domains, then I feel like that’s where our differentiation will come from. But is that just a re-articulation of circling back to, in the long run, our true differentiation is from our higher margin, our own businesses, higher LTV? Where does that leave just customers who— SN: I think it’s not higher margin. The overall margin dollars from our infrastructure business may be higher. In fact, they already are getting close to being higher than our total margin dollars from our high margin businesses. So I think that Microsoft has always benefited from having a portfolio of businesses, and we’ve been comfortable managing through it, where it’s not one margin profile. But in aggregate, we will have high ROIC, and we will make sure that we have an infrastructure business that’s got ROIC that’s commensurate with an infrastructure business, and we have a business that builds on top of it, which I’d like call it like the new apps are agents. So we’ll have agent businesses in security, in coding, in knowledge work, as the three big domains. We’ll get to agents in a little bit, but I didn’t expect to ask this question, big news this week, will you ever issue equity to fund this build out ? SN: Yeah, I just saw the news, I think Google just did it. Were you as surprised as everyone else? SN: I’m not sure, exactly, I’ve not studied it, it came last night, I think, so I’ve got to go understand what’s happening. But, it’s like maybe it’s the thing to do is everybody is going public or reissuing equity, maybe that’s the season. Gobble up some of the money. Is software dead? SN: I think software is alive, but the way I think this entire meme has come about is, like, if you take the SaaS question in particular, right? We built in a particular way where I had a data model, and then I had a business logic tier, and then I had a UI tier, I coupled the three, then had a business model. Integration is a beautiful thing. SN: Look at this, Ben, right now, we took what is the database that no one knows about underneath Microsoft 365 and said, “Oh, WorkIQ is available , it’s just a skill/MCP, and it’s out there”, and suddenly people are falling in love with, “I can now interrogate and have an agent continuously hit this database to reason over and plan over, act over from any place”. By the way, it requires a new business model. So, for example, when Cowork is using WorkIQ, that’s going to be a usage-based business model, so I think what needs to happen is we now need to take what we built, rebuild it for the agent era and change the levers of the business model such that you have a per-user business model and you have a consumption business model. So the hybrid business model, you do think that is going to be the future? SN: 100%. And once you have that then I think what happened between servers — even I had not understood it when we moved to the cloud, even I was a little worried about, “Oh man, we move to the cloud, we’ll sell the same servers”, and it turned out we sold a lot more subscriptions because people who never bought servers from us were buying subscriptions. I think that’s what’s happening already with agents, I see that on GitHub, I see that on M365, I see that on security, because everyone is building these agent systems that are continuously “working” and so what we built and thought of as the end-user compute is completely getting rebuilt. Is there a bit where, if you have to zoom out a hybrid system where a combination of per-seat but also usage, where does E7 fit in this idea, it’s like double the price, it seems it’s an attempt to respond to maybe a secular decrease in seats by increasing ARPU? Is that the right way to think about it? SN: The way you think about this is, see per-seat is a very important element still because what is per-seat? Per-seat is basically a set of usage entitlements, so anyone who is budgeting really will push you. That’s right, people don’t like usage, we’re seeing that right now , it could explode . SN: Exactly, so therefore you just want to take packaging or bundling of usage into proceeds so that there’s some way for people to budget. So I kind of think about the E7, E5, these things will continue and then you’ll always have the outcall consumption. People also talk about, “Hey, maybe people want outcome-based pricing”. Outcome-based pricing, we’ll be thrilled about some of that, but remember, outcome-based pricing is also called royalty. When a customer has a great outcome, they necessarily don’t want to share their outcome so I think what is really being thought about is, ultimately, there is real marginal cost to software, that’s kind of what it is, and that’s going to be priced through. When did that really click for you, the implications of that? SN: I think that I would say agents. Before agents, if it is still human interaction— Right, you can imagine a world where just like basic inference got super cheap and easy. SN: Exactly, the Moore’s Law itself. Like, if you think about it, if I just used Moore’s Law, get software efficiency, I used software for efficiency and drive that home for customers to have more functionality. In fact, I used to always think about, “Hey, how much more value did we add in M365 and not raise price?” — we didn’t raise prices for a decade plus. That’s all thanks to the software efficiencies on top of hardware. But now where you are, and if you have a thousand autonomous agents that are all working continuously 24/7 hitting Work IQ, then that is a lot and so that is where I think, and so the real test for me Ben is, that’s why evals, outcomes — no customer will use consumption or their seats if it’s not creating value for them. Therefore, they now are going to be a lot more disciplined on, “What exactly did this stuff do for me?”, “How do I measure it?”, “How do I get into the efficient?”. And if you think back to going back to the 80s or 90s, where back then it’s like, “Don’t waste time on optimization, the next processor will come out and solve all your problems”, is that now totally the wrong paradigm? SN: In some sense, you want that to happen, but you can’t just count on that. It will happen, but your prices will explode. SN: Exactly, and more importantly, you will be found out if you don’t optimize. Take that example we showed with Land O’Lakes today, which is, here’s an agent, and there is an outcome you care about, I was able to use a model that is using 500B, I was able to use a 5B, and have it really deliver the same outcome, why would I not use that? That does seem to be a very different thing about this period. It seems clear that’s going to be a huge thing in enterprise going forward, using the right model, optimizing, it’s like we didn’t get to the optimization stage of the PC era. SN: That’s right. I don’t think we ever did get there. SN: We never got there. Stuff’s still bloated as ever, because everyone just assumes it’s going to get faster, it’s going to be fine. SN: Exactly, because things were not priced for it. Once you have consumption, everyone will optimize. For E7, it does seem like the real lure there is Cowork . It’s like this new capability, it’s super powerful, it’s taking Anthropic’s Cowork, which is on your PC, now it’s in the cloud, has all the niceties around that, permissions, controls, all those sorts of things. Is that why it’s there? Is that the hook? SN: Yeah, there’s also the Agent 365 , so there’s a whole lot. Like always, these things, we’re going to take everything from what I’ll talk about as what is an end-user thing and an IT thing, bring it all together. You guys know bundling. SN: And security. Yeah, definitely, and they’re all about, ultimately, how do we get the value equation right such that the customer can cover, because right now, it’s kind of fascinating. You have an agent, you immediately say, “Oh, I’ve got to secure it, I’ve got to have observability on it, I need a sandbox for it”. So it’s just that if you don’t bundle, you kind of are sending the customer down the chase of five different things. With that, though, the reason I find that striking is you’ve talked a lot about — to what extent do you think the point of integration that really matters is it does seem to be increasingly between the models and the harness themselves ? You’ve talked about things like your CoreAI initiative and GitHub Copilot, a lot of which is, “We’re going to build the harness and you can slip the models in and out”, and that works right now for Copilot and you can choose your model and even then, from what I’ve heard, not quite as easy as you might think it might be, but it’s still there, the selector’s there. Cowork seems like, “Yeah, that’s right, it has to be the whole package and it’s important for us to have a selling point on E7” — that this feels like maybe it’s not easily substitutable. SN: No, it is. The same thing on Cowork. In fact, right now, the Cowork that I’m using is already mostly defaulted GPT. Okay, so it is going to be fully interchangeable? SN: We’re using the same harness that we use in GitHub and the same thing in security, too. So we have the same harness that’s a multi-model harness in which we will rotate through — obviously MAI by default gets trained in our harness, but we will have GPT, we will have Anthropic in there and any open weight model. We will allow anyone to take any of the models they fine-tune or build. In fact, they can take an open weight model from Fireworks, tune it, put it into Copilot, no problem. All right, so I am misinformed, so I will take the L on that. Explain what is Cowork then and what is the connection with Anthropic as far as that product goes? SN: Cowork, to me, it’s kind of like Copilot. I took the term Cowork, it’s part of there and it’s definitely got the Anthropic models in there. Cowork is — think of it as a form factor, the best way to describe it is we built a chat interface first for Copilot, then we now have built Cowork for Copilot, and now we’re building autopilots, as I described it there, think of it as the enterprise-grade OpenClaws. So basically, I think of these as different form factors of agents — chat was the first thing, Cowork is the next thing and in fact, you can even go back to the developer thing. Developers, how did we start? We started with code completions first, then we went to— I get all this, but I’m genuinely confused here, because I go back to the blog post . It says, “Working closely with Anthropic, we took what they’ve done with Cowork…”. SN: Yeah, that’s what we launched first. All I’m saying is it’s evolved. It’s kind of like, Copilot today. Got it, which started out with ChatGPT. SN: ChatGPT, now it has both Opus and GPT models. Got it, okay. SN: So, they’re going to be all over. All right. So, I wasn’t completely off the reservation. SN: That’s right. I failed to catch up, I will accept that. [ Editor’s Note: the FAQ for Cowork still says it uses Anthropic models, just like the original blog post ] SN: Every product of ours, you’ll have both Anthropic and OpenAI models, and MAI models, and your ability to put your own models, and that, I think, is the fundamental promise. Oh, by the way, I should mention this. The amount of auto — I don’t know how much you’re doing selection, I’m mostly auto — and so then one of the biggest pieces of work at Microsoft is all the training models to do auto-routing. That, by the way, is perhaps one of the biggest continuous learning things.** It’s interesting because I probably approach it more from a consumer perspective, so I just literally choose the app that I want to do something in or call from the CLI. What happened to Github Copilot? You’re talking about it very positively, but I think a negative spin would be two or three years ago, you were first to market with autocomplete, everyone assumed you got there, you won, and now it’s like, “We’re going to catch up with GitHub Copilot”. SN: I think what happened is this is one of those classic cases — remember, it was a tools business before, and now it is the business, who would have thought that coding is everything? Right, it should have been everything, but it seems like for some period of time, it wasn’t? SN: For us, I think what has happened is we have continued — there are two things that are happening in GitHub, before I even talk about Copilot, I should talk about GitHub. All these coding agents have shown up to work, and where have they shown up? In GitHub. And so the first thing that, quite frankly, I wish we had anticipated better, was the amount of agenting. The whole GitHub reliability thing is like one thing, but for Copilot specifically. SN: I’ll say the first thing, that’s kind of, at some level I take that job seriously, because job number one before you want to get to Copilot is go make sure that we are scaling, so let’s leave that alone. There’s a lot of people very unhappy about that. SN: Yeah, and we’re going to work it and they should have higher expectations of us and we need to deliver for them. Then the next thing is on the Copilot side, you’re absolutely right, we started by saying, “This must be just a code completions thing in the IDE”, we added chat, we added tasks, and guess what? Let’s give credit where it needs to be given. Anthropic showed up with a model. Well, this is like Cursor’s story , they ate your lunch even before Anthropic did. Or you’re saying that that was also an Anthropic story? SN: Not really, I mean it’s kind of like Cursor/Microsoft, it’s like Borland v us , it’s not like that was not the end all be all. It was really the Anthropic coming in with a completely different approach, a more agentic approach. SN: That’s right, with a different approach. With a model and what they’ve done there, and essentially the agent loop is what the change was. In fact, if you look at it, Cursor never, total volume-wise— They got eaten by the same thing, they’re facing the same challenges. SN: Also even the market share and so on — Cursor did fantastic, they forked VS Code, did a good job, lots of credit to them. But the real thing was agentic coding became real and now the good news is the agentic coding really drives — people want choice, we will be there, we will have our own models. GitHub itself and Copilot itself will have both the Anthropic and Claude. In fact, the rubber duck feature is my most favorite feature , which is I can use it to check the others. The headline announcement from this week, I guess is these new Nvidia-based PCs running Windows . However, the announcement I found much more interesting — or not an announcement, preview — Project Solara , viewing these devices as ways to access agents in the cloud, totally different center of gravity. I don’t know if it was you that said it or the presenter, something which I thought was really compelling, which is a limitation of wearables is if you have to interact with them continuously, they get very tiring, so their utility is fundamentally limited. But if you can ask an agent to do something, then you can go do something else and meanwhile, it’s running in the background. Super compelling. I guess the question is, this feels totally different than Windows — it was weird to start this keynote talking about Windows and the AI PC, and that’s nice, and local inference, but this is like, “Actually, what if everything was in the cloud?”. SN: Yeah, I always find this frame back from 2014 of ubiquitous computing and ambient intelligence and it’s becoming more and more real each day. First of all, the first part of it was, “I’m so thrilled to have these Windows machines”, and the fact that Jensen had that beautiful slide, the picture of him with all the desktops, I was like “God, yes, I’ve been waiting for it”, which is it’s great, so I think because it makes sense, it makes logical sense to have powerful silicon systems with power that really have it with unmetered intelligence. When I worked at Windows, I had to like furtively hide my iPhone and then it was okay to show up on campus with an iPhone, now I’m here with a MacBook Air — next time I interview you do I have to feel bad that I don’t have an Nvidia AI PC? SN: You will always have choice, Ben, and I hope you choose the right thing. I’m excited about that stuff because I think there’s unmetered intelligence, even there was one little feature that we showed, which is that ability to have eight agents running continuously, analyzing logs and so on, but all of them were unmetered. Right, but that feels like it’s a side project, side quest. SN: It’s kind of like a billion users all having that, that’s not a side quest. To me, it’s as fundamental as like I think the people are going to want for their knowledge work, for their security work, for their coding work, machines— They’ll want for themselves. Is this actually the new consumer/enterprise separation? SN: The enterprise — the business model, we had this long conversation about enterprises continuously optimizing — in fact, I think the biggest value prop of a Windows machine in the enterprise will be unmetered intelligence. So people are going to say, “Oh wow, instead of having my cloud bill keep going up, I’m going to have Windows machine and amortize it that way”, so I think that there is going to be a real value to — because in a world where you have infinite amount of tokens you want to consume, you want to optimize, and why would I not optimize using everything? I don’t know, I just feel like — as you know, I’ve been very impressed with the job you’ve done with Microsoft, ending the stranglehold Windows had on the company, I still remember I was actually in the Bay Area, I was sitting at the bar at The Westin by the airport typing The End of Windows , recounting all these things you did to not kill Windows, but not make it the center of gravity for the company. SN: And that I think is what goes to Solara. I don’t think Windows, we are trying to make Windows— SN: Solara, to your point, I thought it was a great question, because the thing that I want us to take a shot at is the following which is, “Can you think of a platform and platform rules, by the way, which are built for the agent era?” — because right now, what is everyone else who are “platform owners” who will try to move from the phone to this wearables will try to bring their apps to the same game, right? I want to open that up, so I would like, for example, like what we were able to do with Teams devices , and that’s where we built some of this sort of distribution capability, so I want to use that connected to this agent world so I’m excited I’m in MediaTek, Qualcomm. Well I have a great analogy for you, I think. So there’s a bit where I think you just circle back to the great job you’ve done as CEO — this is the butter-up portion of the interview — there is a bit where I think you benefited from following the follower as it were. Steve Ballmer’s one that had to go after Bill Gates and he for better or worse created the conditions for you to succeed, I think is one way to put it, is it possible that for this, your opportunity device space — like can Apple ever really make an agent that works everywhere as long as they’re stuck on the phone? SN: That’s a great question. That is the question for all of us which is you know the reality is it’s easy to say for someone who’s been so successful with something that in face continues to have a lot of success and say, “I’m going to burn it all down and build something else”. But to the point, the way they’re architectured, everyone’s vertical. SN: Exactly, it’s not natural. Like you think about it, we’re saying, “Building agents is easy”, the SOCs are jumping out everywhere, they’re there, the silicon is easy, the system is easy, the operating system is built, and now you’re telling me that I have only one choice for an ambient thing in a hotel, in a restaurant, in a healthcare setting? It makes no sense. So therefore, I imagine that building these ambient devices using Project Solara will be as easy — if you’re successful a year from now, everybody, even in the enterprise, is going to say, “Oh, I’m just going to order a bunch of these things from a no-name ODM who just built it for me”. I think it’s super smart to start at the enterprise only. Do you have dreams that maybe this will eventually spill over? SN: Right now, I want us to again do what I think is natural, like where am I seeing people— Well, that’s where you have the Microsoft 365 environment, you have all the context there. SN: And also the agents, where would people build agents? The thing is, the consumer one will be like, “I need the one agent I want”, so it’s not like I’m not building a Copilot device, I’m building an agentic platform where the healthcare provider can have their own agent, so that’s the right place for Microsoft to start, let’s see how it goes. One last question. You had a data center segment appropriately focused on communities, you talked about things like paying your way for electricity, not using water, building up the tax base, education, etc. Why not just pay the residents ? Just pay them a dividend? SN: I’m open to all ideas here, I’m not close-minded at all because at the end of the day, I think the fundamental thing you’re asking about is, “How does this industry, including Microsoft, have permission to do what we’re doing in terms of infrastructure build out?”. My theory is we get to everything backwards in the US, this is how we back into UBI [Universal Basic Income], is we’re just paying people to build data centers. SN: Yeah. And I mean, one thing that I have an issue with things like UBI and so on are the— I’m anti-UBI. That’s how you get there while being anti-UBI. SN: I want people and communities to have control, have agency, humans to have real dignity in their work and you’re 100% right in saying, “Look, we have to do what it takes to get that permission”. And so right now, there’s so much about our industry that’s so glorious, so good, so great. What about the you’re going to lose your job part? SN: Yeah, that’s the problem. Self-obsession about our own glory and our own — if you’re not creating opportunity, why would anybody want you to succeed? That’s the fundamental memo that needs to be re-sent to everyone across our industry, and then we have to live up to it. Satya Nadella, great to talk to you again. SN: Thank you so much, Ben, as always. This Daily Update Interview is also available as a podcast. To receive it in your podcast player, visit Stratechery . The Daily Update is intended for a single recipient, but occasional forwarding is totally fine! If you would like to order multiple subscriptions for your team with a group discount (minimum 5), please contact me directly. Thanks for being a supporter, and have a great day!

0 views
Martin Alderson 1 months ago

Is datacentre sovereignty really that important?

In the UK (and I'm sure elsewhere) politicians and commentators are falling over themselves to suggest that without huge fleets of datacentres built in the UK that we are going to be hopelessly left behind. I'm not convinced this is the case, and it risks really falling into the same (mostly misguided) obsession many politicians have for heavy industry revival. This is going to be a rare UK-centric post on my blog. Apologies for my mostly global readership; the argument may be different where you live. One of the first (and most easily dismissed) arguments I've heard is that without datacentres close to the users, the latency will be too high to use AI services. This would therefore make them too slow to use. Clearly, this isn't the case. Nearly all AI use cases are not hugely latency sensitive. To put this in context, the time to first token (how quickly the AI responds) on Opus models is between 1.6s and 3.6s. The round trip latency introduced from the UK to the East Coast of the US is around 80ms, to Europe 10-20ms, and to Asia around 200ms. So the latency on the providers side is orders of magnitude higher than the latency for a UK based user to reach an overseas datacentre. It is fair to say that real time voice or video applications benefit from lower latency than these typically text based use cases. But these are a tiny fraction of AI usage (at the moment) and even in that case European datacentres can provide reasonable latency for these - it doesn't have to be in the UK itself . And my personal belief is that real time audio based agents are likely to work best when they can run on device entirely (so there is 0 network latency) - so without a data centre requirement at all. Regardless, many of these same commentators also suggest locating datacentres in the very north of Scotland (to take advantage of the excess wind power), but ironically these would have significantly worse latency for users from the densely populated south of England - Paris, Amsterdam, etc all are closer, and thus faster to respond. The next argument that is often floated is that it becomes a tax base - in the UK business rates are applied to commercial buildings and are paid to the local authority in question. The formulae for calculating this is in true UK tax law style overly complicated, but in essence it works on the rateable value of the building in question - what the estimated annual rent would be to rent the property - including relevant fit out. This is then multiplied by 0.508 to arrive at the annual business rate value. To take a very rough example, my research found that buildings 5-8 in the Virtus London campus can support 100MW [1] of load. These are valued as far as I can tell at around £12m/yr of rateable value. So the local authority (London Borough of Hillingdon) gets approx £6m/yr from this in business rates. If we scale that up to 1GW, it's fair to say that the local authority might get somewhere close to £100m/yr of business rates. While this is not nothing - and certainly gives local authorities a valuable source of revenue - it really is a rounding error under the current system . If we moved every single datacentre under construction globally (30GW) to the UK instead, it would bring in approximately £3bn/yr, or around 0.2% of government spending. Detractors may say that this is the current system and the tax base could be changed. But by doing that you massively reduce the attractiveness of the UK as a place to build the aforementioned datacentres. And the potential tax rates to be at all material would have to be punishingly high. This combined with the extremely high price of electricity in the UK would make it completely unfeasible to operate them in the UK. It's a similar story with jobs. Datacentres are famously light on permanent staff - the whole point is that they're highly automated, so even a large 100MW site might employ only a few dozen people once it's running. The construction phase is more labour intensive, but temporary, and much of the capex (the chips especially) is spent overseas rather than in the UK. Even on generous assumptions the direct contribution to a ~£2.8tn economy is a rounding error. The final and perhaps most plausible sounding argument is that in the event of political instability it would give us control over AI usage - which is and will be a growing national priority. There are really two versions of this argument. The cruder one is outright seizure, which I'll come to. The more serious one is that in a global compute crunch, having the datacentres physically here means we won't be left at the back of the queue. But this doesn't survive contact either. If a hyperscaler or a frontier lab owns the racks, a datacentre in Slough serves their global demand - not ours. You can't compel a private operator to give UK users preferential access just because the building sits on UK soil. Location buys you almost nothing. The real leverage here is to contractually lock in the compute - which is something the UK government could do, regardless of where the datacentre is. Onto the cruder version, then. I've even heard certain people suggest that in the event of major turbulence in the world the state could seize control of them. The issue with this is multifaceted - but I think has three main failings. Firstly, this is not a steelworks or power plant. The underlying value is not from the datacentre, it's from the models running on the datacentre. If we assume AI model development continues, the value of a 'seized' datacentre decays rapidly. Imagine the UK government had seized control of a frontier labs datacentre at the start of 2025. They'd have access to GPT4o, or Sonnet 3.7. These models are now outclassed by open weight models that you can run on a relatively powerful laptop. They have virtually no value. Secondly, it completely underestimates the supply chain that modern software runs on. It's highly likely that if the geopolitics had got so bad HM Government was nationalising frontier lab datacentres, the frontier labs would remotely wipe the servers before they could be "seized". And that's not to mention that models have loads of supporting software and operational infrastructure that is not colocated with the models themselves. The concept of the SAS seizing servers running frontier models before they can be wiped in the dead of night is probably best kept to Tom Clancy novels - not government policy. Finally, if we are in some alternate reality where the UK/Europe has been cut off from frontier models, we are almost certainly also cut off from most/all cloud services from big tech, which means no (or much reduced) email, video conferencing, card payments etc. Not being able to run Claude is probably the least of society's worries. By no means am I suggesting that AI datacentres shouldn't be built in the UK - they should - and we should reform the planning system to make it easier to build them. But it's important to get this in perspective. Modern information societies are a huge tangled web of globally interconnected pieces of software. Every day you browse the internet you are connecting to thousands of servers located in dozens of countries. Each one of those servers is sending your requests to various other providers - to store and process data. There are genuine requirements for data sovereignty. It may be preferable to host sensitive health data only in the UK, for example. But that's a simple regulation problem (if desired) - require UK based datacentres for this type of data, including AI usage. But this is a tiny sliver of total AI demand. And the world is too complicated to dream in this "Blitz spirit" self sufficiency era, especially when it comes to digital services. The UK in my opinion has many structural advantages for harnessing the economic power of AI. All of the major frontier labs have significant - and growing - labs and offices in London. We have world class researchers and institutions on the cutting edge of AI. And the UK takes the majority of European tech funding. In my opinion, we need to lean into those strengths and ensure we continue to attract and grow these companies and talent. Not worrying about where exactly we should put huge sheds. Datacentres are measured by the amount of servers it can power, in watts (or megawatts (MW) millions of watts/gigawatts (GW) - billions of watts). ↩︎ Datacentres are measured by the amount of servers it can power, in watts (or megawatts (MW) millions of watts/gigawatts (GW) - billions of watts). ↩︎

0 views
Dangling Pointers 1 months ago

Yield Not Thy Core

Yield Not Thy Core Achilles Benetopoulos, Peter Alvaro, Andi Quinn, and Robert Soule EUROSYS’26 This paper describes a solution to the placement problem in distributed systems. If you model a computation as a directed graph, how do you optimally distribute the graph among a set of cooperating computers? The authors propose a dynamic placement system and implement it in Magpie . One common solution to the placement problem is to ship data over the network. For example, a set of compute nodes could access data via network requests to a separate set of nodes running Redis servers. At the opposite end of the spectrum, code can be shipped over the network. The canonical example is expressing computation as a SQL query which is sent to the node(s) that hold the relevant data. Magpie proposes a more fluid solution, where both code and data can move dynamically. In Magpie, an object represents data that is operated on. What makes Magpie objects unique is that pointers to data stored in an object are encoded as tuples. This allows Magpie to dynamically move objects around the system without invalidating pointers. The downside of this approach is that it prevents traditional libraries (that rely on raw pointers) from being used in user code. Magpie assumes a high degree of inter-object locality, so any given object is stored by exactly one node (i.e., a single object is never split between multiple nodes). User code is expressed in terms of nanotransactions and epics . A nanotransaction runs to completion on a single node and accesses a pre-specified set of objects. The Magpie runtime ensures that all objects accessed by a given nanotransaction are resident on a single node before executing the nanotransaction. The code for a nanotransaction is simple, because there is no need to query data over the network, and there is no need to deal with locking. If a hazard is present between two nanotransactions, they will execute serially. In Magpie, nanotransactions are written in Rust. An epic is a computation graph where each vertex is a nanotransaction and each edge is a data dependency. In contrast to nanotransactions, a single epic can be distributed across multiple nodes. Magpie schedules nanotransactions once all data dependencies are satisfied. Conflicts between concurrently running epics are handled via snapshot isolation . Any particular epic has a consistent view of each object and may abort in the event of a conflict. Scheduling and data movement are implemented hierarchically. A worker node can locally determine if it has ownership of all dependencies required for a nanotransaction. If this is the case, then the worker node executes the transaction immediately. Otherwise, the worker node uses a local ownership cache to try to determine if another node has all required dependencies and communicates with that node if possible. Failing that, scheduling is performed by a global orchestration node. Fig. 9 compares Magpie to memcached executing a workload that involves a user-specified read-modify-write operation: Source: https://dl.acm.org/doi/10.1145/3767295.3803616 Magpie is able to offer a lower latency because it is able to ship the entire read-modify-write operation to the server that holds the relevant data, rather than requiring multiple roundtrips. Some applications may benefit from being able to indicate that an object is rarely changed and thus can be distributed among multiple nodes at the same time. Thanks for reading Dangling Pointers! Subscribe for free to receive new posts.

0 views
Unsung 1 months ago

Writing about fonts

In last week’s post , I made an off-hand comment about Vercel’s Geist Pixel announcement , and I thought it might be interesting to turn this into more of a full-fledged critique. I don’t think it’s a good announcement, but its flaws are pretty universal, so I want to put words to these flaws. This will extend to a lot of other writing about design, not even necessary even just about typography. Here’s my advice that I believe would make announcements like this better: I know the elephant in the room here is “how big companies do things.” A lot of redesign announcements and font unveils exist chiefly to make the execs who started it happy, and perhaps as fodder for future promotion – I bet the whole “Already shaping what’s next” section isn’t really written for external audience – and they get chewed by the big PR machine that often files away whatever personality and quirkiness might have been there. Your job is to fight the machine! But I acknowledge that it might be hard. However, I’ve also seen all this seeping into personal font announcements, which is unfortunate. (I don’t want to link to specific examples, since that’d be punching down.) Also, this is not just about the joy of reading or some general notion of “craft” – although they are important, too. This is also purely informational. I feel I haven’t learned enough from the Geist Pixel announcement for the amount of time I spent with it. I don’t understand “multiple variants for different densities and use cases” or “semi-mono approach” or what stylistic sets are included. (My general goal is to write in a way that people can learn something new from any design announcement, even if they don’t have any prior context, and if they never actually use the font.) It‘s a shame, because the work itself seems thoughtful and excellent, deserves a better intro, and could help others interested in typography as a jumping off point, particularly because this feels like a typeface off the beaten path. Just to round up this post, some recent counterexamples: #craft #david jonathan ross #storytelling #typography Write like a human being would. This is famously hard, and takes practice. Here, we see stuff like “unapologetically digital,” “a functional tool within a broader typographical system,” “the result feels both nostalgic and contemporary,” and “constraints weren’t a limitation, they were the design tool.” No one talks like this. I think people believe font releases have to use these words and phrases, as a way to bring legitimacy to the project. I do not subscribe to that way of thinking. I think it leads to writing that’s optimized only for admiration, which is not as much fun for anyone. Show a specific example of a problem you solved. This page hints at some things – “They don’t scale properly across viewports, their metrics conflict with existing typography, or they’re purely decorative.” – but that feels altogether too vague to be useful or even interesting. These are actually fascinating and hard challenges, yet I know as much at the bottom of the page as I did at the top. Show details you are proud of. Zoom in literally or figuratively. “Each glyph was manually refined to avoid visual noise, uneven weight distribution, and awkward diagonals.” I would love to see a few examples. Show work in progress! Show stuff you discarded. This will be hard, but why not? It’s good practice and I believe this, more than anything else, will have people appreciate what you did. Plus, everybody loves a blooper reel. Related: talk about struggle. Don’t motion in the direction of struggle, or performatively announce that this was the hardest project of your life. Actually talk about something that was hard, and why. Be vulnerable. Be honest. People didn’t care that Rocky lost in the first movie, because people cared about Rocky. Talk about your inspiration or history. What we all do here is part of something bigger. Why a pixel font to begin with? Why is this interesting to you? Is that because Vercel is filled with nerds, or because you got bored with bold and italic, or because it just seems visually interesting in a new way? Let me type! Immediately and everywhere. I don’t think any modern font announcement/​tester can exist without this. This is the easiest way to getting to know the font and explore specific things that matter to you. (To do this here, you have to go to the font page , switch to Geist Pixel at the top, and then scroll all the way to the bottom. This feels entirely too far away.) Show, don’t tell, generally. The Geist Pixel announcement feels rife for an avalanche of this, but has so little. I mentioned above wishing to see examples of manual refinements. There is a visual for “seamless mixing,” but it’s really a marketing photo, not a real-use example – it visualizes what, but you want to visualize what and why at the same time. I would love to see the spread of variants, specific examples of how the font is not “breaking in production” or not “scaling properly across viewports.” I don’t know what is a “semi-mono approach” and I would like to learn. Motion is okay, but it has zero nutritional value. If you have limited resources, don’t spend it on motion. Anything interactive is better. (But again, the best interactive thing is letting you type.) The “Already shaping what’s next” is a narratively unsatisfying section, as it promises stuff that you cannot see yet. Either show those, or skip the tease altogether. Fran Sans announcement post by Emily Sneddon (complements the font page ) – personal, distinctive, talks about the process, shows interesting artifacts. I feel that every small essay from David Jonathan Ross’s Font Of The Month teaches me something new – pick a font you like on that page, then click Notes next to it. Departure Mono font page by Helena Zhang doesn’t use a lot of words, but still tells a lot. Shantell Sans process post by Shantell Martin (complements the font page ) – already talked about it before .

0 views
ava's blog 1 months ago

rose ▪ bud ▪ thorn - may 2026

Reply via email Published 31 May, 2026 It was my wife's birthday, and our wedding anniversary! Baked some cakes and had a great time. Mine is the Donauwelle attached at the bottom of the post, my wife baked the fruit cake. My friend who visited Japan bought us great gifts from there; I got two gachapon (Cinnamoroll and My Melody), some matcha and My Sweet Piano chopsticks. I finally have it in writing and it's been communicated officially that I am my department's data protection coordinator now. I blogged more. I bought myself a big Build-A-Bear Usahana and a tiny one for my bag. Also, new matcha and I restocked my skincare and supplements :) I feel spoiled by myself. I'm having a great time at the gym, going 3 times a week, and incorporating the strength machines now. The added muscle/strength really helps with posture and counteracting the desk sitting. I'm making good progress. I reduced negativity from my online space. I went to a protest for ME/CFS! I have been better with keeping up with emails. Anita, if you are reading this, I cannot reply to you because it says sending key is not valid. We have a bread cutting machine now! Makes it easier to cut the bread my wife is baking for us :) I attended CPDP 2026 in Brussels. I reached Magenta status (35+ translated cases) as a Country Reporter for Noyb. Working on better eating behaviors and no guilt during rest. I am working on slowly booking cool classes and activities for the next few months. Been struggling with my face shape. I have chubbier cheeks anyway naturally, but whenever I need a round of Prednisone or I am stressed or there's hormonal stuff going on, they get bigger (cortisol, water retention). They are bigger lately... definitely a source of discomfort and shame when we live in a time of razor sharp jaws and almost-hollow cheeks. I will now have to do my injections weekly :( Dienogest doesn't work at all for me. Instead of preventing periods, it causes me more of them. Had to get off of it. My soy and rapeseed sensitivities have been extra annoying lately. Can't eat my beloved tofu, and they put rapeseed oil into almost every protein-rich vegan replacement product. I love my lentils, peas and beans, but occasionally I just wanna have some banger vegan köttbullar, schnitzel, or burger patty without a rash, man, or not make everything myself. Not to mention restaurants, or the fact that they drown everything in rapeseed oil based condiments... I haven't been studying nearly as much as I should. Having some issues with the modalities and feeling a bit stressed, like I need more time away from it. I've been very ambitious this month with my blog posting, and it has caused some writer-constipation at times. I had all these drafts ready with some links and loose thoughts already collected, and wanted to write them out fully; but because I set myself arbitrary deadlines or a loose " This needs to be finished and published today! " I felt intense pressure, which made me freeze up... it's really not that serious, but I made it so, for some reason. I also frequently felt stuck between 2-4 equally "important" tasks, posts, topics, whatever, and when I started one I looked at the other and switched, progressing at nothing. Terrible cycle. I moved some planned posts to June and eased up a bit. The menu of my favorite café has been severely reduced and worsened. Also cannot believe that I am paying 10 Euro for a wrap now. The Brussels trip was filled with some disappointments and stress.

0 views
<antirez> 1 months ago

Distributing LLM inference in DwarfStar

High end NVIDIA cards, and the server and power needed to run them, cost a lot of money, especially if you plan to reach enough VRAM to run massive models. The alternative, so far, has been Apple hardware, or the DGX Spark that, even if severely limited because of memory bandwidth, still allows to run LLMs prompt processing (prefill) fast enough. The Mac Studio provided up to 512GB unified memory, a solution with modest memory bandwidth (but much better than the Spark) and compute at a price that was, after all, given the current situation, relatively fair. For instance, with DwarfStar the Mac Studio M3 Ultra 512GB can run DeepSeek v4 PRO at 150 t/s prefill and ~10-13 t/s decoding, not great but at a level that is usable for certain use cases. Even 2-bit quantized, DeepSeek v4 PRO resists very well, like Flash at the same quantization (today I made PRO write a C compiler, I'll publish the video soon). I would not consider a trivial fact to run a frontier model at home, with a ~12k total spending. One could expect this to get better and better, but the situation at the horizon appears cloudy. There is almost zero hope that NVIDIA setups will get less expensive, and even a small company can’t afford to easily purchase and handle a small data center for local inference. At the same time the RAM shortage is making it not exactly likely that we will see a Mac Studio with an M5 Ultra, maybe 1.2T/s memory bandwidth and more compute (the M5 Max is already faster, compute wise, and has the Neural Accelerators inside each GPU core that help with certain models). So the current situation for local inference is that the best machine is probably a laptop. The M5 Max 128GB can run DeepSeek v4 Flash and Mimo V2.5, 2-bit quantized, at very decent prefill and decoding speeds. We are talking of ~500 t/s prefill and ~35-40t/s decoding speed, with a performance slope as the context size increases which is very acceptable. At the cost of 6-7k depending on the configuration, this is currently one of the best deals. If this is the situation, for local inference projects in general, and for DwarfStart in particular, looking at distributed inference starts to be interesting. What we can do if we have two, three, four MacBook M5 Max systems? Or two M3 Ultra with 512 GB of RAM? Traditionally there are two main systems to run distributed inference. One is to duplicate memory by loading 50% of the transformer layers in computer A, the remaining 50% on computer B, and running the inference in a sequential way. In this case there is to send just the activations around, that’s very simple conceptually, and with some micro-batching magic it is possible to not just duplicate the memory but even in theory to increase substantially the prompt processing speed (but not the decoding: for a single token generation you have to wait the first layers on machine A, the remaining layers on machine B, and so forth — but at least less heat will be produced so it is possible to use a sustained load), which is not bad at all. This means, for example, that the lucky ones that have two Mac Studio 512GB machines could run full size DeepSeek v4 PRO (even if even the 2-bit quants are running very, very well) and with micro-batching even enjoy a faster prefill. Another approach is, using Apple RDMA, to parallelize the execution across the two machines, a vertical split basically. For instance one could try to load the same 2 bit quants on machine A and B, so that both fit, and each side has *all* the routed experts. Then for each layer we could try to do the coordination needed in order to execute half the experts in machine A, half in machine B, and so forth (note that both machines have all the experts, so whatever the router says, we can send 50% of the computation to the other machine, and the activations are tiny). This is more viable for the PRO that has much larger routed experts, so the communication penalty is less sensible. But if this could be made to work well, is all to be seen. There is also tensor parallelism, you are thinking, right? But I bet this is not viable at all with the communication speed we have among two Apple computers, two DGX Spark and so forth (go read the speed of NVLink). The magic about the above two models is that you have to send very little data. Ok, so far I bet you are thinking, this is the same shit everybody knows about running LLMs in a parallel fashion, and indeed this is true. But this post was conceived to reach this exact point. What about if we could, instead, parallelize two Mac or DGX in a completely different way? Open weights models are now in a golden age, we have plenty and many are very powerful. In the 128GB 2-bit quants classes there are many interesting: Minimax M2.7, Mimo V2.5, DeepSeek v4 Flash, and a few more. At the same time it was recently noted that LLMs ensemble (https://arxiv.org/abs/2502.18036) is an understudied possibility that allows two models to run in a completely shared-nothing way in two different machines, to only combine the logits or select the best continuation at the end. There are different ways to do that, and it works even if the two models have different vocabularies: you can pick the continuation where the perplexity is lower (that is, pick the model which is more sure: it’s like a two experts MoE where the routing is implicit), and it is even possible to combine the logits (with some complexities given by the different vocabularies) and sample from there. More recent papers suggest that mixing the two techniques is the best approach. Anyway: these techniques seem to really work, models appear to do better than alone. It’s like if their knowledge is improved because each one brings his POV on what to say next. Maybe this is one of the most logical third approach to try, other than the first two. I really hope to find the time to play more with all that, in the next months. Comments

0 views
Krebs on Security 1 months ago

Lawmakers Demand Answers as CISA Tries to Contain Data Leak

Lawmakers in both houses of Congress are demanding answers from the U.S. Cybersecurity & Infrastructure Security Agency (CISA) after KrebsOnSecurity reported this week that a CISA contractor intentionally published AWS GovCloud keys and a vast trove of other agency secrets on a public GitHub account. The inquiry comes as CISA is still struggling to contain the breach and invalidate the leaked credentials. On May 18, KrebsOnSecurity reported that a CISA contractor with administrative access to the agency’s code development platform had created a public GitHub profile called “ Private-CISA ” that included plaintext credentials to dozens of internal CISA systems. Experts who reviewed the exposed secrets said the commit logs for the code repository showed the CISA contractor disabled GitHub’s built-in protection against publishing sensitive credentials in public repos. CISA acknowledged the leak but has not responded to questions about the duration of the data exposure. However, experts who reviewed the now-defunct Private-CISA archive said it was originally created in November 2025, and that it exhibits a pattern consistent with an individual operator using the repository as a working scratchpad or synchronization mechanism rather than a curated project repository. In a written statement, CISA said “there is no indication that any sensitive data was compromised as a result of the incident.” But in a May 19 a letter (PDF) to CISA’s Acting Director Nick Andersen , Sen. Maggie Hassan (D-NH) said the credential leak raises serious questions about how such a security lapse could occur at the very agency charged with helping to prevent cyber breaches. “This reporting raises serious concerns regarding CISA’s internal policies and procedures at a time of significant cybersecurity threats against U.S. critical infrastructure,” Sen. Hassan wrote. A May 19 letter from Sen. Margaret Hassan (D-NH) to the acting director of CISA demanded answers to a dozen questions about the breach. Sen. Hassan noted that the incident occurred against the backdrop of major disruptions internally at CISA, which lost more than a third of it workforce and almost all of its senior leaders after the Trump administration forced a series of early retirements, buyouts, and resignations across the agency’s various divisions. Rep. Bennie Thompson (D-MS), the ranking member on the House Homeland Security Committee, echoed the senator’s concerns. “We are concerned that this incident reflects a diminished security culture and/or an inability for CISA to adequately manage its contract support,” Thompson wrote in a May 19 letter to the acting CISA chief that was co-signed by Rep. Delia Ramirez (D-Ill), the ranking member of the panel’s Subcommittee on Cybersecurity and Infrastructure Protection. “It’s no secret that our adversaries — like China, Russia, and Iran — seek to gain access to and persistence on federal networks. The files contained in the ‘Private-CISA’ repository provided the information, access, and roadmap to do just that.” KrebsOnSecurity has learned that more a week after CISA was first notified of the data leak by the security firm GitGuardian , the agency is still working to invalidate and replace many of the exposed keys and secrets. On May 20, KrebsOnSecurity heard from Dylan Ayrey , the creator of TruffleHog , an open-source tool for discovering private keys and other secrets buried in code hosted at GitHub and other public platforms. Ayrey said CISA still hadn’t invalidated an RSA private key exposed in the Private-CISA repo that granted access to a GitHub app which is owned by the CISA enterprise account and installed on the CISA-IT GitHub organization with full access to all code repositories. “An attacker with this key can read source code from every repository in the CISA-IT organization, including private repos, register rogue self-hosted runners to hijack CI/CD pipelines and access repository secrets, and modify repository admin settings including branch protection rules, webhooks, and deploy keys,” Ayrey told KrebsOnSecurity. CI/CD stands for Continuous Integration and Continuous Delivery, and it refers to a set of practices used to automate the building, testing and deployment of software. KrebsOnSecurity notified CISA about Ayrey’s findings on May 20. Ayrey said CISA appears to have invalidated the exposed RSA private key sometime after that notification. But he noted that CISA still hasn’t rotated leaked credentials tied to other critical security technologies that are deployed across the agency’s technology portfolio (KrebsOnSecurity is not naming those technologies publicly for the time being). CISA responded with a brief written statement in response to questions about Ayrey’s findings, saying “CISA is actively responding and coordinating with the appropriate parties and vendors to ensure any identified leaked credentials are rotated and rendered invalid and will continue to take appropriate steps to protect the security of our systems.” Ayrey said his company Truffle Security monitors GitHub and a number of other code platforms for exposed keys, and attempts to alert affected accounts to the sensitive data exposure(s). They can do this easily on GitHub because the platform publishes a live feed which includes a record of all commits and changes to public code repositories. But he said cybercriminal actors also monitor these public feeds, and are often quick to pounce on API or SSH keys that get inadvertently published in code commits. The Private-CISA GitHub repo exposed dozens of plaintext credentials to important CISA GovCloud resources. In practical terms, it is likely that cybercrime groups or foreign adversaries also noticed the publication of these CISA secrets, the most egregious of which appears to have happened in late April 2026, Ayrey said. “We monitor that firehose of data for keys, and we have tools to try to figure out whose they are,” he said. “We have evidence attackers monitor that firehose as well. Anyone monitoring GitHub events could be sitting on this information.” James Wilson , the enterprise technology editor for the Risky Business security podcast, said organizations using GitHub to manage code projects can set top-down policies that prevent employees from disabling GitHub’s protections against publishing secret keys and credentials. But Wilson’s co-host Adam Boileau said it’s not clear that any technology could stop employees from opening their own personal GitHub account and using it to store sensitive and proprietary information. “Ultimately, this is a thing you can’t solve with a technical control,” Boileau said on this week’s podcast . “This is a human problem where you’ve hired a contractor to do this work and they have decided of their own volition to use GitHub to synchronize content from a work machine to a home machine. I don’t know what technical controls you could put in place given that this is being done presumably outside of anything CISA managed or even had visibility on.” Update, 3:05 p.m. ET: Added statement from CISA. Corrected a date in the story (Truffle Security said it found the repo gained some of its most sensitive secrets in late April 2026, not 2025).

0 views
Krebs on Security 1 months ago

CISA Admin Leaked AWS GovCloud Keys on Github

Until this past weekend, a contractor for the Cybersecurity & Infrastructure Security Agency (CISA) maintained a public GitHub repository that exposed credentials to several highly privileged AWS GovCloud accounts and a large number of internal CISA systems. Security experts said the public archive included files detailing how CISA builds, tests and deploys software internally, and that it represents one of the most egregious government data leaks in recent history. On May 15, KrebsOnSecurity heard from Guillaume Valadon , a researcher with the security firm GitGuardian . Valadon’s company constantly scans public code repositories at GitHub and elsewhere for exposed secrets, automatically alerting the offending accounts of any apparent sensitive data exposures. Valadon said he reached out because the owner in this case wasn’t responding and the information exposed was highly sensitive. A redacted screenshot of the now-defunct “Private CISA” repository maintained by a CISA contractor. The GitHub repository that Valadon flagged was named “ Private-CISA ,” and it harbored a vast number of internal CISA/DHS credentials and files, including cloud keys, tokens, plaintext passwords, logs and other sensitive CISA assets. Valadon said the exposed CISA credentials represent a textbook example of poor security hygiene, noting that the commit logs in the offending GitHub account show that the CISA administrator disabled the default setting in GitHub that blocks users from publishing SSH keys or other secrets in public code repositories. “Passwords stored in plain text in a csv, backups in git, explicit commands to disable GitHub secrets detection feature,” Valadon wrote in an email. “I honestly believed that it was all fake before analyzing the content deeper. This is indeed the worst leak that I’ve witnessed in my career. It is obviously an individual’s mistake, but I believe that it might reveal internal practices.” One of the exposed files, titled “importantAWStokens,” included the administrative credentials to three Amazon AWS GovCloud servers. Another file exposed in their public GitHub repository — “AWS-Workspace-Firefox-Passwords.csv” — listed plaintext usernames and passwords for dozens of internal CISA systems. According to Caturegli, those systems included one called “LZ-DSO,” which appears short for “Landing Zone DevSecOps,” the agency’s secure code development environment. Philippe Caturegli , founder of the security consultancy Seralys , said he tested the AWS keys only to see whether they were still valid and to determine which internal systems the exposed accounts could access. Caturegli said the GitHub account that exposed the CISA secrets exhibits a pattern consistent with an individual operator using the repository as a working scratchpad or synchronization mechanism rather than a curated project repository. “The use of both a CISA-associated email address and a personal email address suggests the repository may have been used across differently configured environments,” Caturegli observed. “The available Git metadata alone does not prove which endpoint or device was used.” The Private CISA GitHub repo exposed dozens of plaintext credentials for important CISA GovCloud resources. Caturegli said he validated that the exposed credentials could authenticate to three AWS GovCloud accounts at a high privilege level. He said the archive also includes plain text credentials to CISA’s internal “artifactory” — essentially a repository of all the code packages they are using to build software — and that this would represent a juicy target for malicious attackers looking for ways to maintain a persistent foothold in CISA systems. “That would be a prime place to move laterally,” he said. “Backdoor in some software packages, and every time they build something new they deploy your backdoor left and right.” In response to questions, a spokesperson for CISA said the agency is aware of the reported exposure and is continuing to investigate the situation. “Currently, there is no indication that any sensitive data was compromised as a result of this incident,” the CISA spokesperson wrote. “While we hold our team members to the highest standards of integrity and operational awareness, we are working to ensure additional safeguards are implemented to prevent future occurrences.” A review of the GitHub account and its exposed passwords show the “Private CISA” repository was maintained by an employee of Nightwing , a government contractor based in Dulles, Va. Nightwing declined to comment, directing inquiries to CISA. CISA has not responded to questions about the potential duration of the data exposure, but Caturegli said the Private CISA repository was created on November 13, 2025. The contractor’s GitHub account was created back in September 2018. The GitHub account that included the Private CISA repo was taken offline shortly after both KrebsOnSecurity and Seralys notified CISA about the exposure. But Caturegli said the exposed AWS keys inexplicably continued to remain valid for another 48 hours. CISA is currently operating with only a fraction of its normal budget and staffing levels. The agency has lost nearly a third of its workforce since the beginning of the second Trump administration, which forced a series of early retirements, buyouts, and resignations across the agency’s various divisions. The now-defunct Private CISA repo showed the contractor also used easily-guessed passwords for a number of internal resources; for example, many of the credentials used a password consisting of each platform’s name followed by the current year. Caturegli said such practices would constitute a serious security threat for any organization even if those credentials were never exposed externally, noting that threat actors often use key credentials exposed on the internal network to expand their reach after establishing initial access to a targeted system. “What I suspect happened is [the CISA contractor] was using this GitHub to synchronize files between a work laptop and a home computer, because he has regularly committed to this repo since November 2025,” Caturegli said. “This would be an embarrassing leak for any company, but it’s even more so in this case because it’s CISA.”

0 views
Stratechery 1 months ago

Data Center Discontent, Understanding the Opposition, Fixing the Problem

There are understandable reasons for people to oppose data centers; the only solution that will work is simply paying them off.

0 views
Martin Alderson 2 months ago

Managed agents are the new Lambda

Managed agents (cloud-hosted agents) are the next big push from the frontier labs. They're genuinely incredible. They're also going to be the AWS Lambda of this cycle - powerful, sticky, and an absolute nightmare to migrate off once you're in deep. While the exact definition is up for debate, in my mind a managed agent is an agent harness (like Claude Code) running in the cloud , not on your local machine. This has a few major advantages. The most obvious one is that you don't need a machine running locally - it can do its work 24/7, in the background. The other that running in the cloud means it can be notified of changes and act on them. Imagine, for example, agents responding to incoming emails or webhooks and doing some activity based on them (this is very possible locally - but easier with the agent running on the server). The other advantage is security - probably the key part of the "managed" agent. Much like PaaS (platform-as-a-service) products like Heroku, AWS ECS/App Runner/Lambda and Azure App Service/Functions, the provider manages not just the underlying physical infrastructure for you, but also manages patching the operating system and related server software on your behalf. Sandboxing is another related benefit. Managed agents only get access to what you give them - no risk of an agent wandering into files it shouldn't. If you're already running Claude Code/Codex/OpenCode in Docker on a server, you've basically built one yourself. The frontier labs are just productising the pattern. Anthropic has really been pushing their managed agents product hard lately. This makes a lot of sense - cloud hosted agents are genuinely incredible in what they can do - but I'd urge real caution on locking yourself into a vendor - at least at this point. Fundamentally, agents are not particularly difficult to swap out. While there are important differences and nuances in how they work and operate, switching from Claude Code to Codex (or OpenCode, or Pi, or one of the many other agent harnesses) is a fairly simple process. Fundamentally the pattern is the same - run a harness with a prompt, context and tools and capture output and logs. All agent harnesses have the same primitives. And at least having the ability to swap the agent harness and model out is really important. Clearly pricing is one important dimension, but equally so is being able to use new models from different labs. The competition is absolutely cutthroat and shows absolutely no sign of slowing down. Once you start using a managed agent product from a frontier lab this gets far more difficult. A lot of your data and workflows are embedded in their cloud. While Anthropic have gone to lengths to say it is your data and it can be exported, in my many years of experience of vendor lock in this definition drifts and gets harder and harder to migrate to another provider. As many people found out with AWS, moving Docker container workloads is fairly easy if you want to move hyperscaler clouds. Moving AWS Lambda [1] functions is far, far more difficult - I've seen organisations spend months upon months unpicking Lambda code and assumptions when they realise it isn't a good fit after all the hype dies down. Yesterday Anthropic announced huge changes to their pricing model which underlines this point. If you run Claude Code non-interactively (which includes nearly all cloud-hosted agent usage - and many others [2] ), these now are not eligible for your subscription token allowances and will instead use some new credit. After this allowance is exhausted then it is very expensive API tokens ahead. It's fair to say if you were using a lot of "non interactive" Claude Code you are looking at a 5-20x price increase with these changes. It's clearly Anthropic's prerogative to do this - and (I think) points to their compute shortages more than anything, but it has given OpenAI a real opening for users to switch to Codex - OpenAI (currently, at least) have been very explicit you can use your included allowances on your plan with any tool and however you like. Expect to see a lot more talk around Codex (which has been already gaining significant traction over the past few months) and other providers in the future - developers are often remarkably price sensitive around things like this, especially for personal 'side projects' - which often then end up informing enormous purchasing decisions in the companies they work in months and years down the line. [3] Now it's easy to say don't use a frontier lab's managed agent product, but what are the solutions? I think there's two main ways you can solve this in your organisation. Firstly, roll your own managed infra. This is a good option for developers and tech adjacent teams - they will have the expertise to do this. Essentially, it's just running a Docker container which they do all day every day. Using something like OpenCode as a harness allows you to use any model provider and switch between them in minutes. Secondly, there's a flood of startups and other companies that allow you to run managed agents with any model or provider you want. I haven't (yet) evaluated them in detail as the market landscape is switching so fast to give any real thoughts on quality, but providers include Cloudflare Agents, Vercel and the hyperscaler options (AWS AgentCore, Azure AI Foundry and GCP Vertex AI Agent Engine). My personal view is until this shakes out a bit more, stick to self hosting them. It's not difficult, allows you to secure them inside your current infrastructure and builds organisational competence around agent primitives. Outsourcing this knowledge at this point is a path to serious organisational knowledge gaps. However, expect this to change as the platforms introduce more capabilities that become more and more difficult to replicate. One to keep an eye on. The one ointment in this plan is that I have a strong gut feeling the frontier labs are going to start introducing new models and capabilities that are only available on their managed agent platforms. This is where the pendulum (maybe) starts swinging to having to use managed agents - but again, maybe not. Lambda is a way of running applications "serverless" which in theory allows much easier deployment and scaling - more of the primitives of application hosting is abstracted. However, it means you start really having to lean into AWS specific code, techniques and patterns, that can be really difficult to revert ↩︎ It also includes alternative frontends to Claude Code, like the excellent Conductor Mac app, despite this really being the definition of interactive usage. ↩︎ This is why I really hope that Anthropic rethinks this at some point. ↩︎ Lambda is a way of running applications "serverless" which in theory allows much easier deployment and scaling - more of the primitives of application hosting is abstracted. However, it means you start really having to lean into AWS specific code, techniques and patterns, that can be really difficult to revert ↩︎ It also includes alternative frontends to Claude Code, like the excellent Conductor Mac app, despite this really being the definition of interactive usage. ↩︎ This is why I really hope that Anthropic rethinks this at some point. ↩︎

0 views
Stratechery 2 months ago

The Deployment Company, Back to the 70s, Apple and Intel

Listen to this post: Good morning, President Trump is on the way to China, and Sharp China is your go-to podcast for understanding what happens next. Add it to your podcast player now in anticipation of the next few episodes breaking down the trip. On to the Update: From Reuters : OpenAI said on Monday it is setting up a new company with more than $4 billion in initial investment to help organizations build and deploy artificial intelligence systems, and will acquire an AI consulting firm, Tomoro, to quickly scale up the unit. After its early models saw strong resonance with consumers, OpenAI has been working aggressively to sign corporate contracts and establish a large presence in the business world where its AI will see large-scale deployment. The venture, which will be majority owned and controlled by OpenAI, also comes as rival Anthropic enjoys strong success in its enterprise AI push with its Claude family of models seeing rapid adoption among businesses. The new firm, called OpenAI Deployment Company, will help the ChatGPT maker embed engineers specializing in frontier AI deployment into organizations that will then work closely with various teams to identify where AI can make the biggest impact, OpenAI said. Its acquisition of Tomoro, a consulting firm that helps enterprises deploy AI, will bring around 150 experienced AI engineers and “deployment specialists” to the new unit from day one. Tomoro was formed in 2023 in alliance with OpenAI, and counts companies such as Mattel, Red Bull, Tesco and Virgin Atlantic as its clients, according to its website. That was on Monday; on Tuesday, from The Information : Google plans to hire hundreds of engineers to help customers start using its business-focused AI products, according to a person familiar with the situation. Google’s new “forward deployed engineers” will form a new team within Google Cloud, the unit’s chief, Thomas Kurian, said on LinkedIn on Tuesday, without disclosing the size of the effort. Matt Renner, Google Cloud’s chief revenue officer, said in a separate post that the move would help Google “show up for our customers with more technical resources (vs just an ocean of salespeople).” The announcement is one of several in the industry in recent weeks as tech companies are deploying armies of humans—often described as “forward deployed engineers”—and partnerships with consulting companies to get customers using AI-driven technology intended to automate work. On Monday, OpenAI launched the “OpenAI Deployment Company” in partnership with consulting and investment firms. Last week, Anthropic announced the creation of a joint venture with private equity firms to sell its AI to the PE firms’ customers. It is, needless to say, tempting to drop some snark about AGI apparently not being good enough to deploy AI, but instead I’m going to go with “as predicted”. In 2024’s Enterprise Philosophy and the First Wave of AI , I made the case that the proper analogy for AI in the enterprise was not SaaS, but rather the first wave of computing in the 1970s. Agents aren’t copilots; they are replacements. They do work in place of humans — think call centers and the like, to start — and they have all of the advantages of software: always available, and scalable up-and-down with demand…Benioff isn’t talking about making employees more productive, but rather companies; the verb that applies to employees is “augmented”, which sounds much nicer than “replaced”; the ultimate goal is stated as well: business results. That right there is tech’s third philosophy: improving the bottom line for large enterprises. Notice how well this framing applies to the mainframe wave of computing: accounting and ERP software made companies more productive and drove positive business results; the employees that were “augmented” were managers who got far more accurate reports much more quickly, while the employees who used to do that work were replaced. Critically, the decision about whether or not to make this change did not depend on rank-and-file employees changing how they worked, but for executives to decide to take the plunge. Specifically, I don’t think that the Deployment Company is going in to help employees use chatbots; that’s even more clearly the case with the PE firms that both OpenAI and Anthropic are doing deals with. I expect there to be an ever-increasing number of deals where PE buys software firms with reliable cash flows and conducts significant layoffs, forcing AI to pick up the slack, solving stock-based compensation issues in the process. I don’t know if the mandate for the Deployment Company is going to be quite so harsh, but I assume this is a company that is hired by the executive suite to fundamentally rethink business processes in a way that hasn’t been done since the mainframe: Most historically-driven AI analogies usually come from the Internet, and understandably so: that was both an epochal change and also much fresher in our collective memories. My core contention here, however, is that AI truly is a new way of computing, and that means the better analogies are to computing itself. Transformers are the transistor, and mainframes are today’s models. The GUI is, arguably, still TBD. To the extent that is right, then, the biggest opportunity is in top-down enterprise implementations. The enterprise philosophy is older than the two consumer philosophies I wrote about previously: its motivation is not the user, but the buyer, who wants to increase revenue and cut costs, and will be brutally rational about how to achieve that (including running expected value calculations on agents making mistakes). That will be the only way to justify the compute necessary to scale out agentic capabilities, and to do the years of work necessary to get data in a state where humans can be replaced. The bottom line benefits — the essence of enterprise philosophy — will compel just that. What I wonder is how much of the work ends up reworking data; that, as I noted in that article, is why I was bullish on Palantir: That leaves the data piece, and while Benioff bragged about all of the data that Salesforce had, it doesn’t have everything, and what it does have is scattered across the phalanx of applications and storage layers that make up the Salesforce Platform. Indeed, Microsoft faces the same problem: while their Copilot vision includes APIs for 3rd-party “agents” — in this case, data from other companies — the reality is that an effective Agent — i.e. a worker replacement — needs access to everything in a way that it can reason over. The ability of large language models to handle unstructured data is revolutionary, but the fact remains that better data still results in better output; explicit step-by-step reasoning data, for example, is a big part of how o1 works. To that end, the company I am most intrigued by, for what I think will be the first wave of AI, is Palantir… That integration looks like this illustration from the company’s webpage for Foundry, what they call “The Ontology-Powered Operating System for the Modern Enterprise”: What is notable about this illustration is just how deeply Palantir needs to get into an enterprise’s operations to achieve its goals. This isn’t a consumery-SaaS application that your team leader puts on their credit card; it is SOFTWARE of the sort that Salesforce sought to move beyond. Google’s Kurian, by the way, did dismiss any sort of Palantir comparison in a Stratechery Interview last month: This all makes perfect sense, particularly this bit about the Knowledge Catalog definitely fits how I’ve been thinking. I wrote about this a few years ago about this importance of this whole layer and understanding it, it’s a bit of a big lift to get this in place. You have some sort of analog, say, with like a Palantir that’s putting in like their ontology thing. They have FDEs out on the site, multi-month projects doing this. You have OpenAI talking about Frontier, their agent layer, and they’re partnering with all the tech consultancies to build this out. Is this going to entail a lot of boots on the ground to get this graph working and functional in a way that your agents can operate effectively across it? TK: We’re not competing with Palantir, we’re not building a semantic dictionary or an ontology. What we’re doing is, today I’ll give you the closest analogy. TK: Today when you use a model, let’s say you use Gemini, and you ask a question, Gemini goes through reasoning, and then it shows you a citation. A citation is, “How did I answer the question and what’s the source I derived from?” Now imagine that citation was a query that needed to go to a folder in, for example, a storage system because there’s some documents there and a database because, for example, in a part number, just think about there’s a part number document that lists all the part numbers and sits in a drive and then that part number you need to fetch out to say it’s the modem that the guy is coming to repair, and that’s mapped to a table in a database. So what the graph does, we use Gemini, so we don’t need humans, we use Gemini to say, “Hey, go and read all these documents in these drives and extract the information from it and then match that to the database table that has the reference to the part number”, and so then when Gemini turns around and says, “I got this query about how much inventory of modems they are”, the first thing it does is it says, “Okay, go to the Knowledge Catalog and it says modem is part number one, two, three, four, five”, and then it says, “By the way the table in the database that has the inventory information about this part number is this table, here’s a SQL”, it then makes the quality of what we generate higher and then when it answers the question it shows back — back to your, “Trust my data”, it shows a grounding citation saying, “That’s where we got it from.” Well, so much for not needing humans! I joke, mostly — Kurian was referring to not needing a Palantir-like ontology, not necessarily dismissing the need for FDEs — but it sure is interesting how AI is creating the need for new kinds of jobs. It’s almost as if the world is more dynamic, and pure intelligence, unadulterated by what already exists and the burden of reflexivity, is more static, than the most pessimistic prognosticators may have anticipated. More prosaically, OpenAI and Anthropic need the revenue, enterprises need the imagination, and Google needs to stay in the game. From the Wall Street Journal : Apple and Intel have reached a preliminary agreement for Intel to manufacture some of the chips that power Apple devices, according to people familiar with the matter. Intensive talks between the two companies have been ongoing for more than a year, and they hammered out a formal deal in recent months, these people said. Bloomberg News previously reported the talks. It’s still unclear which Apple products Intel would make chips for, these people said. Apple ships more than 200 million iPhones a year as well as millions of iPads and Mac computers. Ming-Chi Kuo reported on X late last year that Intel would make Apple’s most basic M processor on its 18A process; he didn’t specify which generation. Regardless, while the Wall Street Journal cites Trump administration pressure, and an earlier Bloomberg article Apple’s concentration risk on TSMC and Taiwan, the most obvious reason for a deal — assuming it exists — is economic. Specifically, Apple has for two quarters running said it can’t satisfy demand because it can’t get enough capacity at TSMC. CEO Tim Cook referenced this point multiple times on the last earnings call , but I think this was the most important articulation: The constraint in the March quarter and the June quarter, the primary constraint is the availability of the advanced nodes our SoCs are produced on, not memory. And so I don’t want to predict for supply and demand to match because if I look at it realistically, I think on the Mac mini and the Mac Studio, I believe it will take several months to reach supply-demand balance. And so we’re not at the point where we’re saying this is going to end anytime soon. And it’s not because of a problem per se other than we just undercalled the demand. And there are lead times to this, as you well understand, and it takes a while to correct that. And the primary constraint from a product point of view, or the majority of it for this quarter, for the June quarter will be on the Mac. And it’s Mac mini, Mac Studio and the MacBook Neo. It’s all of those. Cook talked about lead times last quarter as well, and the important thing to note is that while it does take five months or so to make new chips, assuming Apple realized it needed more iPhone 17 Pro chips right away, those new A19 Pro lines only started producing chips partway through last quarter (which is why iPhone 17 Pro sales weren’t as high as they could be). Critically, however, what seems likely is that Apple took capacity away from the Mac to make more iPhone chips, and now doesn’t have enough chips for the Mini and Studio either. The long-and-short of it is this: Apple doesn’t have flexible access to TSMC capacity anymore, because so much of that capacity is going to AI in particular, and it’s costing Apple meaningful money across multiple product lines. This was always the thing that would bring companies to Intel; I wrote in TSMC Risk : Becoming a meaningful customer of Samsung or Intel is very risky: it takes years to get a chip working on a new process, which hardly seems worth it if that process might not be as good, and if the company offering the process definitely isn’t as customer service-centric as TSMC. I understand why everyone sticks with TSMC. The reality that hyperscalers and fabless chip companies need to wake up to, however, is that avoiding the risk of working with someone other than TSMC incurs new risks that are both harder to see and also much more substantial. Except again, we can see the harms already: foregone revenue today as demand outstrips supply. Today’s shortages, however, may prove to be peanuts: if AI has the potential these companies claim it does, future foregone revenue at the end of the decade is going to cost exponentially more — surely a lot more than whatever expense is necessary to make Samsung and/or Intel into viable competitors for TSMC. This, incidentally, is how the geographic risk issue will be fixed, if it ever is. It’s hard to get companies to pay for insurance for geopolitical risks that may never materialize. What is much more likely is that TSMC’s customers realize that their biggest risk isn’t that TSMC gets blown up by China, but that TSMC’s monopoly and reasonable reluctance to risk a rate of investment that matches the rest of the industry means that the rest of the industry fails to fully capture the value of AI. We’re already here (reportedly). TSMC’s failure to invest aggressively enough over the last several years will, in the end, give Intel the single most important thing it needs to become a viable competitor: the customer who did more than any other to make TSMC into the leader in the first place. This Update will be available as a podcast later today. To receive it in your podcast player, visit Stratechery . The Stratechery Update is intended for a single recipient, but occasional forwarding is totally fine! If you would like to order multiple subscriptions for your team with a group discount (minimum 5), please contact me directly. Thanks for being a subscriber, and have a great day!

0 views