Posts in Sql (20 found)

How to stay in the coding flow using LLMs

We all know that moving to LLMs and agents has caused the feeling of losing touch with parts, or maybe even all, of a code base. This isn’t just something that is problematic for managing and handling the translation from business logic to implementation it is a problem because it feels exhausting . I’ve had coding sessions that lasted 12 hours and afterwards felt great. Meanwhile I’ve done LLM prompting for a few hours and felt exhausted or unsure of what I did. Lately I’ve been keeping this in mind and have been looking for a few ways in which I can maintain a flow state and take advantage of LLMs. Remember back in 2025 when this was the default way of using them? I actually still find this to be my preferred way. Using LLMs with code harnesses in projects injects so much unnecessary information that asking simple questions gets out control. For example, here I’m exploring some data, and I wanted a quick regex, I turned over to my VSCode chat window, and forgot that it was an agent, and asked it the question. It proceeds to start looking at the files, wanting to run code etc. All off target of what I need . So next I switched VSCode to “Ask” instead of agent, again the LLM is flooded with context about my project and proceeds to output a massive amount of distracting and off topic code suggestions. Switch to a browser chat window which has little to no context about what you’re working on and ask it my specific question, boom it spits out a few quick regexes for my Python list comprehension that are exactly what I need . Is this bad advice? Well, maybe. But was this what you’re already doing, definitely. But the point here is to multitask coding on more than one thing at a time. I’ve found that this keeps me in the flow state much better than if I let myself browse the news. So instead of switching from your agent -> browse social media switch between multiple projects. This depends on how your code / work is structured, but depending on the scope this means either switching between several agents in the same project or having several projects open at once. Types of positive multitasking to stay in the zone: For me, working on AppGoblin’s free ASO and mobile app ecosystem data , I have certain areas that *I* need to understand what is happening, for those reasons I do not let AI write anything more than boiler plate code. The clearest example of this I can give is SQL, where a lot of my most important relational logic exists. Sure, I can let an LLM one shot a complicated SQL and it will “work” but come weeks (or months!) later and I’ll find a complicated bug that slipped in. It’s not even necessarily about who was right/wrong in this situation, it’s that *I* need to know what’s going on in certain parts of the codebase. Something that ‘looks fine’ is a terrible feeling that later it was not what I wanted. This last one is probably best suited for other data crunchers out there, but it’s where I find a great sweet spot for staying in the zone. My favorite way to write code has always been to write code in an editor and send line to a REPL. This is also more or less how SQL gets written as well where you build queries in your SQL editor by slowly making changes to the data, checking values / assumptions and eventually getting to your final SQL query. With the LLMs, I find myself using this flow lately: It’s more or less the same as I did before, just a lot less writing and let’s me hold onto the difficult concepts longer. If you’re actually in the flow of editing code, the best way to augment your coding is with code completion. I’ve found this to be the most powerful in that I don’t even have to start letting my mind wander for how to do some boiler plate code, it just pops up automatically. I love this because it helps me think at a high level in the code without the distractions of trying to remember how to do something when the how is not the important part. Probably the only issue with this is that code completion can be quite annoying and distracting in some situations. For example, writing free form and handling imports at the top of a file are examples where LLM ‘helpful’ code completion is just not helpful. If you enjoyed this feel free to share. Working on related projects File and project cleanup. LLMs generate many extra files and code and it’s best to stay on top of that yourself. Go through and delete extra files. Try asking LLMs for advice on what to remove, but do be careful with this idea. Tell LLM to write new code for processing data Step through the code my self line by line, checking the hotspots where I know assumptions / tricky data might be

0 views
Evan Hahn 5 days ago

Prefer STRICT tables in SQLite

In short: I prefer strict tables in SQLite because they avoid some datatype problems, such as putting text in number columns. SQLite has a feature that I think is underrated: strict tables . Strict tables help enforce rigid typing, preventing mistakes like putting text into integer columns. I like them, and wrote this post to promote their use! To make a strict table, add to the end of its definition. Like this: That’s it! But what does it do? Broadly, strict tables help enforce rigid types, like other SQL engines do. Most significantly, strict tables keep you from inserting the wrong type into a column. For example, SQLite normally lets you put text into an column, but not with strict tables. Personally, I think it’s a mistake to try to put text in an integer column, or vice-versa. I don’t want SQLite to let me make this error! The same validation happens for s, too. Notably, if a value can be losslessly converted, it will still be accepted. For example, the string can be perfectly converted to an integer, so it’s allowed. These two lines are equivalent, even for a strict table: By default, you can create columns with bogus types. For example, all of these work even though they aren’t valid SQLite datatypes: I think these aren’t what the developer intended. Some of these are typos, some of them are misunderstandings of which datatypes SQLite supports , and some are egregious mistakes. Appending to any of these statements makes them error. In my opinion, that’s the correct behavior! Only , , , , , and are allowed. Strict tables also require a column type, so you can’t do . If you still need a column to be flexible, you can use the datatype. As the name suggests, it allows anything—even in a strict table. I haven’t found a use for this, but maybe you will! I prefer strict tables but I must share a few cons. Not everything is better! I think it’s best to use strictness from the start, but that’s not always possible. Unfortunately, I don’t think there’s a way to a table to make it strict. I think you have to copy the data out of the non-strict table into the strict one. Something like this: Note that this could be tricky if the non-strict table has invalid data! For example, if the old data accidentally contains text in an integer column, you’ll get errors when doing the migration. You’ll probably need to clean the data or cast it . You could make a rule for your codebase that all new tables are strict. That might be useful—at least some of your tables are valid! But it might also mean you have inconsistent validation across your tables, which might be more surprising than having weak validation on all tables. It’s up to you to decide whether this is a good fit for you. SQLite has a whole page called “The Advantages Of Flexible Typing” , where they argue that SQLite’s flexible behavior is good, actually. I hesitate to wade into the controversy of static-versus-dynamic, but I disagree in most cases. I’ve personally encountered many bugs where an unexpected data type caused subtle headaches. I’d much rather these mistakes explode loudly. But it’s worth noting that SQLite’s developers seem not to share my preference for strict tables! They point out a few good uses for flexible tables, such as “a pure key-value store” or “a place to store miscellaneous attributes” of different types. They also mention that you might want to keep the invalid data in some cases, like if you’re directly importing a messy CSV and don’t want to lose any data. I still prefer strict tables, but acknowledge there are some reasonable cases for non-strict ones. (There’s also at least one comment in the SQLite source that calls non-strict tables “legacy” , but I trust that less than the official documentation.) SQLite introduced strict tables in version 3.37.0 , released November 2021. If you’re on an older version of SQLite, you can’t use strict tables. It’s worth noting that old versions of SQLite can’t read databases with strict tables. For example, if you create a strict table in the newest version of SQLite and then try to read that database in SQLite 3.36.0 (before strict tables were added), you’ll get an error—even if the strict table is already in the database. Strict tables are theoretically slower because they have to do a little extra work. For example, they check datatypes when doing an insert or update . But in practice, I don’t think this is an issue. I wrote a hacky script that inserted millions of rows into a table with 100 columns, and there was no obvious difference on multiple machines I tried. The file size on disk was also the same. I didn’t test this thoroughly, so maybe there’s something I missed, but I don’t think strict tables present a performance problem. In fact, one might expect better performance because you won’t be accidentally mismatching SQLite’s column affinities. But again, I haven’t tested this. Personally, I think the pros of strict tables outweigh the cons. I generally prefer when types are rigidly enforced. It squashes a class of mistakes, and help enforce good data integrity. They’re not a panacea, but they’re usually easy to add and go a long way. If there’s a SQLite feature you think is underrated, please tell me .

0 views
Simon Willison 1 weeks ago

sqlite-utils 4.0, now with database schema migrations

This morning I released sqlite-utils 4.0 , the 124th release of that project and the first major version bump since 3.0 in November 2020. In addition to some small but significant breaking changes (described in this upgrade guide ), this version introduces three major features: database migrations , nested transactions (via a new method), and support for compound foreign keys . Schema migrations define a sequence of changes to be made to a SQLite database, plus a mechanism for tracking which migrations have been applied and applying any that are found to be pending. Migrations are defined in Python files using the sqlite-utils Python library , which includes a powerful method providing enhanced alter table capabilities that are not supported by SQLite's statement. ( implements the pattern recommended by the SQLite documentation - create a new temporary table with the new schema, copy across the data, then drop the old table and rename the temporary one in its place.) Here's an example migration file which creates a table called , adds an additional column to it in a second step, then changes the types of two of the columns in a third: Save that as and run it against a fresh database like this: Then if you check the schema of that database: You'll see this SQL: The table is used to keep track of which migration functions have been run. The table above is the schema after all three migrations have been applied. To see a list of migrations, both pending and applied, run this: If you don't specify a migrations file, the command will scan the current directory and its subdirectories for files called and apply any instances it finds in them. You can also execute migrations from Python code using the method, which is useful for building tools that manage their own database schemas over multiple versions. My own LLM tool has been using a version of this pattern for several years now, as shown in llm/embeddings_migrations.py . My favorite implementation of this pattern remains Django's Migrations , developed by Andrew Godwin based on his earlier project South . Fun fact: Andrew, Russ Keith-Magee, and I presented our competing approaches to schema migrations for Django on the Schema Evolution panel at the very first DjangoCon back in 2008! My attempt was called dmigrations , developed with a team at Global Radio in London. Django's migrations can be automatically generated from model definitions and include the ability to roll back to a previous version. The approach is deliberately simpler: unlike Django, encourages programmatic table creation rather than a model definition ORM, so there isn't anything we can use to automatically generate migrations. I decided to skip rollback, since in my experience it's a feature that is rarely used. With a SQLite project, an easy way to achieve rollback is to create a copy of your database file before you apply the migrations! The design of migrations is three years old now - I had originally released it as a separate package called sqlite-migrate , which never quite graduated beyond a beta release. I've used that package in enough places now that I'm confident in the design, so I've decided to promote it to a feature of to make it available by default to all of the other tools in the growing sqlite-utils/Datasette/LLM ecosystem. I made one last release of , which switches it to depend on and replaces the file with the following: Any existing project that depends on should continue to work without alterations. Here are the release notes for this version, with some inline annotations: The 4.0 release includes some minor backwards-incompatible fixes (hence the major version number bump) and introduces three major new features: I think of migrations as the signature new feature, hence this blog post. has long had a confused relationship with database transactions, partly because when I started designing the library back in 2018 I didn't yet have a great feel for how those worked in SQLite itself. Adding migrations to the core library made me determined to finally crack this nut, since transactions make migration systems a whole lot safer and easier to reason about. I ended up building this around a context manager which looks like this: SQLite supports Savepoints , and as a result can be nested to carry out transactions inside of transactions. It's pretty neat! This came about when I asked a coding agent to review all open issues and PRs for things that should be included in a 4.0 release since they would represent breaking changes if I added them later, and it correctly identified that compound foreign keys were exactly that kind of feature. I started with a breaking change to the table.foreign_keys introspection method, and then decided to see if Claude Fable 5 could handle the more fiddly job of integrating compound foreign key creation into the library. The API design it helped create felt exactly right to me - consistent with how the rest of the library worked already. Other notable changes include: This was the change that first pushed me to consider a breaking-change 4.0 version bump. I built this to help support sqlite-chronicle , which uses triggers to keep track of rows in a table that have been inserted, updated or deleted. Probably the most disruptive breaking change - I've had to update a few places in my own code to switch from to as a result. The flag was a later addition to allow column types (text, integer, real) to be automatically detected based on the data in a CSV. It should be the default, and releasing a 4.0 means I can make it so. The oldest issue addressed by this release - the underlying bug was opened (by me) in October 2020. See Upgrading from 3.x to 4.0 for details on backwards-incompatible changes. The detailed release notes for the features and fixes shipped during the 4.0 pre-release cycle are available in 4.0a0 , 4.0a1 , 4.0rc1 , 4.0rc2 , 4.0rc3 and 4.0rc4 . The upgrade guide was entirely written by Claude Fable 5, Claude Opus 4.8 and GPT-5.5. The same is true of the release notes. This is the kind of documentation I've slowly become comfortable outsourcing to the robots. It doesn't need to convince people of anything, or express any opinions - its job is to be as accurate and detailed as possible. I've reviewed the release notes closely and can confirm they are accurate and comprehensive. I released the first alpha of sqlite-utils 4.0 over a year ago . I've been dragging my heels on the stable release because of the amount of work it would take to track down and clean up the many other minor design flaws that a major version number allowed me to take on. Assistance from Claude Fable 5 (and to a lesser extent Opus 4.8 and GPT-5.5) gave me just the boost I needed to overcome inertia and make the most of the time I could afford to spend on this library. Fable has really good taste in API design, and is relentlessly proactive if you give it a more open goal. My most successful prompt was a review task that I issued against what I thought was the last release candidate: I tried this with GPT-5.5 xhigh in Codex Desktop and Fable 5 in Claude Code. GPT-5.5 wrote 5 Python scripts and didn't turn up anything particularly interesting - its final report is here . Fable 5 wrote 12 scripts , identified 4 release blockers and 10 additional issues in its report , and built a neat combined repro script , which, when run, output the following: I found myself agreeing with almost all of them. Here's the PR with 16 commits where we worked through them in turn. There's no doubt in my mind that sqlite-utils 4.0 is a significantly higher-quality release than if I had built it without the assistance of the latest frontier models. You are only seeing the long-form articles from my blog. Subscribe to /atom/everything/ to get all of my posts, or take a look at my other subscription options . Database migrations , providing a structured mechanism for evolving a project’s schema over time. ( #752 ) Nested transaction support via , plus numerous improvements to how transactions work across the library. ( #755 ) Support for compound foreign keys , including creation, transformation and introspection through table.foreign_keys . ( #594 ) Upserts now use SQLite’s syntax, detect existing table primary keys automatically and reject records that are missing required primary key values. ( #652 ) now executes immediately and rejects statements that do not return rows; use for writes and DDL. CSV and TSV imports now detect column types by default, while inserts into existing tables preserve those tables’ column types. ( #679 ) and no longer create lookup table records for all- values. ( #186 )

0 views
David Bushell 1 weeks ago

Behold the perfect algorithm!

1984, Minority Report, Black Mirror — bedtime stories compared to the horrors the UK Government publish, am I right? I’m led to believe “Watch this space” is the latest propaganda piece from His Majesty’s Nanny State . I haven’t read past the title but according to gaming site Dexerto, YouTube lawyers read it and YouTube ain’t happy. Poor little YouTube. The government is consulting on options, considering whether to make public service news easier to discover on sites like YouTube and TikTok, with greater prominence and with more visibility during periods of major public importance. It also seeks to discuss misinformation and online viewing habits. YouTube urges creators to fight proposed UK algorithm changes - Matthew Benson, Dexerto I glossed over the Dexerto article too. This whole thing is something about kids being hooked on Skibidi and not paying their racketeering license fee . Minecraft Let’s Plays will be spliced with a BBC impartiality report on what some fascist gammon thinks. Should the proposal become law, of course. This is somewhat of a dilemma for a guy like me. If there’s one thing I hate more than a meddling GOV.UK, that might just be Big Tech . The thought of Google et al being ruffled warms my heart like a hot cup of tea on the summer solstice. That was too many words on something I never read so I’ll get to the lede. I’m about to reveal the secret sauce that Big Tech has tried to suppress. The one true algorithm, which ironically might be their saviour. Only one parameter is required in the perfect algorithm: who I choose to follow. I’m literally providing the exact data needed to curate my feed. I know what defenders of the deceptive arts are thinking: but algorithms are proven to increase engagement! — I know, Sherlock. Do you enjoy your doomscrolling misery? Not every metric needs to be min-maxed at the expense of human health. Modern apps sucks. Modern media sucks. Stick your “algorithm”. † It’s been decades since I studied SQL and database normalisation so please have mercy. Thanks for reading! Follow me on Mastodon and Bluesky . Subscribe to my Blog and Notes or Combined feeds.

0 views
Simon Willison 1 weeks ago

sqlite-utils 4.0rc2, mostly written by Claude Fable (for about $149.25)

I wrote about the sqlite-utils 4.0rc1 release a couple of weeks ago. Since we only have Claude Fable on our Max subscriptions for a few more days, I decided to see if it could help me get to a 4.0 stable release that I felt truly comfortable about, since I try to keep to SemVer and like my incompatible major versions to be as rare as possible. I started with this prompt, in Claude Code for web on my iPhone: Here's that initial report it created for me. There were some significant problems that I hadn't myself encountered yet - 5 that Fable categorized as "release blockers". Here's the worst of the bunch: 1. never commits and poisons the connection (data loss) ( ) runs its DELETE via a bare with no wrapper — compare at , which wraps correctly. The connection is left , so every subsequent call takes the savepoint branch ( ) and never commits either. Reproduced end-to-end: That's a really bad bug! Very glad I didn't ship that, although at least it would have been a bug I could fix in a 4.0.1 point release, not a design flaw that would force a 5.0. Over the course of 37 prompts, 34 commits and +1,321 -190 code changes over 30 separate files, we worked through the entire set of feedback in turn, making several other design improvements along the way. A weird thing about coding agents is that harder tasks like this one actually provide more opportunity to do other things at the same time, since the agent sometimes needs 10-15 minutes to churn away on a new task. I went out to enjoy the Half Moon Bay 4th of July parade, occasionally checking in and prompting the next step for Fable from my phone. Full details in the PR and this shared transcript . I switched to my laptop for the final review, which I conducted through GitHub's PR interface. The most significant changes relate to transaction handling, which was the signature new feature in the earlier RC . The new RC now includes comprehensive documentation on the new transaction model, the intro to which I'll quote here in full: Every method in this library that writes to the database - , , , , , , , , and the rest - runs inside its own transaction and commits it before returning. Your changes are saved to disk as soon as the method call finishes: The same applies to raw SQL executed with db.execute() - a write statement is committed as soon as it has run. You never need to call , and you do not need to close the database to persist your changes. There are exactly two situations where you need to think about transactions: You want to group several write operations together, so they either all succeed or all fail - use db.atomic() . You are managing a transaction yourself with , in which case nothing is committed until you commit - the library will never commit a transaction you opened. In reviewing Fable's documentation - I find that reviewing the documentation edits first is an excellent way to build an initial understanding of what has changed - I spotted this detail : and the automatic per-method transactions are designed for connections in Python's default transaction handling mode. Connections created with the Python 3.12+ or options are not supported, because and behave differently on those connections. I admit I hadn't thought about how would react to the more recent autocommit setting , added in Python 3.12. It turns out "behave differently on those connections" equated to almost the entire test suite failing, so I worked with the model to ensure that this difference would not break how the library works. I used to think that the idea of having one model review the work of another was somewhat absurd - it felt weirdly superstitious. The problem is it really does work - I've started habitually having Anthropic's best model review OpenAI's work and vice versa, because I've had that turn up interesting results often enough to be valuable. I prompted Codex Desktop and GPT-5.5 xhigh with the following: Which was enough to turn up two issues worth investigating: I pasted that into a fresh Fable session, which ran some experiments to confirm the problem: Both findings were confirmed. called first, which auto-commits writes, and only then checked — so committed the update before raising . And the commit lived at the end of the returned generator, so it never fired unless you exhausted the iterator — or an un-iterated call left the transaction open, contradicting what the changelog and docs promise. Here's the PR with the fix, and the full Claude Code transcript . Reviewing this code helped me build a better mental model of the edge cases of SQLite transaction semantics! I upgraded to the Claude Max $200/month plan (I was previously on $100/month) to increase my Fable allowance for the remaining time until the July 7th Fablepocalypse , when even Claude Max subscribers will have to pay full API cost for the model. I was curious as to how much this would have cost me if I had been paying those costs directly. At first I thought those numbers weren't available to me since I had run the work remotely using Claude Code for web, and then I realized I could run AgentsView inside that existing session to get that cost estimate! Claude figured out how to use the command and came out with the following: I'm very glad I'm on that subscription! I really should have followed my own advice and leaned more heavily into subagents with cheaper models. Here's what claude.ai/settings/usage is showing me right now: I have several other major Fable-driven projects on the go right now as well, with the goal of hitting 100% on that Fable bar just in time for the price increase. Here are the full release notes for the RC. I had Fable add these to an "Unreleased" section of the changelog as each change landed, reviewing them as it went. This has the neat side effect that the commit history of the changelog acts as a concise summary of each of the changes that went into the release. In the past I've had a policy of writing release notes by hand, but honestly these are better than I would have created myself. Release notes are a great example of writing that I'm OK to outsource to agents because they need to be boring, predictable and accurate. Breaking changes: Everything else: You are only seeing the long-form articles from my blog. Subscribe to /atom/everything/ to get all of my posts, or take a look at my other subscription options . You want to group several write operations together, so they either all succeed or all fail - use db.atomic() . You are managing a transaction yourself with , in which case nothing is committed until you commit - the library will never commit a transaction you opened. [P1] sqlite_utils/db.py:663 now rejects non-row statements only after calling , and sqlite_utils/db.py:705 auto-commits those writes first. So raises but the update is already committed. That is a surprising side effect for a method documented as “can only be used with SQL that returns rows.” [P1] sqlite_utils/db.py:672 through only commits after the returned generator is fully exhausted. without iteration, or common usage, leaves the transaction open and the write can be rolled back on close. This contradicts docs/changelog.rst:15 and docs/python-api.rst:232 , which say it takes effect without iteration. Write statements executed with are now committed automatically, unless a transaction is already open in which case they join it. Previously they opened an implicit transaction that stayed open until something committed it - writes appeared to work when read on the same connection but were silently rolled back when the connection closed. Code that relied on rolling back uncommitted writes should use the new method to open an explicit transaction first. The transaction model is documented in full at Transactions and saving your changes . now executes its SQL as soon as it is called, rather than waiting until the returned generator is first iterated. Rows are still fetched lazily during iteration. SQL errors are now raised at the call site, statements such as are executed and committed immediately without needing to iterate over their results, and passing a statement that returns no rows - previously a silent no-op - now raises a recommending instead. A statement rejected this way is rolled back before the error is raised, so it has no effect on the database. Python API validation errors now raise instead of . Previously invalid arguments - such as with no columns, on a table that does not exist, or passing both and - were rejected using bare statements, which are silently skipped when Python runs with the flag. Code that caught for these cases should catch instead. and now raise if a record is missing a value for any primary key column, or has a value of for one. Previously such records - which can never match an existing row - were quietly inserted as brand new rows, or triggered a confusing after the insert had already taken place. and now raise a if called while a transaction is open. Previously they would silently commit the open transaction as a side effect of changing the journal mode, breaking the rollback guarantee of and of user-managed transactions. The class no longer has an method. It existed only to raise , since full-text search is not supported for views - calling it now raises instead, and the method no longer appears in the API reference. The command shows a clean error when pointed at a view. The no-op flag has been removed from the and commands. Type detection has been the default for CSV/TSV data since 4.0a1, so the flag did nothing - invocations using it should simply drop it. remains available to disable detection. now raises a if passed a connection created with the Python 3.12+ or options. and behave differently on those connections, which previously caused every write made by the library to be silently discarded when the connection closed. Fixed a bug where , and did not commit their changes, leaving the connection inside an open transaction. Their work - and any subsequent writes - could then be silently rolled back when the connection was closed. All three now use , consistent with the other write methods. The command now refuses to drop a view, and refuses to drop a table. Previously each would silently drop the wrong type of object if the name matched. Both now exit with an error suggesting the correct command to use. Migrations applied by the new migrations system now run inside a transaction, together with the record of the migration having been applied. If a migration raises an exception its changes are rolled back and it stays pending, so it can be safely re-applied after the error is fixed. Migrations that cannot run inside a transaction, such as those executing , can opt out using - see Migrations and transactions . and now detect the primary key or compound primary key of an existing table, so the argument is no longer required when upserting into a table that already has a primary key. can now be used to insert a row consisting entirely of default values into an existing table, using . ( #759 ) Improvements to the command: values that do not match any known migration are now an error instead of being silently ignored, now works correctly with migration files that still use the older class, and is now a read-only operation that no longer creates the database file or the migrations tracking table. now returns migrations in the order they were applied. New , and methods for taking manual control of transactions, as an alternative to the context manager. New documentation: Transactions and saving your changes describes how transactions work and when changes are committed, and a new Upgrading page details the changes needed to move between major versions.

0 views
Aran Wilkinson 1 weeks ago

Owning the Harness

Over the past year I've started noticing something in the conversations I have with others when talking about AI usage at work. Ask someone how their company is handling AI tooling and you'll almost always get one of two answers. Either they've gone all in. Every engineer gets a Copilot or Cursor or Claude licence, token budgets don't exist, and leadership has basically said "spend what it takes, we'll figure out the ROI later." Or they're in the other camp: tight budgets, a handful of engineers with access, everything scoped to a pilot project with measurable outcomes before anyone else gets a look in. There isn't much middle ground. Companies pick a lane and commit to it, and that choice says more about their culture than any AI strategy document ever could. But the longer I watch this play out, the more I think both camps are arguing about the wrong number. One side doesn't track cost at all and the other tracks it obsessively. Yet the thing that actually decides the bill, and whether any of this is sustainable, is one most of them never touch: the harness their models run through, and who controls it. That's what I want to get into here. The all-in companies aren't subtle about it. Every engineer gets a Copilot, Cursor, Claude or ChatGPT Enterprise seat as part of onboarding. There's no token budget to track, no approval chain to navigate. The philosophy is straightforward: AI is the biggest productivity shift since the internet, and the companies that embed it deepest and fastest will pull ahead. Worrying about per-seat costs right now is like worrying about electricity bills in 1890. In practice this looks like engineers using AI for everything. PR reviews get run through a model before a human sees them. Architecture discussions start with an AI-generated proposal that the team then critiques. Onboarding documentation gets drafted by pasting the codebase into a chat window and asking for a summary. Some teams are running AI agents that pick up tickets, open pull requests, and write their own tests, with varying degrees of success. The cultural shift is the part that interests me most. When everyone has unlimited access, the conversation changes from "should we use AI for this?" to "why wouldn't we?" That's a genuine acceleration. Junior engineers who might have spent a week figuring out a Kafka consumer are shipping in a day because they've got a model walking them through it step by step. Senior engineers are spending less time on boilerplate and more time on the hard problems they actually enjoy. The downside, of course, is that unlimited access doesn't come with unlimited judgement. I've written before about what happens when you run AI-generated SQL without understanding it , and that problem scales with the size of the organisation. When everyone's moving fast and the AI is confidently wrong about something subtle, the blast radius gets bigger. The other camp is just as deliberate, but their starting assumption is different. They see AI as a tool that needs to prove its value before it gets rolled out broadly. So they scope a pilot: maybe five engineers on a single project, maybe a specific workflow like test generation or documentation. Someone owns the budget spreadsheet. Someone else is tracking which prompts produced useful output and which ones burned tokens on nothing. The philosophy here is that cost optimisation is a first-class concern, not something you figure out after the invoices land. These companies want to know what they're getting for their money before they commit to a hundred seats. They're not anti-AI (most of them are genuinely interested), but they're treating it like any other tooling investment rather than a cultural transformation. What this looks like in practice is more restrained. A small team experiments, reports back, and leadership decides whether to expand. Token budgets are real and sometimes tight enough that engineers think twice before asking the model to rephrase a function they wrote six months ago. Project selection is careful: you pick something where the AI's contribution can actually be measured, so you have data to justify the next round of spending. I understand the logic. If you're running a team where every pound matters, you don't hand out AI subscriptions like stickers at a conference. But there's a cost to caution too, and I think it shows up in ways that don't appear on the budget spreadsheet. Here's the part both camps tend to underestimate: a lot of the cost was never theirs to control in the first place. The two biggest levers (which model you run and which harness you run it through) sit largely with the providers, not with you. The model is the obvious one. Running everything through the frontier model versus a cheaper, smaller one can be an order of magnitude difference on the invoice, for output that's often good enough either way. But the harness matters just as much and gets talked about far less. The same task, on the same model, can burn wildly different amounts of tokens depending on the tool wrapping it. A harness that re-reads the entire codebase on every turn, or pads each request with a bloated system prompt, will quietly cost you several times what a leaner setup would for an identical result. You can watch this happen in the tools you already use. Something like Claude Code doesn't send a fixed system prompt. It assembles one on the fly for every request, pulling in your project instructions, the definition of every tool it can call, the skills you've enabled, the MCP servers you've connected, and a pile of environment context on top. Add more skills, connect more servers, write more project config, and that prompt grows, and it gets sent again on every turn of the conversation. Under per-token pricing that's a standing cost most people never see, because the harness assembles it for you and never shows you the system prompt it built. Caching softens the repeated static part, but it doesn't make it free, and the parts that change from request to request aren't cached at all. That has an awkward implication for the cautious camp. You can count seats and cap budgets all you like, but if the real spend is being driven by model choice and harness efficiency, you're optimising the wrong variable. And it's just as awkward for the all-in camp: you're building on pricing and tooling decisions the provider can change under you at any time. A model gets more expensive, a harness gets chattier in an update, and your costs move without you having touched a thing. This is getting harder to ignore as the billing model shifts. The industry is quietly moving from flat-fee subscriptions to consumption-based pricing, and the early numbers are startling. In a June 2026 report , Gartner found that nearly a quarter of technology leaders are already spending between $200 and $500 per developer each month on tokens, with around 6% over $2,000. It also projects that by 2028 AI coding costs will overtake the average developer's salary. That headline deserves a caveat: Gartner's "average" is a global one, pegged to roughly $2,000 a month, not a senior Western salary. But the direction of travel is the point. When a harness can quietly burn 50,000 tokens on a single test-suite run, your spend is set by tooling decisions the provider can change under you, not by how many seats you bought. I've watched this play out up close. When the bill lands on the company card rather than your own, spending thousands of dollars a month on tokens stops feeling like spending at all. I've seen engineers burn through more in a month than their whole tooling budget used to be for a year and not think twice about it, because it simply isn't their money on the line. That's fine while the models hold their price, but they don't. Each new frontier model tends to arrive more capable and more expensive than the last, often close to double, and usage only ever climbs. A way of working that already looks careless at today's prices doesn't become sustainable when the underlying cost doubles. It becomes less. Neither approach is cost-free. The all-in camp gets speed and cultural momentum, but they're burning through budget on something whose ROI is genuinely hard to measure, and they're building a dependency on tools that might change pricing or disappear tomorrow. The cautious camp has control and cost visibility, but they risk falling behind competitors who are iterating faster, and they're potentially demoralising engineers who see peers elsewhere shipping with better tools. The risk I think about most with the cautious approach isn't the money they're saving, it's the talent cost. Good engineers know what tooling is available elsewhere. If they're stuck on a team that's still "evaluating" AI while their friends at other companies are shipping with it daily, that's a retention problem with a price tag that doesn't show up on the AI budget line. I don't think this has to be binary. The answer that makes the most sense to me is structured experimentation: give engineers access, but with guardrails that create visibility without creating friction. Let people use the tools, but measure what's actually working and redirect effort towards the patterns that produce results. That's closer to how I work personally. I use AI heavily, but never as a black box. I write a PRD first, break the work into small, reviewable tasks, and treat the AI's output as a draft that I'm responsible for, not a solution I'm rubber-stamping. The model generates, I review. That separation, with generation and judgement living in different hands, is the part that keeps me from repeating the mistakes I made when I lost that database. It's also why I've spent time building my own tooling and harnesses rather than living entirely inside off-the-shelf ones. Earlier I said the harness is one of the levers the provider controls. Building your own is how you take some of it back. When you own the loop, you decide how much context gets sent on each turn, which tools and MCP servers the model can actually reach, and where a task should stop and hand back to you. That control shows up directly on the invoice: the same work, on the same model, costs a fraction of what it does through a harness that reloads the world on every request. It's more effort up front, but it turns token spend from something that happens to you into something you decide. Owning the loop also means you stop paying frontier prices for work that doesn't need them. Not every step of a task wants the same model. The expensive, heavy-thinking models earn their keep on high-level work such as planning an approach, breaking a problem into reviewable steps, and weighing up an architecture. But once the plan exists, most of the implementation is narrow, well-specified work that a cheaper, faster model handles perfectly well. Route the thinking to the expensive model and the grunt work to the cheap one, and the bill drops again with no real hit to the output. That kind of routing is hard to pull off inside an off-the-shelf tool that runs everything through whichever single model it defaulted to. You can see the payoff in the wild. Mitchell Hashimoto recently described running exactly this kind of split, using one model as a planner and architect, a different one as the coder, and then the first model again as a judge to check the work. The numbers are the striking part. At API pricing he put the planning and judging steps in the region of a few dollars, against the $50 or more that a single full round trip through one frontier model would typically cost. It's an early experiment, and he's the first to say the longevity isn't proven, but the shape of the saving is hard to argue with. Same work, broken across the right models at the right price points, for a fraction of the bill. I'm not the only one going down this road. There's a small but growing group of engineers doing the same, a lot of them building on Pi , a deliberately minimal coding-agent harness you're meant to reshape around your own workflow rather than bend yourself to fit. That framing is the whole point. People are shaping harnesses to match the way they actually work, and tuning them to get the best out of the specific models they're driving. That's exactly the control the off-the-shelf tools don't hand you. There's enough here for its own article, which I'll write separately. For now the point is simply that owning the harness isn't hypothetical. People are already doing it, and it changes both what the tools cost and how well they fit. The approach I landed on isn't complicated: write a plan, break it into tasks, review every piece of AI output before it touches anything real. The structure matters more than the specific tool or model you're using. Without it, you're just hoping the AI doesn't lead you off a cliff. I don't see why that pattern couldn't scale to a team or a company. Give people access. Expect them to use it. But also expect them to own what they ship, to understand the code they're committing, and to stay curious enough to catch the model when it sounds certain and isn't. The guardrail isn't a token budget. It's a culture of reviewing output before trusting it. I don't think either extreme gets it right. The all-in approach risks the kind of blind trust that cost me a database. The cautious approach risks paralysis dressed up as prudence, and in an industry that moves as fast as ours, that's its own kind of expensive. The answer is almost certainly somewhere in between. Give people access to AI tools. They're genuinely useful and they're not going away. But teach them to use those tools with judgement. Create a culture where questioning the AI's output isn't seen as a lack of skill but as a basic professional reflex, the same way you'd review a colleague's pull request even if they were the best engineer on the team. What matters isn't how much you spend or how many tokens you burn. It's whether that spend is something you control or something that just happens to you. The teams that come out of this well won't be the ones with the biggest budgets, or the strictest ones. They'll be the ones who own the harness their models run through, route the right model at the right cost to the right work, and keep a human in the loop whose judgement can tell a right answer from one that only sounds right. Spend you can't see or steer is the real risk. Spend you own is just a tool doing its job. That ownership, not the size of the invoice, is what decides whether AI makes your team better or just makes them faster at being wrong.

0 views
matduggan.com 2 weeks ago

Clickhouse is winning the Observability Wars

For roughly the last ten years, a meaningful percentage of my working hours have been spent thinking about observability. If you're not familiar with the term, "observability" is what we call it now that "monitoring" doesn't sound expensive enough. The actual work is unglamorous in that you collect a lot of logs, some metrics, a few traces, and then you give them to people. I generally like my job. I like that we're always trying new ideas and approaches. I like the fact that when things go wrong, the answer is almost always sitting there in the data, waiting to be found by whoever is patient enough to look. But I want to be honest with you: in ten years of doing this work, across a half-dozen companies and every observability platform you've heard of and a few you probably haven't, logs have never stopped being the worst part of the job. They were the worst part when I started. They are the worst part today. I fully expect them to be the worst part of this job forever until the robots rise up and rip my head off in one clean sweep. I've written about why logs are terrible before , so I'll spare you the full lecture and give you the short version. Every developer's expectations for logs are set by a single formative experience: the syslog box. Or a container running locally. Or tail -f on a production server they probably shouldn't have SSH'd into. The point is that at some early, tender moment in their career, they had an experience with logs that was flawless. They ran and something useful came back. They piped it into jq and got exactly what they needed. This experience is the observability equivalent of a first kiss. It ruins them for everything that comes after. Because here is the thing about that flawless experience: it works because the system is small, the volume is trivial, and the person querying is the same person who wrote the log line. There is no schema drift, no cardinality explosion, no cross-team consumer with dashboard expectations, no VP asking why the "revenue events" graph has a gap in it. Then there are forty services. Now there are four hundred. Now the logs are being consumed not just by developers but by customer service, who need to look up a specific user's failed checkout from Tuesday. And by the data team, who are quietly building a business-critical dashboard on top of a log line that a backend engineer is about to refactor without telling anyone. And by the on-call, who at 3 AM does not want to learn a new query language, does not want to think about index patterns, and would like the search bar to just work. So you have a technical problem — the volume is enormous, the shape is inconsistent, the queries are unpredictable — sitting on top of an expectations problem, which is worse. Developers want logs instantly, they want to run arbitrary operations on them, and they will not commit to a schema. Meanwhile the less-technical consumers of that same data want the dashboards to be stable forever, the UI to be forgiving, and the whole thing to feel like a normal product. These two audiences are, in most practical respects, at war with each other, and you are the diplomat. ClickHouse came out of Yandex, where it was built to chew through analytical queries against absurd volumes of clickstream data. It was not designed for observability. It just happens to be shockingly good at it, because clickstream data and observability data have a lot in common: high volume, append-heavy, time-ordered, mostly read in aggregate, and every so often you need to reach in and find one specific needle. You can run it yourself with Helm charts. You can point Grafana at it via the ClickHouse plugin, or use their own web UI, or bring your own frontend. Their docs are actually good, which I mention because it's rare enough to be worth flagging. I've never used their ClickStack setup though, so YMMV. For observability specifically, the OpenTelemetry Collector has a ClickHouse exporter, which means you can pipe OTLP data straight in and let it manage the initial schema for you. ClickHouse is designed to scan billions of rows and ingest an amount of data that, when you first see the numbers, makes you assume they're lying. They're not lying. You query it with SQL, which is a language that already exists and was not created by a startup two weeks ago. I'm ranting about logs and then I'm explaining why I like to administer Clickhouse more. Let me take a second and explain why Clickhouse is really good at logs at scale. Logs, as a data shape, have some peculiar properties. They're append-only. You never update a log line, and you almost never delete a single one, though you delete a lot of them at once when retention kicks in. They arrive roughly in time order, though never actually in order. They're read in bursts where nobody looks at logs for days, and then during an incident somebody wants to scan a billion of them in seconds. They're highly compressible, because most of the bytes in your logs are repeated: the same service names, the same hostnames, the same error strings, the same JSON keys, over and over and over again. And critically, when you query them, you almost always want either a narrow time range across all fields or an aggregation across a wide time range with a few filters. You very rarely want "give me one specific row by ID" the way you would from a transactional database. (There are exceptions when its something like GDPR or compliance logging which is its own subgenre of nightmares). In a row-oriented database — Elasticsearch, Postgres, MySQL — the data for a single log line is stored together on disk. If your log has 40 fields and your query only cares about 3 of them, tough luck, you're reading all 40 from disk anyway. The database will filter it in memory, but the disk I/O has already happened. ClickHouse stores each column separately. If your query says SELECT service, status_code, count() FROM logs WHERE timestamp > now() - INTERVAL 1 HOUR GROUP BY service, status_code, ClickHouse reads exactly three columns off disk: timestamp, service, and status_code. The other 37 columns in your schema might as well not exist. On observability data, where you often have dozens of attributes but any given query touches three or four, this is the difference between scanning 800GB and scanning 40GB. This is also why the compression numbers look absurd. Columnar data compresses far better than row-oriented data because the values within a single column are, by nature, similar to each other. A column of service_name values might have a hundred distinct strings across a billion rows. ZSTD eats that for breakfast. You'll routinely see 10–14x compression ratios on real observability data, compared to 2–3x for Elasticsearch. The amazing thing is that ClickHouse scales without changing shape. I don't know how else to say this. Every other observability backend I've worked with mutates as it grows. The architecture at 1 TB a day and the architecture at 10 TB a day are recognizably different systems, with different failure modes, different ops burdens, and different mental models. ClickHouse at 10 TB a day looks like ClickHouse at 1 TB a day with more shards. That's it. That's the pitch. That's the whole reason I'm writing this. Let me show you what I mean. At 1 TB a day, every modern observability stack is roughly okay. If you're at this scale, you can pick almost anything and be productive. The differences below are real but they're not yet painful. Here is the honest truth: at 1 TB a day, ClickHouse is not less complicated than its peers. It's roughly the same. Maybe slightly more, if you count the schema design work you have to do up front. You get 10–14x compression with ZSTD and proper codecs, the Altinity Operator handles keeper coordination and the whole thing runs in about seven pods. But you do have to design your schemas. ORDER BY keys matter enormously. There is no native PromQL, so metrics workflows go through the Grafana plugin or through chproxy and an adapter. Roughly $1.5–2.5K/month. If you took the diagrams at this tier and squinted, you'd say they're all in the same weight class. And you'd be right. Now watch what happens next. This is where the exponential curve kicks in for everybody except one of these. You'll notice, if you look at the diagram, that I basically just added shards. That's it. That's the change. Same operator, same query engine, same query language, same mental model. Rebalancing after adding shards is manual, which is a real trade-off — most teams pre-provision or use weighting on Distributed tables to sidestep it. Materialized views for dashboard rollups shift from "nice to have" to "essential." Roughly $7–11K/month. The gap between ClickHouse and everything else opens up here. It doesn't close. This is where most solutions genuinely stop working, in the sense that even a well-staffed internal team cannot keep up with the operational load. If you've read this far, the point is probably already obvious, but I want to say it directly. Every observability stack works at 1 TB a day. If you're small, pick whatever your team already knows. Life is short. We're all just waiting for the robots to kick our heads off like soccer balls. The question is not which stack works today. The question is which stack still resembles itself two years from now, when your data volume has 5x'd and your team has 2x'd and the person who originally designed the whole thing has left the company. Elasticsearch mutates. LGTM mutates. Datadog stays operationally simple but mutates financially into something that requires its own dedicated team of accountants and pipeline engineers just to keep the bill from spiraling. ClickHouse just gets wider. You add shards. That's the whole trick. There is a real cost to this: you have to eat the schema-design and query-engine complexity up front, at a scale where the other options are objectively easier. You will be, briefly, the one making things harder for your developers. They will not always appreciate this. But the trade you're making is that their experience — and yours — remains roughly the same as the data grows by an order of magnitude, and the next order of magnitude, and probably the one after that. I have spent ten years watching observability stacks change shape underneath me while I tried to keep them running. ClickHouse is the first one that hasn't and that has been able to actually scale with me . That's pretty incredible. A relatively vanilla Elasticsearch cluster with Logstash providing some buffer between ingest and the Lucene indexes. Users get full-text search, which is genuinely good — this is the thing Elasticsearch is actually best at, and at this scale it delivers. Mapping explosions are already a background risk with mixed data, so dynamic mapping needs to be disabled or carefully templated from day one. ILM policies (hot → warm → delete) are non-optional even at this size, because forgetting to set them is how you get paged on a Saturday about disk pressure. Roughly $6–9K/month. Nothing too crazy. Alloy (formerly Grafana Agent, RIP) unifies the collection story into a single daemon, which is nice. Loki works well as long as you spend some time educating developers on how to attach useful labels — a conversation you will have many times, with many people, for the rest of your career. Mimir and Tempo largely do what it says on the tin. Roughly $3.5–5K/month. At 1 TB a day, Datadog is genuinely great. This is the scale it was built for, and it shows. You install the agent, you look at dashboards, you go home. There is almost nothing to think about, which is the entire point. You can already see the shape of the cost problem lurking in the diagram — the metered pipelines, the indexed-vs-ingested logs distinction, the custom metrics cardinality tax — but at this scale it's manageable. Roughly $45–75K/month, though negotiated pricing varies enough that I'd take that number with a grain of salt the size of a fist. Datadog's whole pricing philosophy is that they save you a full-time engineer. I think that framing is somewhat deranged, but they are extremely rich and I am not, so consider your source. Kafka is no longer optional. At 5 TB a day, direct writes into Elasticsearch cause bulk-reject storms and backpressure that will absolutely take your cluster down during a traffic spike. So now you're running Kafka, which means you're either running Kafka well or you're about to have a second, entirely different set of problems. Shard math becomes critical — at 50GB target shards, you're minting ~200 shards a day counting replicas, and your cluster state size becomes its own concern. You almost certainly need Elastic's commercial license for searchable snapshots and the frozen tier. Roughly $40–55K/month before licensing. That but Kafka You are now in microservices mode, whether you wanted to be or not. That means 65+ pods across three separate systems, each with its own compaction pipeline, its own hash ring, its own memcached tier. The gossip/memberlist ring becomes a real operational concern; ingester rollouts require careful -ingester.autoforget-unhealthy tuning, and if you get it wrong you either lose data or duplicate it. Roughly $22–32K/month. The operational complexity is still low, in that you don't run any servers. But you now need a full pipeline team whose entire job is reducing your Datadog bill. Exclusion filters, sampling rules, cardinality caps, tag allow-lists, the whole apparatus. This is what I call the "you build a system to avoid using the system you're paying for" trap, and once you're in it, you are in it forever. Roughly $180–350K/month, depending on how aggressive the pipeline team gets. This is also where you are basically fighting with your SaaS provider all the time, pouring over their billing documentation to figure out how to reduce costs. It's a hostile relationship and one I don't enjoy. You are now running three separate Elasticsearch clusters — one for logs, one for metrics, one for APM — federated through Cross-Cluster Search. Hot-tier NVMe cost dominates the bill. This is the scale at which teams start seriously evaluating alternatives, and where a lot of the recent migrations to ClickHouse have originated. Roughly $95–140K/month plus commercial licensing. You need people who are legitimate experts on Elasticsearch. Now thankfully Elastic just laid a ton of those people off, so they're probably possible to get, but still. Running this thing at this size is very complicated . Around 180+ pods, zone-aware everything, split-and-merge compaction, per-tenant limits, shuffle sharding to prevent noisy neighbors. You almost certainly have a dedicated observability platform team of three to five engineers at this point. If you don't, get ready for a bad fucking time. Roughly $55–85K/month. Still very easy to run, in the strict sense that you don't run anything. But your bill is now measured in six or seven figures a month, and the org has almost certainly built a pre-processing pipeline team whose entire existence is dedicated to reducing that bill. Most companies at this scale have gone hybrid: Datadog for APM and high-value metrics, self-hosted (increasingly ClickHouse) for logs. The complexity paradox at this scale is that you now have Datadog's simplicity plus your pipeline complexity plus a second self-hosted stack. Pricing is all over the goddamn place. You might be over a $1 million a month here. Look at the diagram and then look back at the 1 TB diagram. It's the same diagram. There are more shards. That's the difference. Materialized views for rollups are now mandatory rather than optional. Schema design mistakes you made two years ago will start to hurt, so hopefully you didn't make many. Rebalancing after adding shards is still manual; most teams pre-provision or use clickhouse-copier or a dual-write migration when they need to grow the cluster. Kafka starts to become useful as a buffer for very bursty ingest, though it's not required. Roughly $18–28K/month.

0 views
Simon Willison 3 weeks ago

sqlite-utils 4.0rc1 adds migrations and nested transactions

sqlite-utils is my combined Python library and CLI tool for working with SQLite databases. It provides an extensive set of higher-level operations on top of Python's default sqlite3 package , including support for complex table transformations , automatic table creation from JSON data and a whole lot more. I released sqlite-utils 4.0rc1 , the first release candidate for sqlite-utils v4. The major version bump indicates some (minor) backwards incompatible changes, so I'm interested in having people try this out before I commit to a stable release. There are two significant new features in this RC compared to the previous 4.0 alphas. The first is support for database migrations . This isn't a completely new implementation - it's a slightly modified port of the sqlite-migrate package I released a few years ago. I think that package has proved itself over time, so I'm now ready to bundle it with directly. Here's what a set of migrations in a file looks like: This defines a set of two migrations, one creating the table and another adding a column to it. You can then run those migrations either using Python: Or with the command-line command: The system is deliberately small: it doesn't provide reverse migrations, so any mistakes you make should be fixed by deploying a fresh migration to undo them. Its predecessor has been used by LLM and various other projects for several years, so I'm confident that the design is stable and works well. The new migrations feature is documented here . This feature is a lot less exercised than migrations, so it deserves more attention from testers. Previously, mostly left transaction management up to its users, via a construct that reused the mechanism directly. SQLite supports nested transactions in the form of savepoints, so I wanted an abstraction that could make those as easy to use as possible. I borrowed the terminology "atomic" from Django and Peewee. Here's what the new API looks like: More details in the documentation . The backwards incompatible changes in v4 were described in the alpha release notes. For 4.0a0 : And for 4.0a1 : You can install the new RC like this: Or try the CLI version directly with like this: Come chat with us about it in the sqlite-utils Discord channel , or file any bugs in GitHub Issues . You are only seeing the long-form articles from my blog. Subscribe to /atom/everything/ to get all of my posts, or take a look at my other subscription options . Upsert operations now use SQLite's syntax on all SQLite versions later than 3.23.1. This is a very slight breaking change for apps that depend on the previous followed by behavior. ( #652 ) Python library users can opt-in to the previous implementation by passing to the constructor, see Alternative upserts using INSERT OR IGNORE . Dropped support for Python 3.8, added support for Python 3.13. ( #646 ) is now provided by the sqlite-utils-tui plugin. ( #648 ) Test suite now also runs against SQLite 3.23.1, the last version (from 2018-04-10) before the new syntax was added. ( #654 ) Breaking change : The method now only works with tables. To access a SQL view use instead. ( #657 ) The and methods can now accept an iterator of lists or tuples as an alternative to dictionaries. The first item should be a list/tuple of column names. See Inserting data from a list or tuple iterator for details. ( #672 ) Breaking change : The default floating point column type has been changed from to , which is the correct SQLite type for floating point values. This affects auto-detected columns when inserting data. ( #645 ) Now uses in place of for packaging. ( #675 ) Tables in the Python API now do a much better job of remembering the primary key and other schema details from when they were first created. ( #655 ) Breaking change : The and mechanisms no longer skip values that evaluate to . Previously the option was needed, this has been removed. ( #542 ) Breaking change : Tables created by this library now wrap table and column names in in the schema. Previously they would use . ( #677 ) The CLI argument now accepts a path to a Python file in addition to accepting a string full of Python code. It can also now be specified multiple times. ( #659 ) Breaking change: Type detection is now the default behavior for the and CLI commands when importing CSV or TSV data. Previously all columns were treated as unless the flag was passed. Use the new flag to restore the old behavior. The environment variable has been removed. ( #679 )

0 views
Lalit Maganti 4 weeks ago

syntaqlite 0.6: SQLite dot commands and pyodide

Since my original launch post for syntaqlite, I’ve been quietly working away on it in the background. A lot of the work has been fixing correctness bugs which I discovered as I integrated it into production as the parser for PerfettoSQL in the Perfetto trace processor: as I wrote previously, this has been my dream for over 8 years so it’s amazing to see it finally realized. Just today, I released syntaqlite 0.6 , the most interesting release since the original launch, so I wanted to talk about what’s new. The biggest step forward for real world applicability is that we now support SQLite dot commands : SQLite scripts are very common in the wild and in the past we would simply error on dot commands like and , causing spurious warnings and an inability to format files like this properly. Now, these lines will be silently ignored while still parsing, formatting and validating the SQL inside!

0 views
Hugo 1 months ago

The Transactional Outbox Pattern with PostgreSQL and RabbitMQ

How do you write to two different systems, for example RabbitMQ and PostgreSQL? You might say it's pretty straightforward. But what happens if there's an error, if the SQL transaction fails? You end up with a message published but for an operation that never actually took place. This is a relatively common problem and we're used to handling it with 2 Phase commit transactions. The idea being to introduce a transaction across all operations to external systems, including RabbitMQ here. It's a somewhat complex mechanism that requires an additional coordinator to ensure everything goes well before validating the write across all systems. But there are several problems. The first is that it's not possible to do 2PC transactions with RabbitMQ (also called XA transactions). Which, you'll agree, is already a relatively significant problem in itself. We could use ActiveMQ which supports 2PC transactions. But on the other hand, it would be a shame to write an article about PostgreSQL and RabbitMQ just to conclude that you need to use ActiveMQ instead, right? The second problem is that in any case, a 2PC transaction penalizes the overall system performance. To synchronize two or more systems, not only will the total operation time never be less than the minimal time of the slowest system, but you also add a cost related to coordination. The last problem is that you degrade overall availability. Availability being the multiplication of the average availability of each system. For example with two systems with 99% availability, the total system achieves 98% availability. $0.99 \times 0.99 = 0.9801$ And that was basically the state of my knowledge until yesterday. I had already used XA transactions, I had already dealt with distributed systems issues and I've worked on plenty of mitigation scenarios for this type of problem. Yesterday I added RabbitMQ to Writizzy's stack and I used another solution: the Transactional Outbox pattern. While the underlying concept isn't necessarily new, this specific pattern was popularized by Chris Richardson between 2014 and 2016. The concept is "simple": The code becomes We now have only a single transaction , and the guarantee that the event is published only if the transaction succeeds . Event that will be sent to RabbitMQ later. Now, to actually send to RabbitMQ, you need a job to read the table: Note the use of SchedulerLock here which allows the use of schedulers in a multi-node context, to avoid simultaneous reading by multiple applications. This allows replacing Quartz which I frequently used until now. And you need the publication service code Note that we've enabled RabbitMQ's publisher confirms mode and we're waiting for write confirmation: But you could say to me: "you have the problem of double write to two systems again". And... you're not wrong. But it's simpler here. I only have 3 cases: The 3rd case is important, it will happen so you need to handle it. This implies that each consumer must be able to be idempotent on message reception. That is, each consumer must be able to accept the same event twice without incident. Here strategies are multiple: But there's an issue, if RabbitMQ goes down, if it's unreachable, if the disk is full, we're going to log the exception and with a batch running every 500ms I'll let you imagine the astronomical amount of logs that will produce. So we need a Circuit Breaker . Conveniently, we can use resilience4j . With a circuit breaker, we'll cut off sending to Rabbit in case of error and wait a bit before retrying. I'll leave the code here but won't detail it, that would be the subject of another article. As I said earlier, I was well acquainted with the XA transaction mechanism and the state of my knowledge for solving this problem stopped there. I wondered with curiosity if Claude would propose a more elegant implementation and I was rather surprised. In this specific kind of case, it was the perfect opportunity to try learning with AI instead of just suffering code we don't understand. Letting AI write code without oversight, unsurprisingly it's rarely good. You need an expert eye and ultimately the code produced remains our responsibility and we must be able to understand it. So you need to find a middle ground between letting it do its thing and micro-managing the AI. By giving it my constraints (stemming from my experience), it was the agent that came up with the Transactional Outbox pattern proposal. And while I was initially a bit skeptical, I tried to understand each part of the code to make it my own by asking multiple questions. The code wasn't perfect, it was following these exchanges that was added: So yes, it wasn't a smooth plan but it was a good way for me to force myself to get updated, do some research and in short, learn new things. Beyond the technical aspect of this post, I mainly wanted to illustrate the method I use to code with an agent, which allows me to combine productivity AND craft. rely only on the database to publish an event (step 1) read the table in an asynchronous job (step 2) publish to the message queue from the job (step 3) Everything goes well, nothing to say, it's perfect. RabbitMQ fails => we go into the exception, we log, we exit, the message stays in the table and will be retried The PostgreSQL transaction fails. We go into the exception but the message is already sent!! read the message ID and store it somewhere to check that we don't do the operation twice, ideal for calls to external services trigger idempotent operations (Example: set status = pending, even if we do it twice, it doesn't pose a problem) the publish confirm pattern resilience with resilience4j the scheduler lock to avoid multi-node issues the addition of messageid in headers (for deduplication) claude had added a state management on messages that wasn't relevant

0 views
Stratechery 1 months ago

An Interview with Microsoft CEO Satya Nadella About Finding Core Competencies

Listen to this post: Good morning, This week’s Stratechery Interview is with Microsoft CEO Satya Nadella . I have previously interviewed Nadella in May 2024 , October 2022 , April 2020 , and May 2019 . As I noted yesterday , I spoke to Nadella shortly after the conclusion of his keynote at Build , Microsoft’s annual developer conference . One notable thing about the keynote was the fact that Nadella was — outside of product demos — the sole presenter; one gets the sense he has shifted into a much more hands-on role at Microsoft over the last year. The reasons why are clear: my first question to Nadella was if he was happy about where Microsoft was currently positioned as a company. We talk about the reasons for that question, the status of the company’s partnership with OpenAI, and whether Microsoft has invested sufficiently in AI infrastructure. Then we talk about the future of software, Microsoft’s business model in the age of AI, and if they can operate independently from the leading edge models. At the end we talk about Project Solara and whether Microsoft will ever pay residents to build data centers. One note, with regards to a misunderstanding towards the end of the interview: there is no documentation I could find about being able to use Copilot Cowork with non-Anthropic models; Microsoft’s own documentation fits my understanding. As a reminder, all Stratechery content, including interviews, is available as a podcast; click the link at the top of this email to add Stratechery to your podcast player. On to the Interview: This interview is lightly edited for clarity. Satya Nadella, welcome back to Stratechery. SN: It’s great to be with you, Ben. So first off, I don’t know if you realize this, but at least according to my daughter, the defining word for the real grinders in Gen Z — first off, LinkedIn is like the social network. SN: That’s great! Number two, the word they all use is “build”, “I’m building, I’m building”, so who knew when I was at the first Build, I think, in 2010? Or was it 2011? Who knew you were such a trendsetter? SN: (laughing) There you go, I’m thrilled that your daughter is building and is on LinkedIn. Yeah, well, I’m not sure if she’s on there, she’s more making fun of people, so we’ll see how it works. We last talked the summer of 2024 after Build, this was up in Seattle. To say a lot has changed since then is an understatement. I had a bunch of questions I wanted to ask you about the business as a whole, things going on, I’m going to start with those, then I have questions about the presentation at the end. But relative to that, I want to ask you one simple question: Are you happy with Microsoft’s current competitive position? SN: You know, always this is the trickiest thing, you can sit here and say, “I’m happy” — that means you’re not ambitious enough and when you say, “If you’re not competitive, what the heck are you doing?”. And plus you have like 57 different product lines. SN: I’d say the thing in these platform shifts in particular is to, one, get the conceptual model of, “Where is the opportunity for us as a company?” — most people measure competitive position as if it’s a complete zero-sum game, and it’s never been the case. Which is, it is not the case with the cloud, it is not the case in client-server, and so to me, “What is Microsoft uniquely capable of doing in this new world” — that’s the key thing that we have to answer before we even get to the competitive position. In that context, “What is it that we really have a shot at?”, which is we can be a trusted purveyor of a platform, which is what we’ve always done, that allows people to create more value on top of a platform, which is again the DNA we have. Even in a world where these frontier models seem to have no limit— A very large appetite. SN: They have large appetite. That is what I feel even this Build , this conference, we are at that state where we can now really turn this from any one frontier model to saying, “Hey, there is actually a way for a frontier ecosystem to emerge where there are many stakeholders who all actually are operating with their own frontier intelligence”, that is a place where I think we have a unique shot, a unique competitive angle, and most importantly, brand permission. This is the other thing I’ve learned, Ben, which is every company thinks they can do everything, and then they realize that the world doesn’t need them to, the world wants them to do the one thing. Is that a lesson that you had to learn? SN: Yeah, absolutely. I’ve always said this, at Microsoft we are at our best when we do what the world expects us to do, we are at our worst when we do things out of envy, which is just because somebody else had some cool hit, somewhere, doesn’t mean we should go do that. But enough about the Zune, right? SN: (laughing) Yeah, Zune was a great device, but the world didn’t need Zune from us, and so that was the end of it. This identification of your unique capabilities, is that one of the changes over the last two years where that has emerged? SN: Yeah, in fact, it has emerged and also the world’s kind of gotten to it. Has it been forced on you to an extent? SN: Yeah, even my own conceptual understanding, I started by thinking of, “What are models?”, models are kind of like some stateless APIs, then I adjusted and said, “Oh, maybe there’ll be like databases” — they’re really more than that. I don’t remember talking about this with you, but last time I talked to [Microsoft CTO] Kevin [Scott], we analogized it to processors at some point, and you actually did make a comparison in terms of the partnership to your partnership with Intel. SN: Exactly. So the question now is, it’s a better conceptual model to think of what we’re doing is you have to really build a learning machine, and any company has to build a learning machine, so what I want to build is essentially a multi-tenant learning system that allows everybody to have their own hill-climbing machine . So that conceptual idea, now I’ve turned what is essentially frontier is not about any frontier model — I want to build whatever you did with M365 or with Azure into a platform which allows everybody to basically build their own hill-climbing machine right because the future of a firm at a foundational level they’ll have human capital they’ll have token capital and for the token capital they need their own hill-climbing machine. All right, so I’ll jump to the end, you released seven new models, you emphasize the work you’ve done to build these models from scratch, not with distilling, not with using other models as teachers — so did you just articulate what the ambitions are with these models? SN: Yeah, there are two sets of things. One is we wanted to build from ground up with clean lineage, the models that we will have that we can license and allow enterprises to continuously hill-climb, so that’s why we want that model. By the way you talked about distillation — the point is to not use distillation during any of our own hill-climbing but at the very end, in fact some of the things that we are doing is, after all, we have all the OpenAI IP, in fact some of the performance gains we get is by doing RKLD, which is reverse knowledge distillation , and RL on top of it. So we have effectively two frontiers, we have our own, we have the OpenAI, and we’re going to use these things to eval match. And the clock is ticking to get to the right state you need to be while you still have that access . SN: Yeah, and there’s five years of it. But the bottom line is at any given point in time, I want to make sure that I’m using the best, most efficient model for whether it’s in coding, whether it’s in security, making sure also in our case, we’ll have a harness that’s independent of these models, we have the GitHub Copilot harness that’s used everywhere across Microsoft. Our goal is to make sure we have a model lineage, which we control end-to-end, we then use OpenAI IP, even with all of the capability it has — ultimately, the tests are going to be the evals for us and our customers. In the long run, the way it was framed today, and I thought it was very compelling, and it speaks to what you just said, was this idea of enterprises being able to take these models and in their own RL environments incorporate their data at a much deeper level than sort of a slap-on RAG implementation or basic post-training. Is that the end goal, though? SN: Yeah, the end goal for me is the following, which is I go back and say, let’s say that they’re a generalist model — if you go back even, Windows could have a release, then another release, and Adobe and Autodesk could keep building and keep going up, what’s the moral equivalent of that? That is the thing. And then in the first time, we said fine-tuning, it kind of didn’t work because we didn’t have the tools, we didn’t have the data collection regime, none of that. But now we have it. So let’s say the generalist models keep getting better, MAI models, let’s say, or OpenAI models, then you have this RLE. Right, but this deep customization of the models you’re talking about is only possible with MAI models. SN: That’s correct, but the thing that we want to start getting everyone on is this multi-tenant hill-climbing system — so if you think about it, we literally turned your use of M365, which already is a multi-tenant system, into a hill-climbing system for you. Okay, I’m gonna have to stop you, I’m going to give you an ELI5 opportunity, explain hill-climbing to the audience. SN: Hill-climbing is basically when you think about, “What does AI do?” — AI is all about taking an objective and continuously learning how to go predict and create that output that is the representation of that objective, and do so continuously. So that’s why a metaphor of hill-climbing is the best way to describe learning. And you want everybody to do this individually on their own hill. SN: Individually on their own. As opposed to like, hitching along. SN: What is your moat as a company? Your moat as a company is your tacit knowledge. In a world where AI exists, and network effects of AI exist, you need your own hill-climbing machine in which the models are learning. So the first thing we want you to do is, people don’t talk enough about this, but the private outputs, the evals, as I think about as, maybe the most important IP a firm creates are these private benchmarks and the private evals where you are tastefully recognizing what’s the output, the quality. And by the way, today’s failure cases are informing you to change the benchmark continuously, it’s not a static thing, that’s kind of how the evals work. And so if you have your private evals, then you have your own reinforcement learning environment that you’ve created, then you invite all the models to show up, and then you say, “Model A, generate the output that is maxing this eval using my environment and my trajectories and model B…”, and I can switch. In that context, the MAI models is one more lineage that you can put into,c and what we proved today was even a very efficiently trained reasoning model or a coding model can hill-climb using your traces and that will be more token-efficient and it will be fundamentally a great advantage. Exclusive to you the customer. SN: Yeah, that’s right. But is that just for now? If you fast-forward, is your vision that actually MAI models are fully competitive on the frontier with the other general models? SN: They are. Even today, when you start saying that — the world will keep getting better in general.** Well, I guess this goes back to, is this about how you need to do what you’re good at? SN: Correct. One, what we’re good at and also what’s the equilibrium of the world? Which is, if you believe there are only going to be two firms in the world, then of course, they only need two frontier models, but if you fundamentally believe that there are going to be as many firms as there are today and more, then what is the firm in the age of AI? It’s going to have human capital and token capital, how did that token capital get created? It’s not a bunch of API calls, it’s actually some set of weights even they have. Right. And so do you want to accrue that advantage or do you want to give it to OpenAI and Anthropic? Well, speaking of the OpenAI partnership, I mentioned you referred to it like the Microsoft-Intel partnership, and sometimes partnerships are the only way to get ahead. How do you think about that partnership now? SN: I still think that it’s — I’m very proud of the fact that we came together, you remember the circumstances in which we came together were very different and the fact that there is a company now that may go public and be a trillion-dollar company— This is my question — how long were the knockdown, drag out fights between in this corner, there’s Satya Nadella, the operator, and in this corner, there’s Satya Nadella, the investor, tussling over what to do? SN: (laughing) At the end of the day, we are an operating company, investment is just more of an accident. Yeah, but the shareholders are ultimately those investors! SN: I’m glad and it’s a fantastic outcome for our shareholders too and what have you. But I think the way I came at this, Ben, is to say genuinely I’ve always approached it as, if there’s a partner that we can partner with and ourselves innovate, and they’re also successful, that’s fantastic. I always go back to the story of having built SQL Server with SAP. SAP was successful, we were successful, we also then went on to do other things. And so therefore, I think OpenAI, I’m glad we worked with them, we’re working with them, they continue to be a premier partner. As I said, until 2032, we still have a lot as a customer of theirs, them as a customer of ours, as an IP partner. So every day OpenAI does well, Microsoft does well. Is there a bit where everyone thought you were so far ahead because of your partnership with OpenAI, and now when we talk about things like your MAI models, it’s like actually “We got a little bit lulled to sleep because we offloaded too much to them, and now we’re having to recalibrate”? SN: Lots of things, one is, like all things, there’s a lot more competition, there is OpenAI, there is Anthropic, there’s Google, there is tons of folks who are in there. And so I think for us, the beginning, it was great that we got started with OpenAI. Think about where we were in 2018 to where we are in 2026, here we are competing with Google and a bunch of people whose names I wouldn’t have known in 2018, and so that itself proves that to your very first question, “How competitive is Microsoft?” — I’m glad Microsoft took that shot. Here we are competing with a bunch of new people, a bunch of old people, and we have our own game. So we already talked about Satya Nadella, the operator, and Satya Nadella, the investor. What about Satya Nadella, the capital allocator ? There were a lot of reports in about early 2025 about Microsoft pausing and a reconsidering some data center investments, you guys have sort of spun that as, “Lots of speculative stuff”, “We’re streamlining”, etc. — but at the same time, your percentage of free cash flow committed to CapEx lags fairly significantly behind your peers. Four months ago, that was a compliment. Now, is it a diss? How are you feeling about that? SN: The last time I checked, my free cash flow is getting allocated pretty well to capital return that makes sense. Is there a case that you’ve underinvested? SN: Not really. I think the key thing that at least we wanted to make sure is we were not upside down on building — we have a hyperscale busines, we have our own application business, and we have our own research compute to allocate, there are three buckets, we wanted to allocate with great discipline on all three. So take the hyperscale business. Hyperscale businesses are about having a few big customers, but also having a massive long tail, so you can’t have a book of business that is just a few model companies — in fact, one model company — that was the fundamental decision. And you wanted to get out of that business. SN: Not just get out. They’re still there, they’re a major tenant. SN: They’re a major tenant. But, let’s face it, Anthropic over time or OpenAI over time will build their own, it makes sense. They would use — I’m not saying that they won’t use other cloud providers. So to me, it was clear as day that, what I wanted to do was not allocate all my compute only to one player and so that was the adjustment. And once you make that adjustment, you can’t build 10 gigawatts in Texas and say, “That’s it”, you’ve got to build a plant that is spread around the world, around the United States, and that adjustment is what we want to do on hyperscale. The other thing that I have to do is make sure we’re doing also the long-term thing for our investors, which is, “Let’s invest in ourselves”, which is inference compute has exploded, whether it’s in GitHub or whether it’s in M365 and we needed to make sure we fund our own applications. And then our own research compute, these MAI models. So I just took the approach of putting these three, we will definitely want to allocate as we see progress on all this and we’ll see how it all shakes out. But to me, I’m not literally matching quarter-to-quarter. By the way, the other interesting thing is the catch-up, we started early. You were early, and you got a lot of the good spots, a lot of the good power generation. SN: Yeah, and also two years of cash flow. Yeah, for sure. Well, speaking of the balance between the three, in January 2026 , you missed Azure earnings by like 0.1%, so it was very small, and you said on the call , you allocated more compute to internal R&D and applications. Setting aside the earlier question about whether or not you erred by the total amount of capacity, you talked in that call about having a portfolio approach in terms of investment, balancing Azure, and those two other businesses. That’s all well and good, but if there is a constraint, you do have to choose, do you think you made the right choice then? And is that the choice you’ll make going forward? Where you are at the end of the day, you have a higher lifetime value, higher margin on your own businesses, and that’s going to be number one. SN: Yeah, and also research compute. Ben, I think that for all of us, quite frankly, we have to really, at the end of the day, that’s why I think quarterly earnings are interesting, which is, of course, The Street should hold every one of us very accountable for “What did you do for me lately?”. But was that a very particular, annoying, being held accountable for the wrong thing? SN: It’s their job, everyone’s got to do their job, and so I can’t accuse them of them asking, “Hey, what did you do for me this quarter?”, that’s the question they rightfully should ask. And the right answer for me is, “I’ve done enough for you this quarter, and we’re also making sure that 10 quarters from now, Microsoft’s continuing to thrive”, and that’s the job, and sometimes there’s a little bit of disconnect on it. But when I look at the three things, you just have to be disciplined that you’re doing what you can add value, it can’t be, “Oh, I’m misallocated”. To your point, you get punished if you do things where you’re not producing. So that’s why research compute, here is now an MAI model output. Today, it’s just not a model output as an academic thing, that’s now in differentiating our Foundry where we now are able to license it, it’s going to grow Foundry revenue. And so as long as I’ve felt that as long as Microsoft can continue to invest in ways that show results, then we will have the ability to do the right thing in the long run and in the short run deliver results. For the last quarter, was there a bit of, “Let’s give a little bit more compute to Azure?” SN: Last quarter, no. In fact, that one was just a little more of the compute — we are supply-constrained. I know, but that’s what makes it so interesting. SN: We are not at all, like at this point, if anything, the thing that we do not want to do is to disappoint especially our enterprise customers on Azure. That was the question, right? Because if they look at that quarter and they’re like, “Hmm, Microsoft’s saying we’re supply-constrained and also we’re prioritizing our higher margin, higher lifetime value businesses, where does that leave me? I’m competing against my supplier”. SN: That’s one of the reasons why we had to make some very hard choices around, for example, raw GPUs. We’re not selling raw GPUs to a bunch of Neolabs, for example. I wish I could add more Neolabs on Azure, we just cannot. And so therefore, we are being very disciplined on some business that we turn away. Were those some of the conversations you had to have? SN: Yeah, and so to me, in a world where you have constraints, you want to basically make sure you’re building for both what the world expects and the customers who have trusted you in the longest and so we will definitely make sure that Azure has capacity, it’s just that we are not going to go for what I’ll call in this context, “easy money”. Which is, you can always, in today’s day and age, if you want to have short term Azure revenue, it’s pretty easy. Oh yeah, we’ve seen that , to say the least. SN: Yeah, all you gotta do is turn up, you know, and go sell it to a Neolab. So when it comes to AI infrastructure specifically, as you look out in the long run, you mentioned it may very well be rational for the frontier labs to build their own hardware, for example. You have all these Neolabs, you have whatever controls [Nvidia CEO] Jensen [Haung]’s allocation of GPUs, you have different ASICs, what is your true differentiation as a hyperscaler? Is it just lower cost of capital? SN: First of all, think of our hyperscale business as this portfolio, everything from what we are trying to get done is build a system which we have to be competitive in when it comes to tokens-per-dollar-per-watt, that’s one side of it. We can unpack that and what our thesis is there. Well, I just noticed when you were talking about some of your chips, sometimes it was tokens-per-watt, sometimes it was tokens-per-dollar. SN: Yeah, I think of all three, right? It’s like tokens as a function of both power and dollars and so that’s a systems thing that we have to be world class at and be competitive at. And I would be able to claim, and that’s where I think [Microsoft AI CEO] Mustafa [Suleyman] talked about it, like unless and until you build your own model, you can’t, there’s no point. I believe that you don’t want to build accelerators without building a model, you kind of have to co-design. In the long run, the only way to be super efficient on that is to think about, the network is a great example, which is you want the network, the model, all to come together in ways that make sense, so therefore that’s one side. Then the other side for us is the differentiation has to come from, “If I’m building agents on top of this infrastructure, what agents does Microsoft produce?”. I have three domains in which we are going to try and major on: coding, security, and knowledge work. Luckily these are three massive domains where tokens make sense — I’m not saying there won’t be others, science is another one we will enable but I think there will be others who will do great work in there. But to me the three primary domains in which all this is going to be exercised use. So when I think about the portfolio of building a system plus model plus these three domains, then I feel like that’s where our differentiation will come from. But is that just a re-articulation of circling back to, in the long run, our true differentiation is from our higher margin, our own businesses, higher LTV? Where does that leave just customers who— SN: I think it’s not higher margin. The overall margin dollars from our infrastructure business may be higher. In fact, they already are getting close to being higher than our total margin dollars from our high margin businesses. So I think that Microsoft has always benefited from having a portfolio of businesses, and we’ve been comfortable managing through it, where it’s not one margin profile. But in aggregate, we will have high ROIC, and we will make sure that we have an infrastructure business that’s got ROIC that’s commensurate with an infrastructure business, and we have a business that builds on top of it, which I’d like call it like the new apps are agents. So we’ll have agent businesses in security, in coding, in knowledge work, as the three big domains. We’ll get to agents in a little bit, but I didn’t expect to ask this question, big news this week, will you ever issue equity to fund this build out ? SN: Yeah, I just saw the news, I think Google just did it. Were you as surprised as everyone else? SN: I’m not sure, exactly, I’ve not studied it, it came last night, I think, so I’ve got to go understand what’s happening. But, it’s like maybe it’s the thing to do is everybody is going public or reissuing equity, maybe that’s the season. Gobble up some of the money. Is software dead? SN: I think software is alive, but the way I think this entire meme has come about is, like, if you take the SaaS question in particular, right? We built in a particular way where I had a data model, and then I had a business logic tier, and then I had a UI tier, I coupled the three, then had a business model. Integration is a beautiful thing. SN: Look at this, Ben, right now, we took what is the database that no one knows about underneath Microsoft 365 and said, “Oh, WorkIQ is available , it’s just a skill/MCP, and it’s out there”, and suddenly people are falling in love with, “I can now interrogate and have an agent continuously hit this database to reason over and plan over, act over from any place”. By the way, it requires a new business model. So, for example, when Cowork is using WorkIQ, that’s going to be a usage-based business model, so I think what needs to happen is we now need to take what we built, rebuild it for the agent era and change the levers of the business model such that you have a per-user business model and you have a consumption business model. So the hybrid business model, you do think that is going to be the future? SN: 100%. And once you have that then I think what happened between servers — even I had not understood it when we moved to the cloud, even I was a little worried about, “Oh man, we move to the cloud, we’ll sell the same servers”, and it turned out we sold a lot more subscriptions because people who never bought servers from us were buying subscriptions. I think that’s what’s happening already with agents, I see that on GitHub, I see that on M365, I see that on security, because everyone is building these agent systems that are continuously “working” and so what we built and thought of as the end-user compute is completely getting rebuilt. Is there a bit where, if you have to zoom out a hybrid system where a combination of per-seat but also usage, where does E7 fit in this idea, it’s like double the price, it seems it’s an attempt to respond to maybe a secular decrease in seats by increasing ARPU? Is that the right way to think about it? SN: The way you think about this is, see per-seat is a very important element still because what is per-seat? Per-seat is basically a set of usage entitlements, so anyone who is budgeting really will push you. That’s right, people don’t like usage, we’re seeing that right now , it could explode . SN: Exactly, so therefore you just want to take packaging or bundling of usage into proceeds so that there’s some way for people to budget. So I kind of think about the E7, E5, these things will continue and then you’ll always have the outcall consumption. People also talk about, “Hey, maybe people want outcome-based pricing”. Outcome-based pricing, we’ll be thrilled about some of that, but remember, outcome-based pricing is also called royalty. When a customer has a great outcome, they necessarily don’t want to share their outcome so I think what is really being thought about is, ultimately, there is real marginal cost to software, that’s kind of what it is, and that’s going to be priced through. When did that really click for you, the implications of that? SN: I think that I would say agents. Before agents, if it is still human interaction— Right, you can imagine a world where just like basic inference got super cheap and easy. SN: Exactly, the Moore’s Law itself. Like, if you think about it, if I just used Moore’s Law, get software efficiency, I used software for efficiency and drive that home for customers to have more functionality. In fact, I used to always think about, “Hey, how much more value did we add in M365 and not raise price?” — we didn’t raise prices for a decade plus. That’s all thanks to the software efficiencies on top of hardware. But now where you are, and if you have a thousand autonomous agents that are all working continuously 24/7 hitting Work IQ, then that is a lot and so that is where I think, and so the real test for me Ben is, that’s why evals, outcomes — no customer will use consumption or their seats if it’s not creating value for them. Therefore, they now are going to be a lot more disciplined on, “What exactly did this stuff do for me?”, “How do I measure it?”, “How do I get into the efficient?”. And if you think back to going back to the 80s or 90s, where back then it’s like, “Don’t waste time on optimization, the next processor will come out and solve all your problems”, is that now totally the wrong paradigm? SN: In some sense, you want that to happen, but you can’t just count on that. It will happen, but your prices will explode. SN: Exactly, and more importantly, you will be found out if you don’t optimize. Take that example we showed with Land O’Lakes today, which is, here’s an agent, and there is an outcome you care about, I was able to use a model that is using 500B, I was able to use a 5B, and have it really deliver the same outcome, why would I not use that? That does seem to be a very different thing about this period. It seems clear that’s going to be a huge thing in enterprise going forward, using the right model, optimizing, it’s like we didn’t get to the optimization stage of the PC era. SN: That’s right. I don’t think we ever did get there. SN: We never got there. Stuff’s still bloated as ever, because everyone just assumes it’s going to get faster, it’s going to be fine. SN: Exactly, because things were not priced for it. Once you have consumption, everyone will optimize. For E7, it does seem like the real lure there is Cowork . It’s like this new capability, it’s super powerful, it’s taking Anthropic’s Cowork, which is on your PC, now it’s in the cloud, has all the niceties around that, permissions, controls, all those sorts of things. Is that why it’s there? Is that the hook? SN: Yeah, there’s also the Agent 365 , so there’s a whole lot. Like always, these things, we’re going to take everything from what I’ll talk about as what is an end-user thing and an IT thing, bring it all together. You guys know bundling. SN: And security. Yeah, definitely, and they’re all about, ultimately, how do we get the value equation right such that the customer can cover, because right now, it’s kind of fascinating. You have an agent, you immediately say, “Oh, I’ve got to secure it, I’ve got to have observability on it, I need a sandbox for it”. So it’s just that if you don’t bundle, you kind of are sending the customer down the chase of five different things. With that, though, the reason I find that striking is you’ve talked a lot about — to what extent do you think the point of integration that really matters is it does seem to be increasingly between the models and the harness themselves ? You’ve talked about things like your CoreAI initiative and GitHub Copilot, a lot of which is, “We’re going to build the harness and you can slip the models in and out”, and that works right now for Copilot and you can choose your model and even then, from what I’ve heard, not quite as easy as you might think it might be, but it’s still there, the selector’s there. Cowork seems like, “Yeah, that’s right, it has to be the whole package and it’s important for us to have a selling point on E7” — that this feels like maybe it’s not easily substitutable. SN: No, it is. The same thing on Cowork. In fact, right now, the Cowork that I’m using is already mostly defaulted GPT. Okay, so it is going to be fully interchangeable? SN: We’re using the same harness that we use in GitHub and the same thing in security, too. So we have the same harness that’s a multi-model harness in which we will rotate through — obviously MAI by default gets trained in our harness, but we will have GPT, we will have Anthropic in there and any open weight model. We will allow anyone to take any of the models they fine-tune or build. In fact, they can take an open weight model from Fireworks, tune it, put it into Copilot, no problem. All right, so I am misinformed, so I will take the L on that. Explain what is Cowork then and what is the connection with Anthropic as far as that product goes? SN: Cowork, to me, it’s kind of like Copilot. I took the term Cowork, it’s part of there and it’s definitely got the Anthropic models in there. Cowork is — think of it as a form factor, the best way to describe it is we built a chat interface first for Copilot, then we now have built Cowork for Copilot, and now we’re building autopilots, as I described it there, think of it as the enterprise-grade OpenClaws. So basically, I think of these as different form factors of agents — chat was the first thing, Cowork is the next thing and in fact, you can even go back to the developer thing. Developers, how did we start? We started with code completions first, then we went to— I get all this, but I’m genuinely confused here, because I go back to the blog post . It says, “Working closely with Anthropic, we took what they’ve done with Cowork…”. SN: Yeah, that’s what we launched first. All I’m saying is it’s evolved. It’s kind of like, Copilot today. Got it, which started out with ChatGPT. SN: ChatGPT, now it has both Opus and GPT models. Got it, okay. SN: So, they’re going to be all over. All right. So, I wasn’t completely off the reservation. SN: That’s right. I failed to catch up, I will accept that. [ Editor’s Note: the FAQ for Cowork still says it uses Anthropic models, just like the original blog post ] SN: Every product of ours, you’ll have both Anthropic and OpenAI models, and MAI models, and your ability to put your own models, and that, I think, is the fundamental promise. Oh, by the way, I should mention this. The amount of auto — I don’t know how much you’re doing selection, I’m mostly auto — and so then one of the biggest pieces of work at Microsoft is all the training models to do auto-routing. That, by the way, is perhaps one of the biggest continuous learning things.** It’s interesting because I probably approach it more from a consumer perspective, so I just literally choose the app that I want to do something in or call from the CLI. What happened to Github Copilot? You’re talking about it very positively, but I think a negative spin would be two or three years ago, you were first to market with autocomplete, everyone assumed you got there, you won, and now it’s like, “We’re going to catch up with GitHub Copilot”. SN: I think what happened is this is one of those classic cases — remember, it was a tools business before, and now it is the business, who would have thought that coding is everything? Right, it should have been everything, but it seems like for some period of time, it wasn’t? SN: For us, I think what has happened is we have continued — there are two things that are happening in GitHub, before I even talk about Copilot, I should talk about GitHub. All these coding agents have shown up to work, and where have they shown up? In GitHub. And so the first thing that, quite frankly, I wish we had anticipated better, was the amount of agenting. The whole GitHub reliability thing is like one thing, but for Copilot specifically. SN: I’ll say the first thing, that’s kind of, at some level I take that job seriously, because job number one before you want to get to Copilot is go make sure that we are scaling, so let’s leave that alone. There’s a lot of people very unhappy about that. SN: Yeah, and we’re going to work it and they should have higher expectations of us and we need to deliver for them. Then the next thing is on the Copilot side, you’re absolutely right, we started by saying, “This must be just a code completions thing in the IDE”, we added chat, we added tasks, and guess what? Let’s give credit where it needs to be given. Anthropic showed up with a model. Well, this is like Cursor’s story , they ate your lunch even before Anthropic did. Or you’re saying that that was also an Anthropic story? SN: Not really, I mean it’s kind of like Cursor/Microsoft, it’s like Borland v us , it’s not like that was not the end all be all. It was really the Anthropic coming in with a completely different approach, a more agentic approach. SN: That’s right, with a different approach. With a model and what they’ve done there, and essentially the agent loop is what the change was. In fact, if you look at it, Cursor never, total volume-wise— They got eaten by the same thing, they’re facing the same challenges. SN: Also even the market share and so on — Cursor did fantastic, they forked VS Code, did a good job, lots of credit to them. But the real thing was agentic coding became real and now the good news is the agentic coding really drives — people want choice, we will be there, we will have our own models. GitHub itself and Copilot itself will have both the Anthropic and Claude. In fact, the rubber duck feature is my most favorite feature , which is I can use it to check the others. The headline announcement from this week, I guess is these new Nvidia-based PCs running Windows . However, the announcement I found much more interesting — or not an announcement, preview — Project Solara , viewing these devices as ways to access agents in the cloud, totally different center of gravity. I don’t know if it was you that said it or the presenter, something which I thought was really compelling, which is a limitation of wearables is if you have to interact with them continuously, they get very tiring, so their utility is fundamentally limited. But if you can ask an agent to do something, then you can go do something else and meanwhile, it’s running in the background. Super compelling. I guess the question is, this feels totally different than Windows — it was weird to start this keynote talking about Windows and the AI PC, and that’s nice, and local inference, but this is like, “Actually, what if everything was in the cloud?”. SN: Yeah, I always find this frame back from 2014 of ubiquitous computing and ambient intelligence and it’s becoming more and more real each day. First of all, the first part of it was, “I’m so thrilled to have these Windows machines”, and the fact that Jensen had that beautiful slide, the picture of him with all the desktops, I was like “God, yes, I’ve been waiting for it”, which is it’s great, so I think because it makes sense, it makes logical sense to have powerful silicon systems with power that really have it with unmetered intelligence. When I worked at Windows, I had to like furtively hide my iPhone and then it was okay to show up on campus with an iPhone, now I’m here with a MacBook Air — next time I interview you do I have to feel bad that I don’t have an Nvidia AI PC? SN: You will always have choice, Ben, and I hope you choose the right thing. I’m excited about that stuff because I think there’s unmetered intelligence, even there was one little feature that we showed, which is that ability to have eight agents running continuously, analyzing logs and so on, but all of them were unmetered. Right, but that feels like it’s a side project, side quest. SN: It’s kind of like a billion users all having that, that’s not a side quest. To me, it’s as fundamental as like I think the people are going to want for their knowledge work, for their security work, for their coding work, machines— They’ll want for themselves. Is this actually the new consumer/enterprise separation? SN: The enterprise — the business model, we had this long conversation about enterprises continuously optimizing — in fact, I think the biggest value prop of a Windows machine in the enterprise will be unmetered intelligence. So people are going to say, “Oh wow, instead of having my cloud bill keep going up, I’m going to have Windows machine and amortize it that way”, so I think that there is going to be a real value to — because in a world where you have infinite amount of tokens you want to consume, you want to optimize, and why would I not optimize using everything? I don’t know, I just feel like — as you know, I’ve been very impressed with the job you’ve done with Microsoft, ending the stranglehold Windows had on the company, I still remember I was actually in the Bay Area, I was sitting at the bar at The Westin by the airport typing The End of Windows , recounting all these things you did to not kill Windows, but not make it the center of gravity for the company. SN: And that I think is what goes to Solara. I don’t think Windows, we are trying to make Windows— SN: Solara, to your point, I thought it was a great question, because the thing that I want us to take a shot at is the following which is, “Can you think of a platform and platform rules, by the way, which are built for the agent era?” — because right now, what is everyone else who are “platform owners” who will try to move from the phone to this wearables will try to bring their apps to the same game, right? I want to open that up, so I would like, for example, like what we were able to do with Teams devices , and that’s where we built some of this sort of distribution capability, so I want to use that connected to this agent world so I’m excited I’m in MediaTek, Qualcomm. Well I have a great analogy for you, I think. So there’s a bit where I think you just circle back to the great job you’ve done as CEO — this is the butter-up portion of the interview — there is a bit where I think you benefited from following the follower as it were. Steve Ballmer’s one that had to go after Bill Gates and he for better or worse created the conditions for you to succeed, I think is one way to put it, is it possible that for this, your opportunity device space — like can Apple ever really make an agent that works everywhere as long as they’re stuck on the phone? SN: That’s a great question. That is the question for all of us which is you know the reality is it’s easy to say for someone who’s been so successful with something that in face continues to have a lot of success and say, “I’m going to burn it all down and build something else”. But to the point, the way they’re architectured, everyone’s vertical. SN: Exactly, it’s not natural. Like you think about it, we’re saying, “Building agents is easy”, the SOCs are jumping out everywhere, they’re there, the silicon is easy, the system is easy, the operating system is built, and now you’re telling me that I have only one choice for an ambient thing in a hotel, in a restaurant, in a healthcare setting? It makes no sense. So therefore, I imagine that building these ambient devices using Project Solara will be as easy — if you’re successful a year from now, everybody, even in the enterprise, is going to say, “Oh, I’m just going to order a bunch of these things from a no-name ODM who just built it for me”. I think it’s super smart to start at the enterprise only. Do you have dreams that maybe this will eventually spill over? SN: Right now, I want us to again do what I think is natural, like where am I seeing people— Well, that’s where you have the Microsoft 365 environment, you have all the context there. SN: And also the agents, where would people build agents? The thing is, the consumer one will be like, “I need the one agent I want”, so it’s not like I’m not building a Copilot device, I’m building an agentic platform where the healthcare provider can have their own agent, so that’s the right place for Microsoft to start, let’s see how it goes. One last question. You had a data center segment appropriately focused on communities, you talked about things like paying your way for electricity, not using water, building up the tax base, education, etc. Why not just pay the residents ? Just pay them a dividend? SN: I’m open to all ideas here, I’m not close-minded at all because at the end of the day, I think the fundamental thing you’re asking about is, “How does this industry, including Microsoft, have permission to do what we’re doing in terms of infrastructure build out?”. My theory is we get to everything backwards in the US, this is how we back into UBI [Universal Basic Income], is we’re just paying people to build data centers. SN: Yeah. And I mean, one thing that I have an issue with things like UBI and so on are the— I’m anti-UBI. That’s how you get there while being anti-UBI. SN: I want people and communities to have control, have agency, humans to have real dignity in their work and you’re 100% right in saying, “Look, we have to do what it takes to get that permission”. And so right now, there’s so much about our industry that’s so glorious, so good, so great. What about the you’re going to lose your job part? SN: Yeah, that’s the problem. Self-obsession about our own glory and our own — if you’re not creating opportunity, why would anybody want you to succeed? That’s the fundamental memo that needs to be re-sent to everyone across our industry, and then we have to live up to it. Satya Nadella, great to talk to you again. SN: Thank you so much, Ben, as always. This Daily Update Interview is also available as a podcast. To receive it in your podcast player, visit Stratechery . The Daily Update is intended for a single recipient, but occasional forwarding is totally fine! If you would like to order multiple subscriptions for your team with a group discount (minimum 5), please contact me directly. Thanks for being a supporter, and have a great day!

0 views
Evan Schwartz 1 months ago

Scour - May Update

Hi friends, In May, Scour scoured 865,266 posts from 28,671 feeds (1,766 of which were newly added), and 260 new users signed up to bring it across the 3,000 user mark! Here's what's new in the product: Scour is now better at finding posts that match your interests. You should see more relevant content and far fewer off-topic articles in your feed. (This sounds simple, but it represents at least a full month's effort 😅.) The way this works under the hood was one of the single biggest changes I've made to Scour's core ranking system since I started working on it. At a high level, scoring now combines Scour's original fuzzy concept matching (embedding vector distance) with how much the article uses relevant vocabulary (lexical search). While these ingredients are well-established, I think the exact way Scour implements them might be a somewhat novel system design. The reason this was so complex to build was that existing approaches to lexical search did not work for Scour. For example, every Scour user has between a handful and hundreds of interests (I have 642), each of which might have 3-10+ relevant keywords. This means that every "search" is actually a search for thousands of terms (for my feed, it's around 5,000). Most search systems are built for individual queries with a handful of terms. The even more tricky issue is that lexical search algorithms like BM25 do not produce scores that are comparable across queries, because they are designed for ranking (ordering results for a specific query), not scoring . Scour, however, needs to know which of your interests a given post is most related to and it sorts the posts in your feed by how relevant they are for any of your interests. I believe that the custom scoring and indexing system Scour now uses provides both cross-query score comparability and efficient lookup for thousands of parallel queries. Stay tuned for more details! 🙏 Help me out! Please like, dislike, and report posts as off-topic as you're browsing. These signals help me tune the system and figure out the edge cases where it could be improved. Scour bolds keywords in the post titles to make the feed easier to skim. The new lexical scoring layer discussed above makes it easier to bold exactly the words related to your interest. Two other small changes let you peek under the hood of the new scoring system. On desktop, hovering over a post's title will show you the score breakdown between semantic and lexical. Separately, if you click on an interest tag and go to the single-interest page, there is now an Advanced link that will show you the terms the lexical scoring system is using to find and rank posts. Here were some of my favorite posts that I found on Scour in May (you can tell from the topic concentration where my mind has been!): Happy Scouring! Latent Terms: Dense Retrievers Contain Trivially Extractable BM25-ready Zipfian Vocabularies Rethinking Agentic Search with Pi-Serini: Is Lexical Retrieval Sufficient? Re-autoresearching MSMARCO BM25, on Vespa How we made a SQL query optimization agent 59% more accurate using autoresearch and LLM Observability Your Vector Database Doesn't Know What Similar Means My Plan with RSS Agentic Coding is a Trap

0 views
Dangling Pointers 1 months ago

Yield Not Thy Core

Yield Not Thy Core Achilles Benetopoulos, Peter Alvaro, Andi Quinn, and Robert Soule EUROSYS’26 This paper describes a solution to the placement problem in distributed systems. If you model a computation as a directed graph, how do you optimally distribute the graph among a set of cooperating computers? The authors propose a dynamic placement system and implement it in Magpie . One common solution to the placement problem is to ship data over the network. For example, a set of compute nodes could access data via network requests to a separate set of nodes running Redis servers. At the opposite end of the spectrum, code can be shipped over the network. The canonical example is expressing computation as a SQL query which is sent to the node(s) that hold the relevant data. Magpie proposes a more fluid solution, where both code and data can move dynamically. In Magpie, an object represents data that is operated on. What makes Magpie objects unique is that pointers to data stored in an object are encoded as tuples. This allows Magpie to dynamically move objects around the system without invalidating pointers. The downside of this approach is that it prevents traditional libraries (that rely on raw pointers) from being used in user code. Magpie assumes a high degree of inter-object locality, so any given object is stored by exactly one node (i.e., a single object is never split between multiple nodes). User code is expressed in terms of nanotransactions and epics . A nanotransaction runs to completion on a single node and accesses a pre-specified set of objects. The Magpie runtime ensures that all objects accessed by a given nanotransaction are resident on a single node before executing the nanotransaction. The code for a nanotransaction is simple, because there is no need to query data over the network, and there is no need to deal with locking. If a hazard is present between two nanotransactions, they will execute serially. In Magpie, nanotransactions are written in Rust. An epic is a computation graph where each vertex is a nanotransaction and each edge is a data dependency. In contrast to nanotransactions, a single epic can be distributed across multiple nodes. Magpie schedules nanotransactions once all data dependencies are satisfied. Conflicts between concurrently running epics are handled via snapshot isolation . Any particular epic has a consistent view of each object and may abort in the event of a conflict. Scheduling and data movement are implemented hierarchically. A worker node can locally determine if it has ownership of all dependencies required for a nanotransaction. If this is the case, then the worker node executes the transaction immediately. Otherwise, the worker node uses a local ownership cache to try to determine if another node has all required dependencies and communicates with that node if possible. Failing that, scheduling is performed by a global orchestration node. Fig. 9 compares Magpie to memcached executing a workload that involves a user-specified read-modify-write operation: Source: https://dl.acm.org/doi/10.1145/3767295.3803616 Magpie is able to offer a lower latency because it is able to ship the entire read-modify-write operation to the server that holds the relevant data, rather than requiring multiple roundtrips. Some applications may benefit from being able to indicate that an object is rarely changed and thus can be distributed among multiple nodes at the same time. Thanks for reading Dangling Pointers! Subscribe for free to receive new posts.

0 views
Farid Zakaria 1 months ago

Leaving performance on the table

I have been working with LLVM at , and I have gotten to become familiar with the benefits of optimizing your workloads. I tend to think of optimizing my binaries as thinking about whether I have attached to my compiler flags; maybe if I’m particularly advanced that day I’ll sprinkle in some (link time optimziation) and call it a day. Turns out though that’s leaving lots of performance on the table. Compilers work under the assumption that every branch is is equally taken, unless you are hints like ( ref ). If we can feed the compilers more information about the likely path that our workloads often take, then they can produce much more performant code. There are two primary ways to optimize a binary: instrumented or statistical. When we instrument our binary, we run our workload with an instrumented binary and capture the exact paths that are executed. We will then optimize the binary perfectly tuned to that workload. If our workloads however are varied, we can collect profiles via over a length of time and create an optimized binary based on the statistical occurence of call graphs. Both approaches have their benefits however let’s start with the instrumented variant first, as it’s a little easier to follow and understand. Let’s look at a very simple benchmark. We will calculate fibonocci using SQL in sqlite3 . This is an ideal workload because it’s purely CPU-bound and ripe for optimizing. We will compile from source by downloading it. We can compile a “traditional” optimized binary that merely has and also a version that has LTO enabled since I was also keen to see how much LTO itself adds. Ok, so it looks like our program takes roughly 14-15 seconds to run. Sounds ok? How much better can we do…. 🤔 Next, we compile our program again but we instrument the binary , which effectively injects counters into the program to count invocations of functions. We get very accurate counts of our calls but the binary itself now runs much slower, which can be a problem if your workload was already very slow. Luckily for us, we are in a time domain (~15 seconds), where that is ok. After we have our instrumented binary, we run our workload again to generate the profile data and rebuild the binary with that data. The last step will be to optimize with BOLT, which is a post-link optimizer, which requires us to keep relocations so I’ve also added . When we run our workload with the final optimized binary, we see massive improvement already! 🤯 We’ve cut our workload time down to ~10 seconds which is a nearly a 1.5x improvement. Now let’s optimize the final binary with LLVM’s BOLT . BOLT is a post-link optimizer designed for “large applications”. What this means, is that it largely works by shuffling code around the binary to keep code-paths that have high temporal locality near each other (spatial locality). This can have positive impact on performance due to the instruction cache for instance. Looks like it was a little faster but not much. That makes sense since itself is a pretty small binary (~6MB), but nontheless was good to run through. Running a more thorough benchmark with we can get a final tally of our results. Looks like the I got from the Fedora ecosystem was the slowest . When all the optimizations were applied I was able to get a maximum of 1.38x faster than what was available. These optimizations would be even more dramatic for code-bases that are a sprawl and can heavily vary. Don’t worry also about getting the profile perfectly tuned to your workloads. I have a coworker who often cites that even poor profiles are still much better than no profile at all.

0 views
Simon Willison 1 months ago

Datasette Agent

We just announced the first release of Datasette Agent , a new extensible AI assistant for Datasette. I've been working on my LLM Python library for just over three years now, and Datasette Agent represents the moment that LLM and Datasette finally come together. I'm really excited about it! Datasette Agent provides a conversational interface for asking questions of the data you have stored in Datasette. Add the datasette-agent-charts plugin and it can generate charts of your data as well. The announcement post (on the new Datasette project blog) includes this demo video : I recorded the video against the new agent.datasette.io live demo instance, which runs Datasette Agent against example databases including the classic global-power-plants by WRI , and a copy of the Datasette backup of my blog. The live demo runs on Gemini 3.1 Flash-Lite - it's cheap, fast and has no trouble writing SQLite queries. A question I asked in the demo was: when did Simon most recently see a pelican? Which ran this SQL query : And replied: The most recent sighting of a pelican by Simon was recorded on May 20, 2026 . The observation included a California Brown Pelican, along with a Common Loon, Canada Goose, Striped Shore Crab, and a California Sea Lion. Here's that sighting on my blog , and the Markdown export of the full conversation transcript. My favorite feature of Datasette Agent is that, like the rest of Datasette, it's extensible using plugins. We've shipped three plugins so far: Building plugins is really fun . I have a bunch more prototypes that aren't quite alpha-quality yet. Claude Code and OpenAI Codex are both proving excellent at writing plugins - just point them at a checkout of the datasette-agent repo for reference and tell them what you want to build! I've also been having fun running the new plugin against local models. Here's a one-liner to run the plugin against gemma-4-26b-a4b in LM Studio on a Mac: Datasette Agent needs reliable tool calls and the ability for a model to produce SQL queries that run against SQLite. The open weight models released in the past six months are increasingly able to handle that. Datasette Agent opens up so many opportunities for the LLM and Datasette ecosystem in general. It's already informed the major LLM 0.32a0 refactor which I'm nearly ready to roll into a stable release, maybe with some additional "LLM agent" abstractions extracte from Datasette Agent itself. I've been exploring my own take on the Claude Artifacts, which is shaping up nicely as a plugin. I'm excited to use Datasette Agent to build my own Claw - a personal AI assistant built around data imported from different parts of my digital life, which is a neat excuse to revisit my older Dogsheep family of tools. We'll also be rolling out Datasette Agent for users of Datasette Cloud . Join our #datasette-agent Discord channel if you'd like to talk about the project. You are only seeing the long-form articles from my blog. Subscribe to /atom/everything/ to get all of my posts, or take a look at my other subscription options . datasette-agent-charts , shown in the video, adds charts to Datasette Agent, powered by Observable Plot . datasette-agent-openai-imagegen adds an image generation tool to Datasette Agent using ChatGPT Images 2.0 . datasette-agent-sprites provides tools for executing code in a Fly Sprites persistent sandbox.

0 views
Aran Wilkinson 2 months ago

How I lost a database and learned to actually use AI

I ran AI-generated SQL without reading it properly and lost a database. The experience changed how I work with AI tools, replacing freeform chat sessions with a structured process built around PRDs, small tasks, and frequent commits.

0 views
Simon Willison 2 months ago

Vibe coding and agentic engineering are getting closer than I'd like

I recently talked with Joseph Ruscio about AI coding tools for Heavybit's High Leverage podcast: Ep. #9, The AI Coding Paradigm Shift with Simon Willison . Here are some of my highlights, including my disturbing realization that vibe coding and agentic engineering have started to converge in my own work. One thing I really enjoy about podcasts is that they sometimes push me to think out loud in a way that exposes an idea I've not previously been able to put into words. A few weeks after vibe coding was first coined I published Not all AI-assisted programming is vibe coding (but vibe coding rocks) , where I firmly staked out my belief that "vibe coding" is a very different beast from responsible use of AI to write code, which I've since started to call agentic engineering . When Joseph brought up the distinction between the two I had a sudden realization that they're not nearly as distinct for me as they used to be: Weirdly though, those things have started to blur for me already, which is quite upsetting. I thought we had a very clear delineation where vibe coding is the thing where you're not looking at the code at all. You might not even know how to program. You might be a non-programmer who asks for a thing, and gets a thing, and if the thing works, then great! And if it doesn't, you tell it that it doesn't work and cross your fingers. But at no point are you really caring about the code quality or any of those additional constraints. And my take on vibe coding was that it's fantastic, provided you understand when it can be used and when it can't. A personal tool for you, where if there's a bug it hurts only you, go ahead! If you're building software for other people, vibe coding is grossly irresponsible because it's other people's information. Other people get hurt by your stupid bugs. You need to have a higher level than that. This contrasts with agentic engineering where you are a professional software engineer. You understand security and maintainability and operations and performance and so forth. You're using these tools to the highest of your own ability. I'm finding the scope of challenges I can take on has gone up by a significant amount because I've got the support of these tools. But I'm still leaning on my 25 years of experience as a software engineer. The goal is to build high quality production systems: if you're building lower quality stuff faster, I think that's bad. I want to build higher quality stuff faster. I want everything I'm building to be better in every way than it was before. The problem is that as the coding agents get more reliable, I'm not reviewing every line of code that they write anymore, even for my production level stuff. I know full well that if you ask Claude Code to build a JSON API endpoint that runs a SQL query and outputs the results as JSON, it's just going to do it right. It's not going to mess that up. You have it add automated tests, you have it add documentation, you know it's going to be good. But I'm not reviewing that code. And now I've got that feeling of guilt: if I haven't reviewed the code, is it really responsible for me to use this in production? The thing that really helps me is thinking back to when I've worked at larger organizations where I've been an engineering manager. Other teams are building software that my team depends on. If another team hands over something and says, "hey, this is the image resize service, here's how to use it to resize your images"... I'm not going to go and read every line of code that they wrote. I'm going to look at their documentation and I'm going to use it to resize some images. And then I'm going to start shipping my own features. And if I start running into problems where the image resizer thing appears to have bugs or the performance isn't good, that's when I might dig into their Git repositories and see what's going on. But for the most part I treat that as a semi-black box that I don't look at until I need to. I'm starting to treat the agents in the same way. And it still feels uncomfortable, because human beings are accountable for what they do. A team can build a reputation. I can say "I trust that team over there. They built good software in the past. They're not going to build something rubbish because that affects their professional reputations." Claude Code does not have a professional reputation! It can't take accountability for what it's done. But it's been proving itself anyway - time and time again it's churning out straightforward things and doing them right in the style that I like. There's an element of the normalization of deviance here - every time a model turns out to have written the right code without me monitoring it closely there's a risk that I'll trust it at the wrong moment in the future and get burned. It used to be if you found a GitHub repository with a hundred commits and a good readme and automated tests and stuff, you could be pretty sure that the person writing that had put a lot of care and attention into that project. And now I can knock out a git repository with a hundred commits and a beautiful readme and comprehensive tests of every line of code in half an hour! It looks identical to those projects that have had a great deal of care and attention. Maybe it is as good as them. I don't know. I can't tell from looking at it. Even for my own projects, I can't tell. So I realized what I value more than the quality of the tests and documentation is that I want somebody to have used the thing. If you've got a vibe coded thing which you have used every day for the past two weeks, that's much more valuable to me than something that you've just spat out and hardly even exercised. If you can go from producing 200 lines of code a day to 2,000 lines of code a day, what else breaks? The entire software development lifecycle was, it turns out, designed around the idea that it takes a day to produce a few hundred lines of code. And now it doesn't. It's not just the downstream stuff, it's the upstream stuff as well. I saw a great talk by Jenny Wen , who's the design leader at Anthropic, where she said we have all of these design processes that are based around the idea that you need to get the design right - because if you hand it off to the engineers and they spend three months building the wrong thing, that's catastrophic. There's this whole very extensive design process that you put in place because that design results in expensive work. But if it doesn't take three months to build, maybe the design process can be a whole lot riskier because cost, if you get something wrong, has been reduced so much. When I look at my conversations with the agents, it's very clear to me that this is moon language for the vast majority of human beings. There are a whole bunch of reasons I'm not scared that my career as a software engineer is over now that computers can write their own code, partly because these things are amplifiers of existing experience. If you know what you're doing, you can run so much faster with them. [...] I'm constantly reminded as I work with these tools how hard the thing that we do is. Producing software is a ferociously difficult thing to do. And you could give me all of the AI tools in the world and what we're trying to achieve here is still really difficult. [...] Matthew Yglesias, who's a political commentator, yesterday tweeted , "Five months in, I think I've decided that I don't want to vibecode — I want professionally managed software companies to use AI coding assistance to make more/better/cheaper software products that they sell to me for money." And that feels about right to me. I can plumb my house if I watch enough YouTube videos on plumbing. I would rather hire a plumber. On the threat to SaaS providers of companies rolling their own solutions instead: I just realized it's the thing I said earlier about how I only want to use your side project if you've used it for a few weeks. The enterprise version of that is I don't want a CRM unless at least two other giant enterprises have successfully used that CRM for six months. [...] You want solutions that are proven to work before you take a risk on them. You are only seeing the long-form articles from my blog. Subscribe to /atom/everything/ to get all of my posts, or take a look at my other subscription options .

0 views
iDiallo 2 months ago

Have You Seen the New Excel?

Stop coding. Stop hiring. Stop building. While the tech world obsesses over large language models and neural networks, I discovered the real disruptor that has been hiding in plain sight. Mine was originally installed on my desktop in 1992. And now, it's about to change everything in the world. We are talking about Microsoft Excel, of course. If you haven't looked at a spreadsheet lately, you are missing the most significant leap in enterprise capability since the invention of the corporation itself. We are entering an era of No-Code where the code was never needed in the first place. My own job as a software engineer is not safe, and I'm looking forward to the future. Developers from every walk of life are afraid, and for good reasons. You hear the complaints constantly: "How can I ensure the code works? I can't possibly review a PR with a thousand files. It's unmaintainable." This is a crisis of confidence in the software engineering sector. This specific anxiety has never existed in the Excel ecosystem. Code is called code for a reason, it is meant for the machine to read, not people. In Excel, we don't worry about "reviewing pull requests." We worry about results. The spreadsheet handles the logic and you handle the business outcome. It abstracts away the complexity so you don't have to pretend to understand it. And let's talk about the intimidation factor. Have you ever opened a modern codebase? It's a labyrinth of directories, dependencies, and config files. Where do you even start? It's paralyzing. How do you get started with Excel? You double-click an icon. It opens. It is a file. It is a grid. You type. It works. The barrier to entry is non-existent, yet the ceiling is infinite. If you are getting paid a high salary, and are watching how efficient excel is, you will be terrified. Companies are realizing they don't need distinct software solutions for distinct problems. They just need a grid. We are seeing enterprises replace entire departments with a single file. That is not an exaggeration. The HR department? Replaced by an org chart linked to a payroll calculator. The supply chain team? Replaced by a real-time inventory tracker. The marketing department? Replaced by a pie chart and a mailing list. Why pay for Salesforce? A well-formatted sheet with conditional formatting is a Customer Relationship Manager (CRM). Who even knows how to write SQL? SQL is legacy. A workbook with 1 million rows is a database. Jira is redundant when you have Gantt charts generated from cell dependencies. On top of it all, it has AI. It comes equipped with Microsoft Copilot for 365 apps, not to be confused with Windows Copilot, Microsoft Copilot, Copilot for Teams, Copilot+, Copilot Chat, or Copilot Web. This is the Copilot. It sits inside your grid, ready to extrapolate trends from column D and write your VLOOKUPs for you. While other AI startups are fighting for funding rounds, this integration is already live, embedded directly into the tool that runs the global economy. You aren't hearing much about Venture Capital funding or Series A rounds when it comes to Excel. Why? Because it is already profitable. It doesn't need a roadmap to profitability because it is the roadmap. While other platforms burn cash to acquire users, Excel is the default operating system of business. It requires no adoption curve. It requires no evangelists. It requires only that you open it and have a Microsoft 365 apps subscription. Total Vertical Integration Excel is versatile. It is a text editor; you can write your novel in cell B2. It is a design tool; pixel-perfect layouts can be achieved by merging cells and removing gridlines. It is an IDE; you can write and execute VBA code directly within the environment. It handles the visual and the logical simultaneously. You can present a quarterly report to the board while the underlying formulas are calculating the ROI of the lunch break. It creates a seamless workflow where the input and the output exist in the same plane. Privacy, Scalability, and The Cloud For the enterprise client, Excel offers the ultimate flexibility. Are you concerned about data sovereignty? Run your entire global operation locally on a ThinkPad from 2012. The file sits on your hard drive, unbreachable by the cloud. Do you need to scale? Push it to the cloud. Collaborate in real-time. Ten thousand employees can edit the same cell, creating a hive mind of productivity that traditional management structures cannot compete with. Oh, if you want to add support for crypto, just add a new worksheet. Batteries are included. The Future is a Cell The economy is shifting. We are moving away from specialized labor and toward generalized grid management. If your job involves inputting data, processing data, or presenting data, Excel has already automated you. It doesn't sleep, it doesn't ask for a raise, and it doesn't make calculation errors unless you tell it to. Best of all, it doesn't hallucinate. The grid is absolute, it is infinite and the grid is the future. Learn Excel now, or get left behind. That’s what AI Hype sounds like to my ears. Yes, it’s a great tool. But I don’t think we are all gonna die and lose our jobs. The same way we didn’t die and or lose our jobs to Excel. None of these things are jokes about Excel by the way, you can run entire companies from it. I'm tempted to just start hyping it everyday until everyone gets annoyed.

0 views
Robin Moffatt 2 months ago

Materialized Tables in Apache Flink

Flink added support for what it calls Materialized Tables in 1.20 , released in 2024. You can read about the design and motivations in FLIP-435 . In a nutshell, Materialized Tables provide a way to include the SQL to populate and refresh a table as part of its definition.

0 views
alikhil 2 months ago

How to Quickly Prepare for Software Engineering Interviews

A few months ago, I found myself needing to prepare for a series of job interviews within a very limited timeframe. It was a stressful experience, but it ultimately worked out well. I decided to share my notes and reflections in case they’re helpful to others in a similar situation. This is especially relevant if you’re not actively job hunting and suddenly receive an interview invitation, leaving you with limited time to prepare but a strong desire to maximize your chances of success. Disclaimer : The tips described in this post may be more useful for senior engineers with hands-on experience and engineering intuition. The internet is full of articles listing all possible HR interview questions. I recommend spending a bit of time on them just to understand what to expect and not be surprised. However, in my humble opinion, there are two main points to focus on during HR interview preparation. First, you need a short story that tells your experience briefly. Avoid listing every bullet point from your CV. Instead, focus on highlighting your key achievements. Also, your story must be aligned with the position you are applying to. Yes, you might need to adjust your story for different jobs at different companies. Second, it’s important to have a clear motivation. Why do you want to change your job, and why this company/role? What kind of job are you looking for? If you have some experience doing System Design interviews or have never done it, start by learning the Delivery framework . Understand each section. Watch at least one video on how it’s done. The more, the better. These videos from Hello Interview channel are really good, though. If you are applying to a FAANG company, you may search for leaked system design questions from that company and spend some time preparing for them. But there is no guarantee that you will get the same topic, thus I would not recommend spending all your time here. If you can, do a mock interview. Ask a friend or find someone to practice with. If you can’t, then try to walk through alone, but talk through everything out loud. During the interview, treat the interviewer as a colleague, ask questions, ensure you understand the problem, and that you have not missed any important requirements before building the design of the system. Don’t rush. This part is really tricky. If the company tends to use LeetCode-style interviews, there is no shortcut here. You need to solve hundreds of them to really feel confident. You may need to refresh your memory on algorithms you feel less confident about (for example, I always forget about corner cases for binary search). Again, if it’s a big / well-known company, you can try to search for leaked coding interview questions. S.T.A.R (situation task action result) & C.A.R.L (context action result learning) There are dozens of questions you could be asked in behavioral interviews. And you’re expected to structure your answers using the STAR framework. This means you need to tell a story by defining a context, your actions, and results. You could go and just prepare a STAR format answer to all such questions, but it will take a lot of time, and it’s suboptimal. This, combined with the fact that the same stories can be used for different questions, makes the situation easier for you. You can prepare 7–10 stories that will cover most of the questions. During preparation, you can write them as text, but don’t read them during the interview. It tends to sound unnatural. When telling your story using the STAR method, make sure your final sentence clearly highlights a positive outcome. Adjust your tone to emphasize this closing part so it stands out. The STAR framework is a standard. But also check CARL in some questions, it would be good to tell what you have learned from that story. Here are some materials that helped me to prepare for a behavioral interview: Some companies have such an interview stage. It’s quite unpopular but still exists. You’re asked to present a project or problem you worked on. You explain the context, problem, solution, results, and your role in this story. It’s like showing the result of your work to colleagues from different departments/teams. This stage is very open-ended. You are not given specific instructions, and there is not much information on the internet with recommendations on how to prepare and conduct such interviews. When I found out I would have this interview, I was initially shocked and unsure how to prepare, as I didn’t know what to expect. It wasn’t until I realized that in reality, it’s you, the interviewee, who rules this interview . You choose the project, decide what to include and omit, control the level of detail, and you are coming up with the story you know, with all the answers for all possible questions, because it’s your story. So, make the most of this stage. Prepare your story, make a few slides / notes / architecture sketches. Don’t dig into details too much. Leave a space for the questions. And even if there is no dedicated interview, you may be asked to tell in detail about a certain problem/project you were working on. So, be prepared. Have your story! When answering open-ended questions, aim to tell stories where the scale of the problem matches the level of the role you’re applying for . For example, if you are asked, “Tell me about a challenging/interesting problem/task you were working on recently.” Optimizing an SQL query by adding an index may be fine for junior roles, but it won’t carry enough weight for senior positions. Interviewers would expect to hear something bigger, challenging, higher stakes, and often involving cross-team collaboration, such as migrating a large system to Kubernetes. Question back . You should ask questions to learn more about the company, their culture, the hiring manager’s management style, and what they like or dislike about their work. Prepare a list of questions before the interview. Start preparing in advance . Even if you’re not planning to change jobs anytime soon, you can begin investing in your future by: Hello Interview - Behavioral Interview Discussion with Ex-Meta Hiring Committee Member - must watch Behavioral interview, although I would recommend watching it even before the HR interview, because it gives a bunch of helpful tips about self-presentation https://thebehavioral.substack.com/ - Strategies, tips, and resources to prepare for your next behavioral interview from a FAANG+ insider. solving one LeetCode problem a day keeping track of tasks/projects you’ve completed, along with your achievements (many companies require this anyway for performance reviews) – this would be a foundation for your stories in behavioral and project walkthrough interviews. keeping your CV and LinkedIn up to date.

0 views