Simon Willison 3 weeks ago

Jev introduces a new shape of LLM - System One, aka Decision Models

Last week TypeSafe AI unveiled Jev , their first example of a new category of model that they are calling "System One models" (I'm with Maggie Appleton, I think "decision models" is a better name for these). Jev is an interesting variant on the usual LLM format: it still accepts text inputs, but instead of text output it returns floating point numbers corresponding to categories, yes/no questions, ratings, and associated confidence scores. TypeSafe describe Jev like this: Think of Jev as a frontier-intelligence function call: unstructured state in, typed probabilistic decisions out. It's also very fast, and really cheap . Regular LLMs are priced in terms of input and output tokens, with output generally charged at significantly higher rates. Jev charges only for input - output is free - and the input price of their first model is $0.042 per million tokens - cheaper even than OpenAI's GPT-5 Nano ($0.05/million). Jev lets you ask questions about text or semi-structured data. You compose a "state" object containing a string, array of strings, or set of name-value pairs - this might describe an article, or a customer, or any other kind of record. You then send that to their API with one or more questions, and get a reply back for each. You can ask three kinds of questions: The Jev API can accept a single document ("state") and as many questions as you can cram into the context window. Questions are evaluated in parallel, so sending many questions should take a similar time to sending just one. The Jev 1.13 jaggedness documentation offers useful guidance as to Jev's strengths and weaknesses. It's currently not great with numbers, dates, or "adversarial content". I think the decision model framing is useful for understanding where to use Jev. It's great for anything that can be expressed as a classification task - think spam detection, suggesting labels, prioritization and ranking. I've also been experimenting with it for search reranking, where you fetch 100 likely matches using an inexpensive algorithm like BM25, then have Jev score those 100 candidates for relevance against the original query. Something I've found a little uncomfortable about Jev is how it very much represents a regression even further towards black box machine learning systems. LLMs are black boxes already - you can ask them to justify their decisions, but you can't guarantee that what they say is useful or accurate. Jev doesn't even give you that: put in all the text you want, the only thing you're going to get back is a floating point number. If Jev marks something as spam, which content signals tipped it off? This also means that concerns about bias should be front and center. I really hope nobody uses Jev to rank job applicants - that floating point number could conceal all manner of unseen bias baked into the models, and experimentally picking that bias apart is going to be a tricky business. (I tried one experiment where I had Jev score every city in the San Francisco Bay Area on a yes/no answer to whether they were a "Good city?" - it rated Cupertino top and East Palo Alto bottom. Huh.) In practice, this all means that evals and structured experiments are even more important than they are for regular LLM projects. Thankfully, Jev is so cheap that running hundreds or even thousands of experimental prompts through it costs just a few cents. It's been really fun watching the wider community come up with potential use-cases for Jev over the past few days. Here are some creative ones that caught my eye: There's also been a flurry of projects attempting to create a model like Jev using on top of open weight models. Kev is one interesting example, using Qwen 3.5 to produce 0.8B, 4B, and 9B models. Here's the accompanying Hacker News thread , where someone linked to a JevBench benchmark that has already cropped up to compare "Jev-class decision models". Given Jev was released just under a week ago, the amount of activity around it is extremely impressive. You are only seeing the long-form articles from my blog. Subscribe to /atom/everything/ to get all of my posts, or take a look at my other subscription options . Yes/No questions, which Jev calls "Noul" questions - their CEO confirmed on Hacker News that this is short for Bernoulli, from the Bernoulli distribution . You pose a statement and get back a floating point number between 0 and 1 for how confident the model is that the statement is true. Choice questions, where the model picks one from a set of provided options - actually a confidence score plus a probability distribution across all of the options. Score questions, where you provide sequence of numeric levels with descriptions and it provides a floating point score somewhere along that range. jevchat by Kyle Pena turns Jev into a (terrible) chat model. "At every step it asks Jev one question: Given the user's question and the reply written so far, which symbol comes next?". ericpruitt on Hacker News : "It's the digital equivalent of Morty speaking with the death crystal". jev-leftpad by Fatih Kadir Akın implements left-pad with the prompt "How many spaces are needed before value to reach targetLength?" and a choice query allowing options from "0 spaces are needed" to "10 spaces are needed". jev-2048 by Andy Gayton uses Jev to play the 2048 sliding puzzle game .

API AI

▲ 0

0 views

neilzone 1 months ago

RSS feeds for individual authors of The Register using The Register's new API

Tech website The Register used to offer author-specific RSS feeds . It has recently changed how it does RSS feeds , to use the new api.theregister.com. Thanks to a kind fedizen for showing how one can still use this new API to get author-specific feeds: One can also use instead of , and this brings back a slightly different set of posts. I don’t know which is “right” or better for my use case. Annoyingly, whichever selector one uses, this results in one single massive block of text, without paragraphs or other formatting. Which is better than nothing, but not ideal. I have yet to find a way to deal with this.

Web Development Open Source API

▲ 0

0 views

Himanshu Anand 2 months ago

Two Parsers, One JSON and a Flag: Intigriti's July 2026 Challenge

Intigriti’s July challenge is called “Canonically Yours” and the whole bug is one silly question: what happens when you put the same key in a JSON object twice? Turns out two different bits of code answer that differently and that gap give you the flag. Its a mini package registry called Registry Observatory . You register you get a private namespace like and you can generate signed “compatibility reports” for packages you own. The flag lives in a report for a package you do not own. A quick read of the app “Observatory archive” endpoints tells you the target for free: there is a package in platform scope called the treasure is . The manifest looks like this: The sign step is the bouncer It only signs manifests for packages you actually own swapping to gets you a flat The title says it: canonicalization it kinda mostly means two pieces of code read the same input differently. The reproduction steps even mention your namespace going in the “first” package object which means there can be a second one. So send the key twice : The bet: the signer reads the first (mine, so it approves) and the report generator reads the last ( it builds that report). Same bytes, two readers with two answers. you cannot use , it silently drops the duplicate key, So You have to build the raw JSON as a plain string and Base64 that yourself. The sign step returns , the report comes back for and its release notes are the flag. Two quick dead ends that pin down exactly how the bug works: putting first gets rejected at signing (the bouncer reads the first key and bounces you) and duplicating inside one package object also fails the two parsers split at the level not deeper. The same signed blob was read by two JSON parsers that disagree on duplicate keys. One kept the first value and the other kept the last. The signature was valid it just covered the wrong half of the message. This class of bug is a parser differential (JSON duplicate-key confusion) and it has caused real auth bypasses and signature-smuggling in the wild. The fix is boring: use one parser, reject duplicate keys instead of guessing and act on the exact bytes you signed. Thanks to Intigriti and @silent_web3_ for a clever one. Go read your JSON parsers, they might not agree with each other. Build a small JSON “manifest” describing the package. Base64 it and send it to . The server checks it and, if it likes it, signs it. Send the same manifest plus the approval to . Read the report at .

Security JSON Backend API

▲ 0

0 views

Simon Willison 2 months ago

Stateless MCP has recaptured my interest (and inspired mcp-explorer and datasette-mcp)

Tuesday was Stateless MCP day - the rollout of MCP 2.0, or the 2026-07-28 Model Context Protocol specification to use the more formal but less memorable name. This is the most significant change to the MCP spec since it first launched, and has also served to reignite my personal interest in the protocol. For background: MCP is the Model Context Protocol, which describes a standard way to expose new tools to LLM-powered agent frameworks. It was introduced by Anthropic back in November 2024 , had a huge spike of interest through much of 2025, and then became somewhat eclipsed by Skills (another Anthropic invention) when it became apparent that an agent harness with access to a terminal and could do most of what MCP did in a more flexible way. I wrote about that in my review of 2025 . I'm coming back around to MCP now. Giving an agent a shell environment with the ability to access the internet is fraught with risk , and requires a strong model that is capable of effectively driving such an environment. MCP tools are easier to audit and control, and simple enough that smaller models that run on a laptop can still drive them reasonably well. The new stateless MCP specification also greatly decreases the complexity of implementing both clients and servers for the protocol. I built three of those this week! The best demonstration of the difference between stateful and stateless MCP is in this May 21st blog post that introduced the RC for the new specification. It included a clear before-and-after example. The older stateful MCP (I'm going to call it "legacy MCP") required two HTTP requests - the first to initialize a session and obtain a , and the second to actually call the tool: The new stateless way uses a single HTTP request which looks like this: This is so much cleaner from both a client- and server-side implementation perspective. It's also a better fit for building scalable web applications, since now you don't need to maintain server-side state to keep track of those session IDs, or worry about routing the same session to the same backend machine. I couldn't find a great CLI tool for interactively probing an MCP server, so I had Codex help build my own. mcp-explorer is the result. It's a stateless Python CLI tool, so you don't even need to install it to try it out - it works with uvx like this: This queries Ade Oshineye's agentic-mermaid.dev demo MCP. The above command returns the following list of tools: Then to inspect a tool: This outputs a whole bunch of information, including the JSON schema of the inputs and outputs. To call that tool and pass arguments to it: Which returns: To get just the raw SVG try adding to that command. I got back this image : There are a few more commands in the README, but you get the general idea. I find building CLI tools like this to be a really productive way to get familiar with a specification, even if an agent writes most of the actual code. The second project is datasette-mcp , a Datasette plugin which adds a endpoint to any Datasette instance. This is probably the fourth time I've tried building this plugin, but thanks to the new stateless MCP specification I finally have a version that feels good to release. It provides just three tools: , , and . They do exactly what you would expect them to do - though is read-only for the moment. Wire these into an agent, or a chat tool like ChatGPT or Claude, and they'll gain the ability to run SQL queries against your hosted Datasette instance. So far I'm running it on the Datasette mirror of my blog, at datasette.simonwillison.net/-/mcp . It took a bit of fiddling to figure out how to attach that to ChatGPT and Claude, but I got there in the end. Here's a new TIL showing exactly how to do that. Here's a shared Claude session where I asked it: It ran 7 separate SQL queries to figure out the answer. My LLM tool is long overdue for an official MCP integration. The new alpha llm-mcp-client plugin is my attempt at exactly that: Here's the output (including reasoning trace, I'm using LLM 0.32rc2 ): Considering note count I see the question "count the notes" is probably asking me to tally up blog notes. It could also mean published notes or drafts, so there's some ambiguity there. I'll need to figure out the total number of notes, likely by querying the count for both published notes and drafts to get a clear answer. Let's execute that count! There are 151 notes . And the output of llm logs for that prompt. Once this is fully baked, I'm considering bringing it directly into LLM core. I'm excited to experiment with MCP in Datasette Agent and llm-coding-agent as well. A few months after MCP was first released, I wrote Model Context Protocol has prompt injection security problems , where I noted that the pattern of having end users mix and match tools pushed responsibility for avoiding data exfiltration attacks out to the users themselves. I hadn't coined the Lethal Trifecta yet, but that was absolutely what I had in mind. Then general agents with arbitrary shell and access came along, and that's so much harder to keep secure! Something I've come to appreciate about MCP is that it's much easier to reason about agent capabilities and what might go wrong than with arbitrary command execution in an open network environment - the default for most of today's general and coding agent tools. I plan to lean into MCP a whole lot more when I'm building sensitive applications on top of LLMs. You are only seeing the long-form articles from my blog. Subscribe to /atom/everything/ to get all of my posts, or take a look at my other subscription options .

C# JSON AI Programming API

▲ 0

0 views

Filippo Valsorda 2 months ago

Opaque, Interoperable Passkey Records (and a Go API)

Passkeys are the most important thing happening in information security right now because they are the only principled solution to the overwhelming effectiveness of phishing attacks. Just like memory safety is the only principled solution to memory corruption attacks. Unfortunately, implementing them on the server side can appear more complex than using password hashes. Part of this is unavoidable because passkeys require interaction with the browser to get their phishing resistance properties. Part of it, however, could be abstracted away a little more effectively by defining interoperable passkey record encodings. The WebAuthn specification defines a credential record as an abstract concept with a number of components such as , , , , , and other flags. Google recommends a database table with a Credential ID primary key, and , , and columns. Adam Langley’s stellar Tour of WebAuthn similarly recommends a primary key, and separate and columns. All guidance recommends using libraries to handle WebAuthn authentication, but that still leaves applications with a potentially non-interoperable database schema. It might also be impractical to defer the whole authentication flow and database interaction to a library or framework. Interoperable, well-specified passkey records that the application can handle as opaque strings, like password hashes, can be a middle-ground abstraction layer. c2sp.org/passkey-record is a specification proposal which borrows the syntax of Password Hashing Competition (PHC) Strings and reuses the existing authenticator data encoding for the bulk of the work. A record looks like this: The payload is the authenticator data, a binary encoding of most of the credential record fields that is already specified by WebAuthn and included in the JSON encoding of an AuthenticatorAttestationResponse (which is the return type of even if attestation is not in use). Transports are the only missing field, and they are stored as PHC parameters. The application is then only in charge of keeping track of what passkey records are associated with a user account, which is a task that is familiar to web developers because it is not dissimilar to implementing password authentication (except there are multiple passkeys per account). These opaque strings can be passed to a library to verify the login assertion (or to generate a registration request with the appropriate ). With a well-specified interoperable storage format, it will hopefully be possible to switch passkey library (or even backend language) while preserving a credentials database. Along with the passkey record, applications might still wish to store metadata fields like a user-selected nickname and creation and last use timestamps, to provide pretty passkey management UIs. None of these require any special treatment by the WebAuthn library. The one exception is the backed up state flag. Passkeys report to the server whether they are backed up (e.g. to iCloud Keychain or a Google account), and servers can use that signal to suggest removing a password from an account. This flag can change across logins, while the passkey record is immutable, so it would have to be stored separately and updated on every login. I think this logic is overrated for the average website, which will keep supporting email password resets anyway. Building on top of these passkey records, I drafted a potential stateless Go package API . The registration flow is The login flow is The application is in charge of The library provides JSON values that can be passed straight to and , and accepts JSON values returned by calling on a PublicKeyCredential . This is optimized for the discoverable credential flow (a.k.a. passkeys, where the authenticator stores and provides to the server the user ID) but the method can also be used for second-factor flows or re-authentication prompts. The API works for both modal and conditional UI (autofill) flows. There are a few helpers to extract information from the passkey record ( , ) and from the JSON-encoded PublicKeyCredential ( ). Currently, there is no implementation; I would like to get feedback on the passkey record format and on the Go API , before potentially making a proposal for Go 1.28. One thing you can’t do with this storage model is ensure that different accounts don’t share passkeys with the same Credential ID, which the spec says you SHOULD do. The reason for that check is avoiding attacks where you look up a credential by its ID and land at the wrong public key or user ID because the attacker intentionally injected a colliding Credential ID through their own account. This attack simply can’t happen if you don’t have a Credential ID index in the first place! The index is only needed to mitigate an attack introduced by the existence of the index. Login attempts carry the user ID, and if you use that to look up the user’s passkey records to verify the login against, it doesn’t matter if some other user has a passkey with the same Credential ID, just like it doesn’t matter if two users share a password. Don’t let the attacker dictate your PRIMARY KEY and you won’t have PRIMARY KEY collision attacks. For more Go API previews, follow me on Bluesky at @filippo.abyssdomain.expert or on Mastodon at @[email protected] . More from this year’s CENTOPASSI (a GPS-tracked motorcycle competition involving careful planning, 100 coordinates, and 1700 km of secondary roads over three and a half days). Here’s a glimpse of Castel del Monte (AQ), after climbing down from a deserted and still snowy Campo Imperatore. My work is made possible by Geomys , an organization of professional Go maintainers, which is funded by Ava Labs , Teleport , Datadog , Tailscale , and Sentry . Through our retainer contracts they ensure the sustainability and reliability of our open source maintenance work and get a direct line to my expertise and that of the other Geomys maintainers. (Learn more in the Geomys announcement .) Here are a few words from some of them! Teleport — For the past five years, attacks and compromises have been shifting from traditional malware and security breaches to identifying and compromising valid user accounts and credentials with social engineering, credential theft, or phishing. Teleport Identity is designed to eliminate weak access patterns through access monitoring, minimize attack surface with access requests, and purge unused permissions via mandatory access reviews. Ava Labs — We at Ava Labs , maintainer of AvalancheGo (the most widely used client for interacting with the Avalanche Network ), believe the sustainable maintenance and development of open source cryptographic protocols is critical to the broad adoption of blockchain technology. We are proud to support this necessary and impactful work through our ongoing sponsorship of Filippo and his team. call with the logged-in (or otherwise identified) user details and any existing passkey records pass the returned JSON to and then to pass the returned JSON-encoded PublicKeyCredential to store the returned passkey record in the database call while generating the log-in page store the returned request in a key-value cache with a short TTL, under , and pass the returned JSON to and then to pass the returned JSON-encoded PublicKeyCredential to , and use the returned requestID to retrieve the request from the key-value cache and use the returned userID to retrieve the passkey records from the database pass the JSON PublicKeyCredential, the request, and the passkey records to associating an opaque, permanent, privacy-preserving user ID with each user; storing passkey records associated with a user; and caching request challenges produced by .

Security API Go JSON

▲ 0

0 views

Simon Willison 3 months ago

Kimi K3, and what we can still learn from the pelican benchmark

Chinese AI lab Moonshot AI announced Kimi K3 this morning, describing it as their "most capable model to date, with 2.8 trillion parameters". It's currently available via their website and API, but an open weight release is promised "by July 27, 2026". Moonshot are calling this the first "open 3T-class model" (I guess they're rounding 2.8 trillion up to 3 trillion), taking the crown from DeepSeek's 1.6T v4 Pro . Their self-reported benchmarks have K3 mostly beating Claude Opus 4.8 max and GPT-5.5 high, while losing out to Claude Fable 5 and GPT-5.6 Sol. A few highlights from the Artificial Analysis report on the model: The model is also now the leading model on Arena.ai's Frontend Code arena , surpassing even Claude Fable 5. The new model is notable for the pricing: $3/million input tokens and $15/million output tokens, putting it at the same level as Anthropic's Claude Sonnet series and making it the most expensive model released by a Chinese AI lab to date. This is a significant increase on their earlier models such as Kimi K2.6 at $0.95/$4. 2.8 trillion parameters is also more than twice the size of that 1T model. I used OpenRouter (to avoid signing up for a Moonshot API key) with the llm-openrouter plugin to generate an SVG of a pelican riding a bicycle: Here's the transcript . It looks like this: That pelican took 95 input tokens and 16,658 output tokens (13,241 were reasoning tokens), for a total cost of 25 cents ! Since K3 accepts image input I ran it against that rendered SVG above (with my alt text prompt ) and got back (for 0.6 cents ): Cartoon illustration of a white pelican wearing a red scarf, riding a red bicycle along a gray road with white dashed lines; the pelican has a large orange beak and webbed orange feet pedaling, with white motion lines behind it; the background shows a light blue sky with white clouds, a yellow sun, two small black birds in flight, and green grass with tiny white flowers in the foreground My Generate an SVG of a pelican riding a bicycle test is 21 months old now. It was never a particularly great benchmark. It started out as a joke on how absurdly difficult it is to compare these models, but then for the first year it turned out to have a surprising correlation to how good the models actually were. That connection has been mostly severed now. The GPT-5.6 and Claude Fable 5 pelicans are outclassed by GLM-5.2 , and much as I love GLM I don't think that's a Fable-class model. (I'm still not convinced that labs are training for the benchmark - if they were, I'd expect much better results. There's a chance that Gemini has optimized for any combination of an animal on a vehicle though!) The biggest limitation of the pelican is that it doesn't touch at all on the thing that matters most for today's model: agentic tool calling and the ability to operate tools reliably as conversations grow in length. So don't go using pelicans to compare models! All of that said, I still get a decent amount of value out of running the benchmark myself. Firstly, it's a forcing function for actually trying the model. If I show you a pelican, that means I've managed to run a prompt through it. If the model has an official API I'll use that, if it's open weight (and small enough to fit a 128GB M5 MacBook Pro) I'll try running it on my own machine, usually via llama.cpp or LM Studio or Ollama . I'll frequently use OpenRouter since that usually provides a proxy to an official API without me needing a new API key. Most of my pelicans are generated using my LLM CLI tool , which helps encourage me to ensure the latest models are supported by that (via one of its plugins). More importantly though, even the act of a single prompt to "Generate an SVG of a pelican riding a bicycle" can reveal interesting model characteristics. Consider the result for Kimi K3 today. Running those simple prompts helped emphasize several points about the model. K3 currently only has one thinking effort level, but I've been deriving quite a bit of value recently from running the same pelican prompt through different effort levels to get a quick idea for what impact those have. Here's my matrix for the GPT-5.6 model family , for example. Really though the main things I gain from the pelican test are: You are only seeing the long-form articles from my blog. Subscribe to /atom/everything/ to get all of my posts, or take a look at my other subscription options . "On our private long-horizon knowledge work evaluation, Kimi K3 reaches an overall Elo of 1547, +732 points from Kimi K2.6 and behind only Claude Fable 5." "Cost per task ($0.94) is similar to GPT-5.6 Sol ($1.04), ~1/2 the price of Opus 4.8 ($1.80) and higher than open weights peers" "Kimi K3’s token usage on the Artificial Analysis Intelligence Index decreased significantly, using 21% fewer output tokens than K2.6." It only has one reasoning effort right now, "max" - and it shows. The model consumed 13,241 reasoning tokens to output 3,417 tokens of response. This is expensive - the pelican cost 25 cents! How does the prompt "Generate an SVG of a pelican riding a bicycle" add up to 95 input tokens? OpenAI's tokenizer counts 10, Anthropic's counts 10 for Opus 4.6, 30 for Opus 4.7 and 25 for Sonnet 5/Fable 5. Prompting "hi" to Kimi K3 counted 86 tokens, suggesting there may be an 85 token hidden system prompt. It refused to leak it though. Vision works well: the alt text it generated is very good. It's a "hello world" exercise for prompting a model A rough cost and reasoning estimate for a simple task Confirmation that the model can output valid SVG and has a basic idea of geometry and spatial awareness. This is a much bigger deal for the smaller models that run on my laptop. It's still interesting to compare pelicans between releases in the same model family. K3's pelican is a notable improvement from Kimi 2.5 . It's something I can share that demonstrates I've tried it. Plus a comment with a pelican in it is kind of a tradition on Hacker News at this point, any time I'm late I get comments asking where it is!

Machine Learning AI C++ API

▲ 0

0 views

daniel.haxx.se 3 months ago

Workshop Basel day three

See also: day one, day two . There is only one thing that is better than two days of HTTP workshop, and that is of course three days of HTTP workshop. The final day of this edition of the series started out with us again shuffling around where we parked ourselves around the big table. Except Mr captain of course who once again got to herd us forward through another day from the same seat. MOQ ( Media over QUIC transport ) is not HTTP, but it uses QUIC so it is at least tangentially interesting and it involves a lot of the same people so this status update still felt welcome and suitable. Compared to existing HTTP based solutions, MOQ is supposed to offer less complexity and lower latency. The moon landing was broadcasted with less latency than current live-streamed TV and maybe MOQ can make us come close to those numbers again. In MOQ clients subscribe to a track that then contains a lot of objects that are delivered. It’s not the request + response approach of HTTP. The fact that this is not HTTP of course brings a lot of questions and well, doubts, and we lingered on various aspects of this topic for quite a while. My prize for the best slides of the HTTP workshop 2026 goes to [redacted] for the excellent use of potato images in their presentation. PTTH is HTTP spelled backwards, commonly pronounced as PoTaToH. A client sets up the connection but the actual HTTP request is sent from the server to the client. One of the intended use cases for this, is to allow an origin server to connect to the CDN proxy and then be able to deliver traffic to the world, rather than to have the CDN connect to the origin the way they usually do. Apparently most CDNs already have custom and proprietary solutions for exactly this kind of feature, so maybe doing it in a standard way instead makes sense? The draft explains the new proposed way to continue a previously interrupted upload over HTTP. The upload request gets a Location: header back for the resource being uploaded, and if it gets stopped prematurely, a client can then HEAD that resource, figure out the size and then do a second upload (using the PATCH method) request that tells the server that this transfer should start at offset X. Exactly how this should be supported in browser’ upload forms seemed a little bit uncertain . For my own sake I can see a challenge to implement this nicely for curl in particular when the upload is using formpost upload (curl’s -F flag) which after all still is a very common way to do uploads on the current web. I’ll return to this topic at a later time when I written an implementation to test… io_uring is a Linux asynchronous I/O framework that avoids the overhead of traditional system calls. It uses two shared ring buffers between user space and the kernel, allowing applications to batch I/O operations with zero-copy efficiency. The feature is disabled by Google in ChromeOS, Android and in production Google servers which certainly holds back some use of it. io_uring can be helpful to speed up things, but might be complicated to use in existing software architectures and the presentation went into some details on why this is so. A walk-through of some of the recent developments and improvements in Firefox’s UDP networking stack . Going from single datagrams to the modern ways to ship large chunks of data offloaded to the kernel to speed things up. Upload throughput in Firefox is up 60-90% over the last 11 releases. Lots of fun graphs and metrics were shown. This work is based on the quinn-udp stack. Happy Eyeballs v3 is coming and Firefox is implementing it . It now takes into account many more data sources than before, including alt-svc and HTTPS-RR and races connections against each other to use the one that connects first. There are some recommended timers in the specification and parts of the discussion was around how maybe the timers could instead be tightened a bit, and maybe the delay between the subsequent attempts could then use an exponential backoff instead sticking to a fixed interval? (I know I’ll discuss some of these details with my curl hacker friends and see what we should adjust… curl already supports most of the Happy Eyeballs v3 specification.) As we approached the end of the day a few shorter topics were ventilated to give us a little more to consider before going home: With this, the seventh HTTP workshop had ended. Again a very fine event. This time graciously sponsored and arranged by Adobe. Thank you everyone! The general idea is to continue with these events roughly every second year and I support this. The HTTP workshops are definitely one of my favorite events. The top image on this post was used in the final presentation and the author told me he is aware of the AI errors in there, “of which there are at least two”. Why is there no UTF8 in URIs? “If we would do it again, we would have allowed UTF8 in there” was said by someone who was there in the mid 1990s… Optimistic DNS is a draft. Use stale DNS cache data while getting the new. Connection remains alive for 120 seconds while DNS data is often not cached for even 30 seconds. No one in the room seemed to hate it. Let’s do this! The journey to QUERY. One of the primary authors of the RFC took us through what it took to make it happen. It was sixteen years since the most previous registered HTTP method and maybe this was the last one ever?

Backend API Performance

▲ 0

0 views

daniel.haxx.se 3 months ago

Workshop Basel day one

On this hot summer’s day in Basel, Switzerland, the seventh HTTP workshop started. These events tend to work roughly the same way and the people in the room are also to large extent familiar and known since previous editions. Forty people in a meeting room, where we take turns in doing short talks on HTTP and networking topics, with the following question and discussion session. The rules for the meetings are explicitly Chatham rules, which means that everything I write about the meeting will be sufficiently fuzzy and without many company or personal names. This is not the kind of meeting that can be easily summed up in a short blog post anyway. You really should be here. Present in the room were representatives from all the world’s most prominent and used HTTP deployments: clients, browsers, CDNs, proxies and servers. I’m happy to say that there were also several first-timers. We like fresh blood. (If you think I’m being overly brief or vague about specifics in this post; that is partially on purpose but primarily because I’m a lousy note-taker and mostly write this up after a busy day that also may have involved beer.) After a round of introductions, we started. REST is a set of constraints, and in this presentation it was argued that it can or maybe even should be extended to do more. A number of recent applications like Mastodon/ActivityPub, Bluesky/AT, Matrix, Nostr, IndieWeb, all currently use HTTP to do state synchronization but they all do it differently in their own unique ways. Can REST and maybe HTTP be adjusted to help this for improved interoperability? Looking at the Common Crawl data and comparing data over time, it was observed that responses use the Last-Modified header field more now than they did in the past, and there were great follow-up speculations on why this is so. Data also shows that a large share of these headers present dates that are almost identical to the time the requests were issued. With the cc-lint tool , data was gathered on how HTTP is actually used today, proving that there is work to be done: deprecated headers are used, some headers are done wrong, and many are overly big. This indicates that there are well used both servers and clients out there that would benefit from cleanup. It probably also shows that doing HTTP correctly and all the correct headers is far from an easy task. Another presentation showed data, this time from a well-known CDN, on the impact the existing AI scraper bots have on the Internet from their point of view. It showed that roughly half of the requests and half of the bandwidth are spent by scraper bots. A long discussion followed where the numbers were questioned as maybe the numbers look like this because a sufficiently large number of the “bad AI scrapers” appear as regular users to the classifiers. Speculations of different kinds were made.  As a follow-up from a presentation from a previous HTTP workshop we got to learn how the journey on developing their new HTTP stack has progressed and several fun adventures and lessons from that were shared with the audience. A look into new HTTP API development at Apple . Some discussions and lessons learned from creating new APIs for both servers and clients. We got an excellent walk-through of some details and internals of the Android networking stack. Emphasis was perhaps especially put on ECH and QUIC connection migration, and the final “don’t tell us when your connection closed” led to a long new discussion on how we really should fix the problem: when connection has been left idle for a long time and it is closed by the server, the client (mobile phones) don’t want to be told. This, because getting that RST and more, just wakes up the radio and more on the phone only to tell it to go back to sleep. It was theorized that if we could get rid of this unnecessary battery waste, the accumulated gain across billions of devices would make a serious dent. Several additional HTTP related problems were of course also subsequently solved as we then wandered into the city for dinner and maybe a beer. Of course yours truly returned back to his hotel room in good time to be able to write up this blog post. The best part of these workshops might be the (no pun intended) networking and discussions had completely outside of the agenda. End of day one. Two more to come,

Web Development Data Analysis API

▲ 0

0 views

Steve Klabnik 3 months ago

Too many words about DIDs

Your “Bluesky account” is not just a Bluesky account: it is an account that can be used with a variety of other applications. This post is going to be an exploration of part of what that means from a technical perspective, so if you’re not a software developer, this post isn’t for you. But what I’m going to explain is the technical mechanism for how your account works separate from Bluesky, and in fact, separate from any particular app. Let’s talk about identity: who are you, anyway? Users of a system need some sort of way to describe who they are to use it. If you want to log in, you need to present who you are. If you want to make a post, well, we need to know who the author of that post is. For atproto, the protocol that underlies Bluesky and other apps in the ATmosphere, they use the “Decentralized Identity” standard, also known as DID. The W3C standardized DIDs in 2022 . As you might guess from the name, DIDs are, an identifier that you can use as the basis of identity for building applications. And the idea is that these identifiers are decentralized. However, a lot of people have a lot of feelings about that specific word, and often accuse atproto of not being properly decentralized. We’re going to go over the details so you can understand how this works, and you can decide for yourself if this approach suits you or not. Here is my DID, we’ll use this as an example: There are three parts, separated by colons: The scheme ( ), the method ( ), and the DID method-specific identifier ( ). To use a DID, such as , you resolve it into a DID Document A set of data describing the DID subject, including mechanisms, such as cryptographic public keys, that the DID subject or a DID delegate can use to authenticate itself and prove its association with the DID. That document contains various properties that describe the identity. Here’s my DID Document, at the time of writing: This document gives you everything you need to know to determine who I am, that is, given an arbitrary post that claims it’s written by me, this document describes how you’d verify that claim. We’ll get into how to do that that in a moment, but first, how do you resolve that DID into that DID document? Well, it’s pretty easy: each method is a standard that describes how you do that. So when you see , that means we use the PLC standard, which we’ll be going over in a moment. Another method supported by Bluesky is . In that case, you wouldn’t use the PLC standard, you’d use the Web one. This is the sense in which DIDs are decentralized: when you present your identity, you get to decide what method validates that that is a real identity. There’s no centralized authority that determines which DID types are valid. Now, of course, that doesn’t mean that every application supports every DID method, because while this specification is very generic, you’re still going to have to write some code to implement that particular method. I could say “Hey I’m ” and unless your app supports the method, it’s not gonna inherently just know what to do. So that is one important caveat. Let’s explain this resolution process for the method. While supported by Bluesky, a very small number of users actually use , but it’s a simpler method and so I think it’s illustrative to go over first. I’ll be using Liz Fong-Jones account as an example here. Her identity for that account is . So how do we resolve this DID into a DID Document? We take the method-specific identifier, which in this case is , and put it into this URL template: You can then go fetch this URL to resolve it into the DID Document, which at the time of writing, looks like this: This is very simple! So why might we not want to use ? Why bother with any other system? Well, this relies on the DNS system. One could make the argument that ultimately, this is still centralized in some form. If Liz’s domain registrar were to take away her domain, she would also lose control of this DID. In a more generic sense, if Liz decides she wants to not use that domain anymore, she will lose control of that identity to whoever does. That could be through non-malicious means, like letting it expire and someone else purchases it, or through malicious ones, like a hack which would compromise her registrar account and take the domain over. Also, you need to have a web server running on that domain with infinite uptime; if the server goes down, so does your ability to get the document. When this DID document changes, there’s no mechanism for clients to know that it’s changed, which means applications may use one that’s out of date, or that there is lag between updating the document and updating the application built on it, which may cause temporary problems until the latest document is fetched. All of these drawbacks led Bluesky to develop their own DID method, which attempts to fix these problems and others. This method is called . To resolve a , you take the entire DID, and put it in this template: You can then fetch that URL and get the DID document. So… what’s the difference? Well, in this case, both nothing and something. In a very literal sense, both are resolved in the same way: you fetch a URL. However, the details matter. There is already two ways in which this is different than DID:Web: I’ve presented the above as pros, but there are also cons. Before, I had to trust the DNS system and domain registrars, now I have to trust plc.directory. All of the same caveats apply in that sense, I have to trust that they don’t take my ID away from me, or that it doesn’t get stolen, etc. However, there are also some important details that mitigate this, which we’ll get to. But for some people, neither trusting DNS nor trusting plc.directory is acceptable, and there are other DID methods that use, for example, a blockchain to resolve the name. Bluesky does not support using any of those DID methods, so for this application, it’s not really relevant, but it’s important to know that they exist. Why do it this way? Well, the simplest way to put it is this: setting up a involves a lot of “nerd stuff.” You have to register a domain, and that’s also an ongoing monetary cost. You have to know how to set up a web server, and author some JSON to put on that server. You have to keep it running. You have to know how to store your private keys, and keep them safe. It’s a non-starter compared to “sign up for this web app.” And Bluesky’s goals involves making this platform accessible to non-nerds. By having plc.directory manage all of this, we eliminate all of those steps. While drafting this post, I have also been made aware of , which expands on and attempts to rectify some of its shortcomings. I have not read the spec yet, but it has reached 1.0, so it is probably worth checking out. I wanted to get this post shipped last week, and didn’t want to delay it further by adding another section, but if I were writing this post in the future, I’d probably want to talk about it as well, so just a little heads-up there. But it does also mean that, in some sense, Bluesky still owns your identity. They’ve generated a keypair for you, and the have access to the secret key. That’s unacceptable for some people. So how do you fix that? Well, has some additional features that does not. For example, will allow you to register additional keypairs with your ID and use them to rotate your signing keys. This allows you to remove the Bluesky generated keys and insert your own. While that is true, it’s also the case that your PDS needs to use your keys to sign your posts. As such, most people are likely to store their keys in their PDS, and so if you are using a Bluesky managed PDS, well, you’ve uploaded your keys to their infrastructure, and that’s probably not acceptable if you’re trying to keep your identity away from Bluesky. Of course, the solution there is to run your own PDS and then rotate your keys. At that point, your key is living on infrastructure you own, and Bluesky has no say over it any more. I think that this possibility is an important design property, and allows motivated users to meaningfully own their identity. A criticism of this boils down to “well, most users won’t do that,” and while that’s true, I also think that’s okay for most people, and that having the choice is more important than forcing every user to deal with their own key management. This is kind of an abrupt end to this post, but I just wanted to get some things down ‘on paper’ as it were. I hope you’ve learned a bit about identity and how it works with atproto. Here’s my post about this post on BlueSky: Too many words about DIDs: steveklabnik.com/writing/too-... Too many words about DIDs Blog post: Too many words about DIDs by Steve Klabnik Your DID is no longer tied to a specific domain name. I can let expire and move to and my stays the same. While a web server still needs to be running, that’s the job of plc.directory, not my own job. This is operationally much simpler.

API Crypto Programming JSON

▲ 0

0 views

Xe Iaso 4 months ago

Why are cached input tokens cheaper with AI services?

When you see AI model pricing pages, you usually see things broken down like this: Source: DeepSeek API Docs If you manage to have most of your input tokens be cached, you save a huge amount, in this case $0.20 per million tokens. What does this mean though? What does caching do that makes you save so much, in some cases upwards of tens of kilodollars? Someone explain the cached vs not thing to me for how this is $10,000 worth of savings lol [image or embed] — Chimney Sweepers Local 420 FKA yburyug ( @bobbby.online June 12, 2026 at 12:39 AM Warning I'm gonna be totally honest, I barely understand the basic outline of the math involved here. Where possible I am to not be completely wrong here, but I'm not going to emit something 1:1 accurate with the mathematical truth of large language models' inner workings. Bear with me. When you make an API call to large language model services, you make an API call like the following: That element is the key bit. Every time you accumulate messages from the initial system prompt, initial user request, AI responses and any tool use requests/responses, you add to that array and make it grow bigger and bigger. A good way to think about this is that sending a conversation to a large language model is like having a pair of people share a roll of paper on two different typewriters. Every time you finish your message, you send the roll of paper back to the AI model and it has to re-read through the entire conversation in order to start typing on the end with its response. As the conversation gets longer, this gets more and more expensive because the model has to recalculate its internal state all over again for every additional message. However, large language model inference is complicated but deterministic . Given the same inputs, you will always get the same output. This means that you can use a technique called key-value caching (KV caching) in order to save that intermediate state and use it for next time. Most of the time this cache is a prefix cache because that allows you to just add on more messages to the end of the request pretty easily and be fine. Imagine something like this: If the model has already processed the question about the sky being blue and generated the response about Rayleigh scattering, it doesn't need to process both of those messages again to answer the user's question about sunsets. In production AI model deployments you would put that generated intermediate state into the KV cache so that the model doesn't need to run twice for the same data. This saves time and effort on the side of the AI model provider, and currently model providers decide to pass that savings onto API users in the form of cheaper inference costs for cached lookups. As you develop an application with AI in it, try to avoid changing any inference settings or previous messages between prompts. This makes your application's queries much more likely to read from the cache, making it faster, reducing the environmental impact, and saving you(r users) money.

AI API

▲ 0

0 views

Jack Vanlightly 4 months ago

Kafka Share Groups and Parallelizing Consumption - Part 3: Client-local parallelism

All tests were executed against Kafka 4.3.0 using Dimster.  In the last post Broker-Visible vs Client-Local Parallelism we looked at two ways of scaling Kafka consumption. The final unit of parallelism can be visible to the broker, as consumers, or it can be local to the client, as threads, virtual threads, async tasks, or some other execution mechanism hidden behind a smaller number of consumers.  Broker-visible parallelism is simple to reason about: if each consumer processes records serially, we add more consumers to increase parallelism. But each consumer adds overhead to the brokers: broker-side protocol state, TCP connections, group membership, fetch state, and participation in the consumer or share group protocol. With long processing times and/or high throughput, the required number of parallel workers can easily exceed what is practical to model as broker-visible consumers. That is where client-local parallelism becomes important. Instead of scaling by adding more consumers, each consumer application can poll records and process them concurrently inside the client. This allows a smaller number of Kafka consumers to drive a much larger amount of parallel work. In this post, we’ll compare client-local parallelism with consumer groups and share groups using the Apache Kafka clients, by way of Dimster, the benchmarking tool used throughout this series. Dimster uses the official Apache Kafka clients under the hood. The main comparison is between two styles of client-local parallelism: blocking and continuous styles. At the API level, applications obtain records by calling , then later record their progress by committing offsets or acknowledging records. Under the covers, the client sends fetch requests and commit/acknowledgment requests to the brokers. There is some indirection between API calls and network requests, but every parallel processing style has to fit into this general poll/commit cycle. Consumer-group consumers commit offsets (one offset per partition) whereas share-group consumers commit a set of per-record acknowledgments. Any parallel processing style must fit into this fetch/commit style. We can classify parallel processing within this fetch/commit style into two main methods: Blocking and Continuous Poll -> kick off parallel processing -> block on completion of all -> commit -> (repeat) There are many implementation options for decoupling polling, processing and commits, but the general pattern can be classified into two main mechanisms: Poll -> Dispatch loop : each record submitted for background processing. Keep polling independently of processing (though implement backpressure by limiting the number of inflight records, i.e. stop polling when your processing buffer is full). Accumulate -> Commit loop : Accumulate completed records to commit opportunistically. If you are rolling your own logic using the Apache Kafka clients (rather than choosing a parallel processing library), blocking is by far the simplest to code, but the most inferior in terms of performance profile. The Kafka clients support both blocking and continuous styles with consumer groups, but only blocking with share groups . Why don’t share groups support continuous mode with the Apache Kafka clients? It’s simple. You can only acknowledge a record from the current poll batch. If you try and acknowledge a record from the previous poll, it throws an exception. This may or may not change in the future, but it’s worth knowing this if you were planning on implementing a continuous parallel processing style with the AK clients and share groups. Dimster supports parallel processing simulation with both blocking and continuous mode, but share groups only support blocking mode. Dimster doesn’t actually process records (except for recording metrics), instead it simulates processing time by calculating how long each record would take, based on randomized processing times between the min/max. In Blocking mode , it figures out how long the processing would take to process a poll batch (based on the level of parallelism requested in the workload file) and performs one sleep per poll-loop-iteration for the aggregate processing time. In Continuous mode , it feeds each record, along with its randomized processing time, into an in-memory delay queue (accounting for how much parallel processing is requested). Separately it polls, drains completed records from the delay queue and commits continuously. Let’s run some benchmarks with Dimster in blocking vs continuous modes. We’ll use an example workload with: A long processing time of 1-5 seconds (3 second average) A moderate rate of 1,000 records a second.  Each consumer application is capable of processing around 300 records concurrently. The aggregate parallelism is 3,000 ( ) which puts us in the territory where serial consumers are not a great choice. Firstly, share groups only allow groups of up to 1000 members, and regardless, 3000 consumers would create more than 9000 TCP connections (in a three broker cluster), which is excessive for one use case of this size. We need to parallel process inside the clients. We’ll run 3 tests: Consumer group blocking style Consumer group continuous style Share group blocking style The workload file (single scenario with three test points): In the test analysis we’ll cover the configurations in this workload. All three tests resulted in the same 1000 records/s throughput. But end-to-end latency differed a lot, with consumer group continuous style easily winning. The latency distributions: The latency distribution of only cg-continous: Continuous is the clear winner here. You can download the Dimster result tarball here . Let’s dig into the results and why the workload was configured the way it was. With blocking mode, consumption is a factor of the poll rate and the number of records per poll: The poll rate is determined by how long the application blocks waiting for all records to complete. The number of records per poll is bounded by (though it is a soft cap). When estimating the poll rate in blocking mode, the average processing time is the wrong choice as we’ll really block for the longest processing time of a poll batch (not the average). With 100s of records per poll, we’ll likely hit or get close to the upper bound 5 seconds (assuming uniform distribution). More likely in the real world, we’d see a non-uniform distribution where, for example, p95 might be 500ms, but p99.9 be significantly higher at say, 5 seconds.  The workload we’re using has a rate of 1,000 records/s. Each consumer is capable of processing around 300 records concurrently so we set . With a poll rate of 0.2 (one every five seconds), the consumption throughput per consumer is 60 records/s. To reach 1000 records per second we need at least 17 consumers (and partitions), so I configured 18. The effective workload of test point 1 : The consumers managed the 1000 record/s but for some reason, the max end-to-end latency (processing-start-timestamp - publish-timestamp) was double the worst processing time. It turns out that this is a natural effect of Blocking mode with consumer groups. The highest e2e latency will be at least the blocking time of the previous poll iteration (as records kept arriving in the partition throughout the blocking time). However, you may note that the above e2e latency numbers are p50 is 7.5s and max is 10s. This can occur in blocking mode due to the way polls return buffered records and trigger an asynchronous fetch (pre-fetch) to fill the buffer before the next poll. Think of the consumer as having a two-step delivery path: first Kafka records are fetched asynchronously into the consumer’s internal buffer, and poll() returns buffered records to the application. In the above diagram, we see the application spending 5 seconds processing a batch it just received, but that batch had already spent 5s in the buffer as it was filled by a fetch triggered by the previous poll (5s before this one). This kind of e2e latency might not be a big deal, considering the long processing times. If we want to lower the e2e latency significantly, then we need the continuous style. With continuous style, we have decoupled polling from processing and we can use the average processing time of 3 seconds to calculate the consumption rate per consumer (we are not constrained by the max processing time). Parallelism is not defined by the number of records per poll but the total inflight capacity of parallel work (threads, virtual threads, async tasks). We can feed that capacity with a constant stream of small polls and stop polling once that capacity has been reached (polling again once there is free capacity again). Because the application can poll at a high frequency, buffered records remain in the buffer for only a few milliseconds before being submitted for parallel processing.  In continuous mode, worker throughput is approximately: In Dimster the inflight capacity is set by the workload field . With a capacity of 300 and average processing time of 3s, each consumer can process 100 records/s. To reach 1000 records per second, we need 10 consumers and partitions. I set the capacity to 400 to add some wiggle room. The effective workload of test point 2 : This time we see that e2e latency remains very low, as we don’t block on the longest processing time. Again with the Blocking style, so: The per-consumer poll rate is determined by the highest processing time per batch. The aggregate parallelism is 5000 ( ) Share groups introduce a new constraint, the per-partition inflight budget (aka ). The aggregate inflight budget must exceed the aggregate parallelism of 5000. The default is 2000 per partition and so just three partitions gives us an aggregate inflight budget of 6000. Another difference is that we can have multiple share consumers per partition. If we use then each partition needs parallelism of 1666 ( ). With , we need 6 consumers per partition to cover it. The effective workload of test point 3 : You might expect end-to-end latency to be lower than the blocking consumer group test and you’d be right! Each partition has six consumers so the time period between fetches is lower (records spend less time in the partition before being fetched). We are also using so there is no pre-fetching which inflated the e2e latency in the consumer group test. But it’s still higher than you might expect. Per-partition, we have 6 consumers with a fetch/poll rate of 1.2 per second and 333 records arrive per second. We might expect the worst e2e latency to be 277 ms (333 / 1.2). So what’s going on? The above calculation assumes each fetch arrives evenly spread over time. But fetches cluster to a greater or lesser degree, there is no coordination between the consumers. If a long period passes with no fetches, then the first fetch that arrives can drain the accumulated lag, and subsequent fetches just return the handful of records that arrived since the prior fetch a few milliseconds before. The only way for the consumers in this workload to reach the 1000 records/s is if each fetch returns around 277 records per fetch on average. With fetch request clustering, the only way fetches can be filled to this extent is if lag has built up. If 6 consumers attempt to fetch 300 records at exactly the same time, only if lag has reached 1800 on that partition will all those fetches return full. So the consumers settle into a stable amount of lag that is high enough such that fetches return with enough records to keep up. If consumers catch up and lag goes to 0, consumption throughput will naturally drop down until lag builds up to allow for the full consumption rate. Client-local parallelism is often the only practical way to handle long record processing times. But how that client-local parallelism is implemented in Kafka fetch/commit cycle has a big impact on latency. A blocking poll → process → commit loop is simple, but it couples consumption progress to the slowest record in each batch, which lowers poll frequency and can inflate e2e latency even when there is plenty of processing capacity. Continuous polling decouples polling, processing, and committing, allowing the client to keep records flowing into a processing pool while applying backpressure through an in-flight limit. For consumer groups, this provides much better latency and usually requires many fewer consumers and partitions for the same workload. Share groups improve the broker-visible side of the problem by allowing multiple consumers per partition, but the current Apache Kafka clients still constrain client-local parallelism to the blocking style. If your goal is highly parallel, low-latency processing, consumer groups remain the better fit. Removing the same-batch acknowledgment constraint from the Kafka share consumer would make that style possible with share groups as well. In the next post, I’ll look at some pathological share-group workloads with some gotchas to watch out for. Poll -> Dispatch loop : each record submitted for background processing. Keep polling independently of processing (though implement backpressure by limiting the number of inflight records, i.e. stop polling when your processing buffer is full). Accumulate -> Commit loop : Accumulate completed records to commit opportunistically. In Blocking mode , it figures out how long the processing would take to process a poll batch (based on the level of parallelism requested in the workload file) and performs one sleep per poll-loop-iteration for the aggregate processing time. In Continuous mode , it feeds each record, along with its randomized processing time, into an in-memory delay queue (accounting for how much parallel processing is requested). Separately it polls, drains completed records from the delay queue and commits continuously. A long processing time of 1-5 seconds (3 second average) A moderate rate of 1,000 records a second.  Each consumer application is capable of processing around 300 records concurrently. Consumer group blocking style Consumer group continuous style Share group blocking style The per-consumer poll rate is determined by the highest processing time per batch. The aggregate parallelism is 5000 ( )

Performance Backend API

▲ 0

0 views

Simon Willison 4 months ago

Initial impressions of Claude Fable 5

I didn't have early access to today's Claude Fable 5 release, but I've spent the past ~5.5 hours putting it through its paces. My initial impressions are that this is something of a beast . It's slow, expensive and has been quite happily churning through everything I've thrown at it so far. As is frequently the case with current frontier models the challenge is finding tasks that it can't do. First, let's review the key characteristics. Anthropic claim that Claude Fable 5 offers the same performance as Claude Mythos 5, except with much more strict guardrails in place to prevent it being used for harmful things. Those guardrails trigger often enough that the Claude API has new mechanisms for letting you know when you hit them, and even has a new option to request it falls back to another model automatically if something gets rejected. Claude Mythos 5 is out today as well, Anthropic say it "Shares Claude Fable 5's capabilities without the safety classifiers". The models have a 1 million token context window, 128,000 maximum output tokens and a knowledge cut-off date of January 2026. They are priced at twice the price of Claude Opus 4.5/4.6/4.7/4.8: $10/million input tokens and $50/million output tokens. There's no increase in price for longer context usage. Other than that the upgrade guide is substantially thinner than the similar guide for Opus 4.8 . The best way to describe Fable is that it feels big . Not just in terms of speed and cost, but also in how much it knows. Here's an example prompt, which I used to compare Fable's knowledge with Opus 4.8 (typo included): Opus 4.8's reply started like this : I don't have a reliable, comprehensive, and date-verified list of all of Simon Willison's open source projects, and I want to be honest about that rather than risk giving you inaccurate dates or fabricated entries. Here's what I can tell you with reasonable confidence: Well-known projects by Simon Willison: LLM (~2023) – A command-line tool and Python library for interacting with large language models. Datasette (~2017) – A tool for exploring and publishing data, probably his most prominent recent project. sqlite-utils (~2018-2019) – A Python library and CLI tool for manipulating SQLite databases, often used alongside Datasette. Django (2005) – He is one of the co-creators of the Django web framework, along with Adrian Holovaty, originally developed at the Lawrence Journal-World newspaper. He also maintains a large ecosystem of Datasette plugins and many smaller utilities (often with names like , various tools for importing data, etc.). That's not bad. Here's the start of the response for the exact same prompt from Fable 5 - both models were running without access to search: I think you mean Simon Willison — the prolific open source developer, co-creator of Django, and creator of Datasette. An important caveat upfront: a truly complete list isn't feasible. Simon Willison is famously prolific — he has published hundreds of repositories (Datasette alone has 100+ plugins, many written by him), and my knowledge has a cutoff date, so I'll be missing his most recent work. Below are his notable projects with approximate initial release dates, most recent first: (Here's GPT-5.5 for good measure . It listed even more projects than Fable did!) In the past I've stated that I don't care about how much models know - I want them to be able to manipulate text and code in useful ways and actively look up the information they need via search tools, not bake it into their weights. But knowledge like this is a reasonably good proxy for model size - you can cram a whole lot more details about the world into a larger number of parameters. Does knowing more stuff mean the model is better at the tasks we pose to it? I can certainly imagine how a coding model with deeper knowledge of modern libraries and patterns could crunch through coding tasks more effectively. Is Fable really bigger than Opus? Anthropic haven't said anything about model size, so all we have are tea-leaves, but the speed, pricing and my own poking at its knowledge make me think that it's a large model. Maybe the largest yet from any vendor. Anthropic made Fable 5 available across all of their surfaces - the Claude.ai chat interface, Claude Code for web, Claude Code CLI and Claude Cowork as well. The model is available "until June 22nd" on the subscription plans (I'm on $100/month Max at the moment), after which it will be billed extra. Claude.ai is often under-estimated. Since September 2025 every chat has had access to a full container environment to run code, including the ability to install additional packages and even clone repositories directly from GitHub. Last week I released micropython-wasm , a Python library that uses wasmtime to run a custom build of MicroPython in WebAssembly to act as a sandbox for untrusted Python code. I decided to see if Fable could upgrade that to running full Python instead. I started with this prompt: Fable identified that it could use Brett Cannon's cpython-wasi-build builds for this, but was unable to download them itself due to environment restrictions. So I grabbed the two zip files from that page and uploaded them to Claude: ( , as attachments) And that was that. It churned away for a few minutes and got the entire thing working. Part of the response included: I tried the cleaner single-zip-stdlib approach to shrink the filesystem surface, but CPython's bootstrap fails to find from inside a zip without more prefix finessing — the directory-preopen approach works reliably, so that's what the PoC uses. The zip path is solvable but needs /frozen-getpath work. Then a little later: ... and it gave me this 13.9MB cpython_wasm-0.1.0-py3-none-any.whl file. You can try running Python code in a sandbox using that wheel URL and like this: Here's the full chat transcript . This was a very strong start. Before I'd realized it was Fable day, my stretch goal for today was to add a new feature to Datasette Agent : I wanted tool calls within that agent software to gain the ability to pause mid-execution and request approval directly from the user. This felt like a suitably meaty task to throw at the new model. Over the course of the day Fable not only solved that problem , it also identified and then implemented four issues in my underlying LLM library that would help support this kind of advanced pause-resume mechanism in tool calls. It got everything working first using somewhat gnarly hacks, but the moment I told it that changes to LLM itself were in scope it set to work unraveling the hacks and turning them into supported features of LLM instead. My stretch goal turned into LLM 0.32a3 , almost entirely written by Fable. Here are the release notes: Driven by the needs of Datasette Agent 's human-in-the-loop feature, made the following improvements to how tool calls work: I'm really impressed with the quality of API design, tests, code and documentation that Fable put together for this. I spent several hours on it today, but it feels like several days' worth of work. I recently started using AgentsView to help track my local LLM usage across all of the different coding agents. I published a TIL today about adding custom Fable pricing to that tool, which I expect will not be necessary in the very near future. After setting the price, I ran this command to start a localhost web server to explore my usage: Here's the treemap showing the breakdown of my Fable usage across various projects today: I used $110.42 worth of tokens today, all as part of my $100/month subscription. I ran "Generate an SVG of a pelican riding a bicycle" against all five thinking effort levels with Fable. Here are the results , including the token cost for each one: It's interesting that high ended up using fewer tokens than medium for this particular run. Here are the Opus 4.8 pelicans for comparison. You are only seeing the long-form articles from my blog. Subscribe to /atom/everything/ to get all of my posts, or take a look at my other subscription options . LLM (~2023) – A command-line tool and Python library for interacting with large language models. Datasette (~2017) – A tool for exploring and publishing data, probably his most prominent recent project. sqlite-utils (~2018-2019) – A Python library and CLI tool for manipulating SQLite databases, often used alongside Datasette. Django (2005) – He is one of the co-creators of the Django web framework, along with Adrian Holovaty, originally developed at the Lawrence Journal-World newspaper. files-to-prompt (April 2024) – concatenates files into a single prompt for LLMs datasette-extract (2024) – plugin for extracting structured data using LLMs LLM (May–June 2023) – CLI tool and Python library for interacting with large language models, plus a large ecosystem of plugins (llm-gpt4all, llm-mistral, llm-claude, etc., 2023 onward) symbex (June 2023) – search Python code for symbols ttok and strip-tags (May 2023) – CLI tools for token counting and HTML cleanup for LLM pipelines datasette-lite (May 2022) – Datasette running entirely in the browser via WebAssembly/Pyodide shot-scraper (March 2022) – automated website screenshots via Playwright s3-credentials (November 2021) – CLI for creating scoped AWS S3 credentials django-sql-dashboard (2021) – SQL reporting dashboards for Django Dogsheep suite (2019) – personal analytics tools: twitter-to-sqlite, github-to-sqlite, healthkit-to-sqlite, dogsheep-beta, etc. sqlite-utils (2018) – CLI and Python library for manipulating SQLite databases Datasette (November 2017) – his flagship project; tool for exploring and publishing data csvs-to-sqlite (2017) – convert CSV files to SQLite Various early tools (~2007–2010) – soupselect, json-head, geocoders, and others Django (developed 2003–2005, open-sourced July 2005) – co-created with Adrian Holovaty at the Lawrence Journal-World Tool implementations can declare a parameter named in order to be passed the object for the current invocation. This allows them to access the current . See Accessing the tool call from inside a tool . #1480 Every tool call is now guaranteed a unique - providers that do not supply one get a synthesized -prefixed ULID. #1481 Tools can raise a exception to cleanly pause the tool chain, useful for things like waiting for human approval. The exception propagates to the caller with and (completed sibling results) attached, and no model call is made with a placeholder result. See Pausing a chain from inside a tool . #1482 Failure semantics for concurrent tool execution: async sibling tool calls always run to completion before a pause or hook exception propagates. #1482 Chains can now resume from a history ending in unresolved tool calls: the calls are executed through the normal / machinery before the first model call, skipping any that already have results. The method also accepts a new optional argument for executing an explicit list of objects in place of the calls requested by the response. See Resuming a chain with pending tool calls . #1482 Fixed a bug where the async tool executor silently dropped calls to tools not present in - these now return results, matching the sync executor. #1483

Python JSON API AI HTML

▲ 0

0 views

Xe Iaso 4 months ago

Giving your Go apps Tigris superpowers

Tigris is S3-compatible, which means you can point the AWS SDK at it and most things just work. The catch is that the Tigris-exclusive features—bucket forking, snapshots, object renaming, and the like—need verbose workarounds because the AWS SDK doesn't know they exist. So we wrote a Go SDK that does. It comes in two flavors: the package is a drop-in replacement for the standard S3 client with first-class methods for the Tigris-specific operations, and is a higher-level client for the common single-bucket case that infers its configuration from the environment so you stop passing the same parameters over and over. You can adopt the Tigris features incrementally without refactoring your existing S3 code, and the simpler API still works against other S3-compatible providers. I wrote up how it works and why we built it over on the Tigris blog.

API Programming Cloud

▲ 0

0 views

Aran Wilkinson 4 months ago

Introducing Headcode: A Unified API for UK Rail Data

Headcode is a unified, developer-friendly JSON API that takes the fragmented, legacy feeds of the UK rail network and turns them into clean, enriched real-time data.

Data API JSON

▲ 0

0 views

Simon Willison 4 months ago

Gemini 3.5 Flash: more expensive, but Google plan to use it for everything

Today at Google I/O, Google released Gemini 3.5 Flash . This one skipped the modifier and went straight to general availability, and Google appear to be using it for a whole lot of their key products: 3.5 Flash is available today to billions of people globally: As usual with Gemini, the most interesting details are tucked away in the What's new in Gemini 3.5 Flash developer documentation. It mostly has the same set of platform features as the previous Gemini 3.x series, albeit with no computer use . The model ID is . The knowledge cut-off is January 2025, and it supports 1,048,576 input tokens and 65,536 maximum output tokens. Google are also pushing a new Interactions API , currently in beta, which looks to me like their version of the patterns introduced by OpenAI Responses - in particular server-side history management. Gemini 3.5 Flash is accompanied by a notable price bump. The previous models in the "Flash" family were Gemini 3 Flash Preview and Gemini 3.1 Flash-Lite . The new 3.5 Flash is 3x the price of 3 Flash Preview and 6x the price of 3.1 Flash-Lite (see price comparison here ). At $1.50/million input and $9/million output it's getting close in price to Google's Gemini 3.1 Pro, which is $2 and $12. The Gemini team promise that 3.5 Pro will roll out "next month" - presumably at an even higher price. This fits a trend: OpenAI's GPT-5.5 was 2x the price of GPT-5.4, and Claude Opus 4.7 is around 1.46x the price of 4.6 when you take the new tokenizer into account . Given the price increase it's interesting to see Google roll it out for so many of their own free-to-consumer products. It feels like all three of the major AI labs are starting to probe the price tolerance of their API customers. Artificial Analysis publish the cost to run their proprietary benchmark against models, which is a useful way to take things like tokenization and increased volume of reasoning tokens into account. Some numbers worth comparing: Running the benchmark for 3.5 Flash (high) cost significantly more than 3.1 Pro Preview! Here are some numbers from other vendors: I ran "Generate an SVG of a pelican riding a bicycle" against the Gemini API and got back this pelican, which is a lot : From the code comments: hedgehog on Hacker News : That pelican looks like it's in Miami for a crypto conference. That one cost me 11 input tokens and 14,403 output tokens, for a total cost of just under 13 cents . You are only seeing the long-form articles from my blog. Subscribe to /atom/everything/ to get all of my posts, or take a look at my other subscription options . For everyone via the Gemini app and AI Mode in Google Search For developers in our agent-first development platform Google Antigravity and Gemini API in Google AI Studio and Android Studio For enterprises in Gemini Enterprise Agent Platform and Gemini Enterprise. Gemini 3.5 Flash (high) : $1,551.60 Gemini 3.1 Pro Preview : $892.28 Gemini 3 Flash Preview (Reasoning) : $278.26 Gemini 3.1 Flash-Lite Preview : $93.60 Claude Opus 4.7 (Adaptive Reasoning, Max Effort) : $5,117.14 Claude Opus 4.7 (Non-reasoning, High Effort) : $1,217.23 GPT-5.5 (xhigh) : $3,357.00 GPT-5.5 (medium) : $1,199.14

AI API

▲ 0

1 views

Unsung 5 months ago

“There seems to be a file that is just filled with undecipherable Morse.”

On April Fools in 2021, the popular xkcd comic ran Checkbox , which was a Morse code puzzle in disguise. (It’s interesting to see the community trying to figure out what it actually does .) Engineer Max Goodhart built the front-end and wrote a summary of the whole project : This year was a doozy. We specced and scrapped several different ideas in the months leading up to today. We finally settled on today’s concept just 3 days ago. The need to do something simple was a really useful constraint, and we leaned into the idea of making something primitive but deep. The team seems to have had a lot of fun with it, including even JavaScript being encoded in Morse Code (the link in the blog post no longer works, but you can still see it on the Internet Archive ). Goodhart also wrote about the immense challenge of adjusting the Morse tapping speed to the user, which counterintuitively ended up needing… adjusting the user to the speed. But the best part is that the server communications used the Morse code in URLs, as well: We took great pains to make the API for this project use morse code in the transport. If you take a look at the network inspector, you’ll notice that the URLs requested have morse code in them. This worked for every combination of letters imaginable, with two oddly specific exceptions: a solitary E, and a solitary I. I liked this description of what transpired next, which would have made me think I was going insane, too: Then, an even stranger thing happened . I copied and pasted the correct URL into my browser and pressed Enter, and right before my eyes, it deleted the ”.” from the end of the URL and returned a different result. I was delighted to discover an answer here, not only because in retrospect it’s such an obvious thing that was staring us all in the face for decades, but also because it has interesting URL construction consequences. #bugs #encoding #web

JavaScript API Web Development

▲ 0

0 views

Josh Comeau 5 months ago

Scroll-Driven Animations

The new Animation Timeline API allows us to create dynamic scroll animations without any JavaScript! It’s honestly a very lovely API, and in this blog post, we’ll explore some of the super cool things we can do with it.

JavaScript Web Development API

▲ 0

0 views

Simon Willison 5 months ago

A pelican for GPT-5.5 via the semi-official Codex backdoor API

GPT-5.5 is out . It's available in OpenAI Codex and is rolling out to paid ChatGPT subscribers. I've had some preview access and found it to be a fast, effective and highly capable model. As is usually the case these days, it's hard to put into words what's good about it - I ask it to build things and it builds exactly what I ask for! There's one notable omission from today's release - the API: API deployments require different safeguards and we are working closely with partners and customers on the safety and security requirements for serving it at scale. We'll bring GPT‑5.5 and GPT‑5.5 Pro to the API very soon. When I run my pelican benchmark I always prefer to use an API, to avoid hidden system prompts in ChatGPT or other agent harnesses from impacting the results. One of the ongoing tension points in the AI world over the past few months has concerned how agent harnesses like OpenClaw and Pi interact with the APIs provided by the big providers. Both OpenAI and Anthropic offer popular monthly subscriptions which provide access to their models at a significant discount to their raw API. OpenClaw integrated directly with this mechanism, and was then blocked from doing so by Anthropic. This kicked off a whole thing. OpenAI - who recently hired OpenClaw creator Peter Steinberger - saw an opportunity for an easy karma win and announced that OpenClaw was welcome to continue integrating with OpenAI's subscriptions via the same mechanism used by their (open source) Codex CLI tool. Does this mean anyone can write code that integrates with OpenAI's Codex-specific APIs to hook into those existing subscriptions? The other day Jeremy Howard asked : Anyone know whether OpenAI officially supports the use of the endpoint that Pi and Opencode (IIUC) uses? It turned out that on March 30th OpenAI's Romain Huet had tweeted : We want people to be able to use Codex, and their ChatGPT subscription, wherever they like! That means in the app, in the terminal, but also in JetBrains, Xcode, OpenCode, Pi, and now Claude Code. That’s why Codex CLI and Codex app server are open source too! 🙂 And Peter Steinberger replied to Jeremy that: OpenAI sub is officially supported. So... I had Claude Code reverse-engineer the openai/codex repo, figure out how authentication tokens were stored and build me llm-openai-via-codex , a new plugin for LLM which picks up your existing Codex subscription and uses it to run prompts! (With hindsight I wish I'd used GPT-5.4 or the GPT-5.5 preview, it would have been funnier. I genuinely considered rewriting the project from scratch using Codex and GPT-5.5 for the sake of the joke, but decided not to spend any more time on this!) Here's how to use it: All existing LLM features should also work - use to attach an image, to start an ongoing chat, to view logged conversations and to try it out with tool support . Let's generate a pelican! Here's what I got back : I've seen better from GPT-5.4 , so I tagged on and tried again : That one took almost four minutes to generate, but I think it's a much better effort. If you compare the SVG code ( default , xhigh ) the one took a very different approach, which is much more CSS-heavy - as demonstrated by those gradients. used 9,322 reasoning tokens where the default used just 39. One of the most notable things about GPT-5.5 is the pricing. Once it goes live in the API it's going to be priced at twice the cost of GPT-5.4 - $5 per 1M input tokens and $30 per 1M output tokens, where 5.4 is $2.5 and $15. GPT-5.5 Pro will be even more: $30 per 1M input tokens and $180 per 1M output tokens. GPT-5.4 will remain available. At half the price of 5.5 this feels like 5.4 is to 5.5 as Claude Sonnet is to Claude Opus. Ethan Mollick has a detailed review of GPT-5.5 where he put it (and GPT-5.5 Pro) through an array of interesting challenges. His verdict: the jagged frontier continues to hold, with GPT-5.5 excellent at some things and challenged by others in a way that remains difficult to predict. You are only seeing the long-form articles from my blog. Subscribe to /atom/everything/ to get all of my posts, or take a look at my other subscription options . Install Codex CLI, buy an OpenAI plan, login to Codex Install LLM: Install the new plugin: Start prompting:

Open Source CSS AI API

▲ 0

0 views

Zak Knill 5 months ago

SSE token streaming is easy, they said

I wrote about AI having ‘durable sessions’ to support async agentic applications, and in the comments everyone said: “Token streaming over SSE is easy” . …so I figured I’d dig into that claim. Agents used to be a thing you talked to synchronously. Now they’re a thing that runs in the background while you work. When you make that change, the transport breaks.

AI API

▲ 0

0 views

Ed Zitron's Where's Your Ed At 5 months ago

News: Anthropic Removes Claude Code From $20-A-Month "Pro" Subscription Plan For New Users (Developing)

In developing news, Anthropic appears to have removed access to AI coding tool Claude Code from its $20-a-month "Pro" accounts. This is likely another cost-cutting move that follows a recent change ( per The Information ) that forced enterprise users to pay on a per-million-token based rate rather than having rate limits that were, based on researchers' findings, often much higher than the cost of the subscription. Previously, users were able to access Claude using their Pro subscriptions via a command-line interface and both the web and desktop Claude apps. Users were, instead of paying on a per-million-token basis, allowed to use their subscription to access Claude Code, but will likely now have to pay for API access. Anthropic's Claude Code support documents ( as recently as this April 10th archived page ) previously read "Using Claude Code with your Pro or Max plan." The page now reads "Using Claude Code with your Max plan." Pricing on Anthropic's website reflects the removal of Claude Code on both mobile and desktop. Some Pro users report that they are still able to access Claude Code via the web app and Command-Line Interface. It is unclear at this time whether this change is retroactive or for new Pro subscribers, or whether Anthropic intends to entirely remove access to Claude Code (without paying for API tokens) from every Pro customer. I have requested a comment from Anthropic, and will update this piece when I receive it, or if Anthropic confirms this move otherwise. If you liked this news hit and want to support my independent reporting and analysis, why not subscribe to my premium newsletter? It’s $70 a year, or $7 a month, and in return you get a weekly newsletter that’s usually anywhere from 5,000 to 18,000 words, including vast, detailed analyses of NVIDIA , Anthropic and OpenAI’s finances , and the AI bubble writ large . I recently put out the timely and important Hater’s Guide To The SaaSpocalypse , another on How AI Isn't Too Big To Fail , a deep (17,500 word) Hater’s Guide To OpenAI , and just last week put out the massive Hater’s Guide To Private Credit . Subscribing to premium is both great value and makes it possible to write these large, deeply-researched free pieces every week.  Anthropic appears to have removed access to Claude Code for its $20-a-month "Pro" Plans. Current Pro users appear to still have access via the Claude web app. Claude Code support documents exclusively refer to accessing Claude Code via "your Max Plan," after previously saying you could access "with your Pro or Max Plan."

Business AI API

▲ 0

0 views