Latest Posts (15 found)
<antirez> 2 months ago

AI is different

Regardless of their flaws, AI systems continue to impress with their ability to replicate certain human skills. Even if imperfect, such systems were a few years ago science fiction. It was not even clear that we were so near to create machines that could understand the human language, write programs, and find bugs in a complex code base: bugs that escaped the code review of a competent programmer. Since LLMs and in general deep models are poorly understood, and even the most prominent experts in the field failed miserably again and again to modulate the expectations (with incredible errors on both sides: of reducing or magnifying what was near to come), it is hard to tell what will come next. But even before the Transformer architecture, we were seeing incredible progress for many years, and so far there is no clear sign that the future will not hold more. After all, a plateau of the current systems is possible and very credible, but it would likely stimulate, at this point, massive research efforts in the next step of architectures. However, if AI avoids plateauing long enough to become significantly more useful and independent of humans, this revolution is going to be very unlike the past ones. Yet the economic markets are reacting as if they were governed by stochastic parrots. Their pattern matching wants that previous technologies booms created more business opportunities, so investors are polarized to think the same will happen with AI. But this is not the only possible outcome. We are not there, yet, but if AI could replace a sizable amount of workers, the economic system will be put to a very hard test. Moreover, companies could be less willing to pay for services that their internal AIs can handle or build from scratch

ai AI
1 views
<antirez> 2 months ago

Coding with LLMs in the summer of 2025 (an update)

Frontier LLMs such as Gemini 2. 5 PRO, with their vast understanding of many topics and their ability to grasp thousands of lines of code in a few seconds, are able to extend and amplify the programmer capabilities. If you are able to describe problems in a clear way and, if you are able to accept the back and forth needed in order to work with LLMs, you can reach incredible results such as: 1. Eliminating bugs you introduced in your code before it ever hits any user: I experienced this with Vector Sets implementation of Redis. I would end eliminating all the bugs eventually, but many were just removed immediately by Gemini / Claude code reviews. 2. Explore faster how a given idea could work, by letting the LLM write the throw away code to test ASAP in order to see if a given solution is actually more performant, if it is good enough, and so forth. 3. Engage in pair-design activities where your instinct, experience, design taste can be mixed with the PhD-level knowledge encoded inside the LLM. In this activity, the LLM will sometimes propose stupid paths, other times incredibly bright ideas: you, the human, are there in order to escape local minimal and mistakes, and exploit the fact your digital friend knows of certain and various things more than any human can. 4. Accelerate your work by writing part of the code under your clear specifications. 5. Work with technologies far from your expertise but contiguous with what you can do (for instance: coding in 68000 assembly for an Amiga demo. ) using LLMs as an extension of specific parts of your mind, for the knowledge you don't have. One and half years ago I wrote a blog post called “LLMs and programming in the first days of 2024”. There, I found LLMs to be already useful, but during these 1. 5 years, the progresses they made completely changed the game. However, in order to leverage their capabilities, humans interacting with LLMs must have certain qualities and follow certain practices

ai AI programming Programming
0 views
<antirez> 4 months ago

Human coders are still better than LLMs

This is a short story of how humans are still so much more capable of LLMs. Note that I'm not anti-AI or alike, you know it if you know me / follow me somewhere. I use LLMs routinely, like I did today, when I want to test my ideas, for code reviews, to understand if there are better approaches than what I had in mind, to explore stuff at the limit of my expertise, and so forth (I wrote a blog post about coding with LLMs almost two years, when it was not exactly cool: I was already using LLMs for coding and never stopped, I'll have to write an update, but that's not the topic of this post). But, still: the current level of AI is useful, great too, but so incredibly behind human intelligence, and I want to remark this as lately it is impossible to have balanced conversations. So, today I was working to Vector Sets for Redis, to fix a complicated bug: during the time I stopped working at Redis my colleagues introduced resistance against corruption RDB and RESTORE payloads, even when the checksum of the data passes. This feature is disabled by default, but provides an enhanced layer of safety for people wanting it. But… there is a but as big as an elephant: In order to make HNSWs fast to save into Redis RDBs and to load back, I serialized the *graph* representation, and not the element-vector pairs, otherwise I would have to re-insert back data into HNSWs, and that would be, like, 100 times slower (. ). So I store all the links the nodes have with other nodes, as integers, and then I resolve them into pointers, it’s a nice trick and works great. But if you mix this and random corruptions of the representation, and the fact that my own twist on HNSWs enforce reciprocal links between nodes (I wrote my own implementation of HNSWs with many useful features, but reciprocal links are needed to enable many of them) then this could happen: 1. We load corrupted data that says A links to B, but B no longer links to A (corrupted node IDs). 2

ai AI
0 views
<antirez> 5 months ago

What I learned during the license switch

Yesterday, it was a very intense day. In Italy it was 1st of May, the workers holiday, so in the morning I went for a 4h walk in the Etna with friends Then at 6PM I was at home to release my blog post about the AGPL license switch, and I started following the comments, feedbacks, private messages, and I learned a few things in the process. 1. Regardless of the different few clauses, that IMHO make a difference, the AGPL vs SSPL main difference is that AGPL is "understood". In general, yesterday for the first time I realized that in licensing there is not just what you can do and can't do, but the degree a given license is understood, tested, adopted, . 2. I was very touched by the words of Simon Willison on the matter (https://simonwillison. net/2025/May/1/redis-is-open-source-again/) because it is very peculiar that different persons, living in different parts of the world, but with a similar age and background in software, feel *so similar* about things. I, too, when was writing Vector Sets, was thinking: I would never use it if it wasn't going to be released under the AGPL (or other open source license I understand). This sentiment, multiplied by a non trivial fraction of the community, makes open source eventually win even in the complex software landscape that there is today. 3. People still care a lot about software distributions. Not that I didn't care, but in the past I burned my fingers with it. I was a very initial Linux user, with SlackWare 3. 1 or something like that. During the years I wrote my device drivers, contributed a few patches to the kernel, during the years Debian had maybe ~10 packages of stuff that I wrote, from hping, to the Visitors web log analyzer, dump1090, Redis, and a few more

career Career
0 views
<antirez> 5 months ago

Redis is open source again

Five months ago, I rejoined Redis and quickly started to talk with my colleagues about a possible switch to the AGPL license, only to discover that there was already an ongoing discussion, a very old one, too. Many people, within the company, had the feeling that the AGPL was a better pick than SSPL, and while eventually Redis switched to the SSPL license, the internal discussion continued. I tried to give more strength to the ongoing pro-AGPL license side. My feeling was that the SSPL, in practical terms, failed to be accepted by the community. The OSI wouldn’t accept it, nor would the software community regard the SSPL as an open license. In little time, I saw the hypothesis getting more and more traction, at all levels within the company hierarchy. I’ll be honest: I truly wanted the code I wrote for the new Vector Sets data type to be released under an open source license. Writing open source software is too rooted in me: I rarely wrote anything else in my career. I’m too old to start now. This may be childish, but I wrote Vector Sets with a huge amount of enthusiasm exactly because I knew Redis (and my new work) was going to be open source again. I understand that the core of our work is to improve Redis, to continue building a good system, useful, simple, able to change with the requirements of the software stack. Yet, returning back to an open source license is the basis for such efforts to be coherent with the Redis project, to be accepted by the user base, and to contribute to a human collective effort that is larger than any single company. So, honestly, while I can’t take credit for the license switch, I hope I contributed a little bit to it, because today I’m happy. I’m happy that Redis is open source software again, under the terms of the AGPLv3 license

open-source Open Source database Database
0 views
<antirez> 6 months ago

Reproducing Hacker News writing style fingerprinting

} At this point, the final script, insert. py, could do all the real work: to apply the Borrows method for each user, create the user style vector, and insert it into Redis. The advantage of pre-processing the files (a slow operation) is that the insertion script could be called more easily with different parameters (especially the number of top words to use) in order to see the different results more promptly, without the need to re-process the Parquet files each time. # How the Burrow method works. In the original post, Christopher wrote that you just need to normalize the frequency of the words usage and apply cosine similarity. Actually the process is a bit more involved. First, let’s ask ourselves, how this method actually works, in its essence. Well, it wants to capture words that each specific user over-uses or under-uses compared to the expected “average” language. To do so, we actually use the following steps (from the Python code). That’s what we do for each of the top words: # Convert to relative frequency rel_freq = frequency / total_words # Standardize using z-score: z = (freq - mean) / stddev mean = word_means. get(word, 0. 0) stddev = word_stddevs. get(word, 1. 0) # Default to 1. 0 to avoid division by zero z_score = (rel_freq - mean) / stddev # Set the z-score directly in the vector at the word's index vector[word_to_index[word]] = z_score So we start by “centering” the frequency the user used a given word, by subtracting the *global* usage frequency for that word. This way, we have a number that describes how much the user under (negative) or over (positive) used such word. But, if you think at it, words that have a much higher variance among usage of different writers are less important, when they change. We want to amplify the signal of words that are under of over used by this user in a much greater way compared to the normal variance of the word

programming Programming python Python
0 views
<antirez> 6 months ago

Vector Sets are part of Redis

So basically with VSETATTR / VGETATTR (and equivalent option to set the JSON attribute directly when adding the item in VADD) you can associate a string to the items you want. Then you can do things like that: > VSIM word_embeddings_int8 ele "banana" FILTER ".len == 3" 1) "yam" 2) "pea" 3) "fig" 4) "rum" 5) "ube" 6) "oat" 7) "nut" 8) "gum" 9) "soy" 10) "pua" The filter expression is not a programming language, is what you could write inside the if() statement of high level programming languages, with &&, ||, all the obvious operators and so forth (but I bet we will add a few more). Well, the details are into the doc, there are also memory usage examples, in depth discussions about specific features, and so forth. I will extend the documentation soon, I hope. For now I really (really!) hope you’ll enjoy Vector Sets. Please, ping me if you find bugs :) Comments

database Database json JSON
0 views
<antirez> 6 months ago

AI is useless, but it is our best bet for the future

I used AI with success 5 minutes ago. Just five minutes ago, I was writing a piece of software and relied on AI for assistance. Yet, here I am, starting this blog post by telling you that artificial intelligence, so far, has proven somewhat useless. How can I make such a statement if AI was just so helpful a moment ago. Actually, there's no contradiction here if we clarify exactly what we mean. Here’s the thing: at this very moment, artificial intelligence can support me significantly. If I'm struggling with complicated code or need to understand an advanced scientific paper on math, I can turn to AI for clarity. It can help me generate an image for a project, make a translation, clean my YouTube transcript. Clearly, it’s practical and beneficial in these everyday tasks. However, except for rare, groundbreaking examples like AlphaFold — Google's AI that significantly advanced our understanding of protein folding — AI has yet to genuinely push forward human knowledge in a fundamental way. Aside from these few exceptional results, AI hasn’t (obviously) yet matched the capabilities of the very best human minds. If an AI system were at the same level as the brightest humans (and not better than that: it's not needed for a first humanity jump) we could deploy millions of such systems to accelerate research dramatically, transforming progress expected to take centuries into developments happening within decades, or decades into years. Yet, if artificial intelligence remains stuck at its current level of development indefinitely (even if with small incremental improvements, enough to fire many translators, programmers, drivers, actors, . ), perhaps it might have been better not to have it at all. I mentioned this during a conference here in Sicily. The thought hadn't crossed my mind until I was asked on stage

ai AI
0 views
<antirez> 7 months ago

Big LLMs weights are a piece of history

By multiple accounts, the web is losing pieces: every year a fraction of old web pages disappear, lost forever. We should regard the Internet Archive as one of the most valuable pieces of modern history; instead, many companies and entities make the chances of the Archive to survive, and accumulate what otherwise will be lost, harder and harder. I understand that the Archive headquarters are located in what used to be a church: well, there is no better way to think of it than as a sacred place. Imagine the long hours spent by old programmers hacking with the Z80 assembly on their Spectrums. All the discussions about the first generation of the Internet. The subcultures that appeared during the 90s. All things that are getting lost, piece by piece. And what about the personal blogs. Pieces of life of single individuals that dumped part of their consciousness on the Internet. Scientific papers and processes that are lost forever as publishers fail, their websites shut down. Early digital art, video games, climate data once published on the Internet and now lost, and many sources of news, as well. This is a known issue and I believe that the obvious approach of trying to preserve everything is going to fail, for practical reasons: a lot of efforts for zero economic gains: the current version of the world is not exactly the best place to make efforts that cost a lot of money and don't pay money. This is why I believe that the LLMs' ability to compress information, even if imprecise, hallucinated, lacking, is better than nothing. DeepSeek V3 is already an available, public lossy compressed view of the Internet, as other very large state of-art models are. This will not bring back all the things we are losing, and we should try hard supporting The Internet Archive and other similar institutions and efforts

ai AI
0 views
<antirez> 8 months ago

Reasoning models are just LLMs

It’s not new, but it’s accelerating. People that used to say that LLMs were a fundamentally flawed way to reach any useful reasoning and, in general, to develop any useful tool with some degree of generality, are starting to shuffle the deck, in the hope to look less wrong. They say: “the progresses we are seeing are due to the fact that models like OpenAI o1 or DeepSeek R1 are not just LLMs”. This is false, and it is important to show their mystification as soon as possible. First, DeepSeek R1 (don’t want to talk about o1 / o3, since it’s a private thing we don’t have access to, but it’s very likely the same) is a pure decoder only autoregressive model. It’s the same next token prediction that was so strongly criticized. There isn’t, in any place of the model, any explicit symbolic reasoning or representation. Moreover, R1 Zero has similar reasoning capabilities of R1 without requiring *any* supervised fine tuning, just generating chain of thoughts, and improving it with a reward function, using reinforcement learning, was enough to learn a stronger form of reasoning. Interestingly enough, part of these capabilities were easily distilled into smaller models via SFT, which brings me to the next point. The other fundamental observation is that the S1 paper shows that you need very few examples (as little as 1000) in order for the model to start being able to build complex reasoning steps and solve non trivial mathematical problems. S1, and R1 Zero, hint that in some way in the pre-training step the models already learned the representations needed in order to perform reasoning, just with the unsupervised next word prediction training target

ai AI machine-learning Machine Learning nlp Nlp
0 views
<antirez> 8 months ago

We are destroying software

We are destroying software by no longer taking complexity into account when adding features or optimizing some dimension. We are destroying software with complex build systems. We are destroying software with an absurd chain of dependencies, making everything bloated and fragile. We are destroying software telling new programmers: “Don’t reinvent the wheel!”. But, reinventing the wheel is how you learn how things work, and is the first step to make new, different wheels. We are destroying software by no longer caring about backward APIs compatibility. We are destroying software pushing for rewrites of things that work. We are destroying software by jumping on every new language, paradigm, and framework. We are destroying software by always underestimating how hard it is to work with existing complex libraries VS creating our stuff. We are destroying software by always thinking that the de-facto standard for XYZ is better than what we can do, tailored specifically for our use case. We are destroying software claiming that code comments are useless. We are destroying software mistaking it for a purely engineering discipline. We are destroying software by making systems that no longer scale down: simple things should be simple to accomplish, in any system. We are destroying software trying to produce code as fast as possible, not as well designed as possible. We are destroying software, and what will be left will no longer give us the joy of hacking. Comments

programming Programming devops DevOps
0 views
<antirez> 10 months ago

From where I left

I’m not the kind of person that develops a strong attachment to their own work. When I decided to leave Redis, about 1620 days ago (~ 4. 44 years), I never looked at the source code, commit messages, or anything related to Redis again. From time to time, when I needed Redis, I just downloaded it and compiled it. I just typed “make” and I was very happy to see that, after many years, building Redis was still so simple. My detachment was not the result of me hating my past work. While in the long run my creative work was less and less important and the “handling the project” activities became more and more substantial — a shift that many programmers are able to do, but that’s not my bread and butter — well, I still enjoyed doing Redis stuff when I left. However, I don’t share the vision that most people at my age (I’m 47 now) have: that they are still young. I wanted to do new stuff, especially writing. I wanted to stay more with my family and help my relatives. I definitely needed a break. However, during the “writing years” (I’m still writing, by the way), I often returned to coding, as a way to take breaks from intense writing sessions (writing is the only mental activity I found to be a great deal more taxing than coding): I did a few embedded projects; played more with neural networks; built Telegram bots: a bit of everything. Hacking randomly was cool but, in the long run, my feeling was that I was lacking a real purpose, and every day I started to feel a bigger urgency to be part of the tech world again. At the same time, I saw the Redis community fragmenting, something that was a bit concerning to me, even as an outsider. So I started to think that maybe, after all, I could have a role back in the Redis ecosystem. Perhaps I could be able to reshape the company's attitude towards the community. Maybe I could even help to take back the role of the Redis core as the primary focus of new developments

career Career
0 views
<antirez> 1 years ago

Playing audio files in a Pi Pico without a DAC

The Raspberry Pico is suddenly becoming my preferred chip for embedded development. It is well made, durable hardware, with a ton of features that appear designed with smartness and passion (the state machines driving the GPIOs are a killer feature. ). Its main weakness, the lack of connectivity, is now resolved by the W variant. The data sheet is excellent and documents every aspect of the chip. Moreover, it is well supported by MicroPython (which I’m using a lot), and the C SDK environment is decent, even if full of useless complexities like today fashion demands: a cmake build system that in turn generates a Makefile, files to define this and that (used libraries, debug outputs, …), and in general a huge overkill for the goal of compiling tiny programs for tiny devices. No, it’s worse than that: all this complexity to generate programs for a FIXED hardware with a fixed set of features (if not for the W / non-W variant). Enough with the rant about how much today software sucks, but it must be remembered. One of the cool things one wants to do with an MCU like that, is generating some sound. The most obvious way to do this is using the built-in PWM feature of the chip. The GPIOs can be configured to just alterante between zero and one at the desired frequency, like that: from machine import Pin, PWM pwm = PWM(Pin(1)) pwm. freq(400) pwm. duty_u16(1000) Assuming you connected a piezo to GND and pin 1 of your Pico, you will hear a square wave sound at 400hz of frequency. Now, there are little sounds as terrible to hear as square waves. Maybe we can do better. I’ll skip all the intermediate steps here, like producing a sin wave, and directly jump to playing a wav file. Once you see how to do that, you can easily generate your own other waves (sin, noise, envelops for such waveforms and so forth). Now you are likely asking yourself: how can I generate the complex wave forms to play a wav file, if the Pico can only switch the pin high or low

hardware Hardware tutorial Tutorial programming Programming
0 views
<antirez> 1 years ago

First Token Cutoff LLM sampling

From a theoretical standpoint, the best reply provided by an LLM is obtained by always picking the token associated with the highest probability. This approach makes the LLM output deterministic, which is not a good property for a number of applications. For this reason, in order to balance LLMs creativity while preserving adherence to the context, different sampling algorithms have been proposed in recent years. Today one of the most used ones, more or less the default, is called top-p: it is a form of nucleus sampling where top-scoring tokens are collected up to a total probability sum of “p”, then random weighted sampling is performed. In this blog post I’ll examine why I believe nucleus sampling may not be the best approach, and will show a simple and understandable alternative in order to avoid the issues of nucleus sampling. The algorithm is yet a work in progress, but by publishing it now I hope to stimulate some discussion / hacking. ## There is some gold in the logits Despite the fact that LLM logits are one of the few completely understandable parts of the LLM inner working, I generally see very little interest in studying their features, investigating more advanced sampling methods, detecting and signaling users uncertainty and likely hallucination. Visualizing the probabilities distribution for successive tokens is a simple and practical exercise in order to gain some insights: . ~. In the image we can see the top 32 candidate tokens colored by probability (white = 0, blue = 1), the selected token and the rank of the selected token (highest probability = 0, the previous one = 1, and so forth). In the above example, the Mistral base model knows the birth and death dates of Umberto Eco, so it confidently signals the most likely token with most of the total probability. Other times the model is more perplexed because either there are multiple ways to express the continuation of the text, or because it is not certain about certain facts

ai AI
0 views
<antirez> 1 years ago

Translating blog posts with GPT-4, or: on hope and fear

My usual process for writing blog posts is more or less in two steps: 1. Think about what I want to say for weeks or months. No, I don’t spend weeks focusing on a blog post, the process is exactly reversed: I write blog posts about things that are so important to me to be in my mind for weeks. 2. Then, once enough ideas collapsed together in a decent form, I write the blog post in 30 minutes, often without caring much about the form, and I hit “publish”. This process usually works writing the titles of the sections as I initially just got the big picture of what I want to say, and then filling the empty paragraphs with text. Why I take step 2 so lightly. Because I got other stuff to do, and if blogging would take more than 30/60 minutes I would rather not blog at all, or blog less, or suffer doing it: all things I want to avoid at all costs. Blogging is too important to let it go. It’s better, for me, to give up on the form. At the same time, this is why many of my blog posts, regardless of the content that may be more or less informative, more or less useful, are generally badly written. I hope that the fact I can write well enough in my mother language in some way it is still visibile in my English posts, but I have the feeling that the extremely limited vocabulary I possess, the grammar errors, the sentence construction that oftentimes I just take from Italian and turn into English, all those limits irremediably damage the reading experience. This is why, for the first time, to write my blog post about LLMs and programming I tried a different approach: I wrote the post in Italian and I translated it to English using GPT-4

writing Writing ai AI
0 views