Latest Posts (20 found)
Phil Eaton 5 days ago

I started a software research company

I quit my job at EnterpriseDB hacking on PostgreSQL products last month to start a company researching and writing about software infrastructure. I believe there is space for analysis that is more focused on code than TechCrunch or The Register, more open to covering corporate software development than LWN.net, and (as much as I love some of these folks) less biased than VCs writing about their own investments. I believe that more than ever there is a need for authentic and trustworthy analysis and coverage of the software we depend on. This company, The Consensus , will talk about databases and programming languages and web servers and everything else that is important for experienced developers to understand and think about. It is independent of any software vendor and independent of any particular technology. Some people were surprised (in a positive way) to see me cover MySQL already, for example. But that is exactly the point. I don't want The Consensus to be just "Phil's thoughts". I have already started working with a number of experienced developers who will be writing, and paid to write, for The Consensus. I also hope that this is another way, beyond the many communities I already run, to give back to the community such as in highlighting the work of open-source developers (the first interview with a DataFusion developer is coming soon), and highlighting compelling events and jobs in the software infrastructure world. The Consensus is entirely bootstrapped and will depend on the support of subscribers and, potentially, sponsors . The first few subscribers signed up just this past week. You can read more about the background and goals here , you can read about how contributors will work with The Consensus here , and you can get a sense for where this is going by browsing the homepage of The Consensus already. Thank you for your support in advance! Thank you to the folks who have subscribed already despite very little fanfare. Feedback is very welcome. I'm very excited and having quite a bit of fun already. We're all going to learn a lot. I started a software research company pic.twitter.com/PY3L0yhJlW

c# C# business Business
0 views
Phil Eaton 3 weeks ago

Paths of MySQL, vector search edition

This is an external post of mine. Click here if you are not redirected.

database Database mysql MySQL
0 views
Phil Eaton 1 months ago

LLMs and your career

The most conservative way to build a career as a software developer is 1) to be practical and effective at problem solving but 2) not to treat all existing code as a black box. 1 means that as a conservative developer you should generally use PostgreSQL or MySQL (or whatever existing database), Rails or .NET (or whatever existing framework), and adapt code from Stack Overflow or LLMs. 2 means that you're curious and work over time to better understand how web servers and databases and operating systems and the browser actually work so that you can make better decisions for your own problems as you adapt other people's code and ideas. Zooming out, coding via LLM is not fundamentally different from coding with Rails or coding by perusing Stack Overflow. It's faster and more direct but it's still potentially just a human mindlessly adapting existing code. The people who were only willing to look at existing frameworks and libraries and applications as black boxes were already not the most competitive when it came to finding and retaining work. And on the other hand, the most technically interesting companies always wanted to hire developers who understood fundamentals because they're 1) operating at such a scale that the way the application is written matters or they're 2) building PostgreSQL or MySQL or Rails or .NET or Stack Overflow or LLMs, etc. The march of software has always been to reduce the need for (ever larger sizes of) SMBs (and teams within non-SMBs) to hire developers to solve problems or increase productivity. LLMs are part of that march. That doesn't change that at some point companies (or teams) need to hire developers because the business or its customer base has become too complex or too large. The jobs that were dependent on fundamentals of software aren't going to stop being dependent on fundamentals of software. And if more non-developers are using LLMs it's going to mean all the more stress on tools and applications and systems that rely on fundamentals of software. All of this is to say that if you like doing software development, I don't think interesting software development jobs are going to go away. So keep learning and keep building compilers and databases and operating systems and keep looking for companies that have compiler and database and operating system products, or companies with other sorts of interesting problems where fundamentals matter due to their scale. LLMs and your career pic.twitter.com/lxu1HLF2LC

career Career programming Programming ruby Ruby
1 views
Phil Eaton 2 months ago

Year in community

This year I ran three book club readings over email with 1,230 unique attendees. I ran 12 coffee club meetups in midtown Manhattan with 170 unique attendees. Angelo and I ran 6 NYC Systems meetups with 12 different speakers and 281 unique attendees. I took 3 visiting PhD students out for Banh Mi . I raised $6,915 for educational non-profits, offering chats in return. I got coffee, lunch, or took 30 minute calls with 55 people I'd never spoken to before in person or on video. (Most, but not all, were in return for fundraising receipts.) This list included women and men based in the USA, Germany, Canada, Nigeria, Nepal, India, the United Kingdom, Brazil, New Zealand, Israel, and Australia. (I think I'm forgetting one or two.) Thank you to every person who has been a part of these efforts, making them so special and so valuable. See you in the new year! Year in community pic.twitter.com/n7jrmsZiKN

culture Culture books Books career Career
0 views
Phil Eaton 2 months ago

Year in books

Among the 50 books I read in 2025, I recommend the following 11 non-fiction and 7 fiction works (complete list here ). These were the 18 books that I rated a four or five out of five stars. This is the third or fourth time I've read this book and it has stood the test of time. It's been a few years since I last read it so it was a good reminder that a lot of the things I believe and tell people about writing actually just came from this book. The last 25% is a bit of a slog but nonetheless it remains one of the single books I think every professional should read. 5/5 I really like reading about how writers make their living. I've also been a modest fan of Asimov's works (I've loved what I've read I just haven't read that much). I also love to hear stories of first-generation immigrants to the US and also he lived in New York his whole life so it was quite enjoyable. 4/5 This was 100 years of the evolution of the film industry told basically entirely in disparate interviews edited together. 5/5 I love the history and business of newspapers and media. Moreover it's the guy that Citizen Kane was based on. 4/5 I have never learned about the history of Brazil and I found this introduction enjoyable. 4/5 I loved the retelling of human history focused on Persia (and later Iran). 4/5 I have never read about a supreme court justice before. This was a well-written biography and introduction to the history of law and law education. 4/5 Once again I love reading about newspapers and media and the business and history. This was told by the publisher of The Washington Post. 5/5 I've had this book about Warren Buffett on my shelf for nearly 10 years and finally went through it this year. A delightful and easy read despite the bulk. I only am unhappy that it focused more on family drama than on business decisions. Par for the course with biographies unfortunately. 4/5 This story spanned three or four major wars and a couple of continents. I didn't think I'd be interested in the history of luxury businesses but it has a lot in common with certain modern industries in tech too. You put premiums on relationships and building good faith and so on. 4/5 A history of Coca-Cola over the last 100 years or so. Lessons on how they dealt with competition (Pepsi and Keurig Dr. Pepper) and product revitalization (New Coke, Diet Coke, etc.). Quite an interesting read. 5/5 I got into horror fiction last year (not so much slashers but more just one of the better written categories of genre fiction). This book was one of my two favorite novels of the year. It's a fictional retelling of American history where a Native American becomes a vampire and takes revenge on American colonizers in the American West. 5/5 This was my other favorite novel of the year. I am embarrassed not to have read it before. It's a dystopian story about the USA if all women were required to give birth to deal with a fertility crisis. 5/5 A French woman gets to live forever but everyone she meets forgets her after leaving her presence. An easy and enjoyable read. 4/5 I love a good vampire story, and I love fictional retellings using fantastical horror elements to emphasize atrocities. Vampires employed by the US military help the US in the 1840s take Texas from Mexico. 4/5 This was a cute cozy mystery about British witches forced to hide from society, learning how to accept themselves and develop trust in their community. 4/5 I am likewise embarrassed I have not read this before, nor anything else by Austen. I'm told I haven't rated it highly enough. I will undoubtedly reread it. I loved the wit. It required closer reading than I expected. 4/5 This was a retelling of Adventures of Huckleberry Finn as told by the slave, Jim. If you saw the movie American Fiction a few years ago, it's the same author (of the original book). Everett has very interesting ideas and I look forward to reading more by him. 4/5 My 2025 year in books. 18 to recommend among the 50 I read. pic.twitter.com/mIcbPk7e5x

writing Writing media Media books Books
0 views
Phil Eaton 3 months ago

An individual can change an organization

One of the biggest lessons I learned early in my career was from Drew DeVault at Linode, 10 years ago. He was one of the youngest developers in the company (only I was younger, at 20, at the time) but he cared really strongly about thinking through architecture and code decisions when the culture at the time was, and I love those guys, a little haphazard. Drew had no special position. We all had the same title, "Developer". But he argued so persuasively and so doggedly even when the entire organization seemed against him and somehow he eventually transformed the entire engineering organization. That's supposed to be impossible! It was entirely new to me. That you don't need to wait behind people with more experience to make the right decision. That you can be part of making the right decision if you can find the logic and the will to do it. It isn't that simple of course. Politics is politics. But there are plenty of companies with people who will make a good faith effort to do what makes sense but might, without someone's unasked-for effort, do not what makes sense but what is popular because what's popular just kinda seems easiest. And I always like working for these companies, and for the most part have been able to identify them during the interview process. I learned from Drew to put limited value in seniority. I learned that it's ok to debate. I learned to be prepared and to try to present the facts. I learned to be persistent when I wanted change. I learned that with these skills, it's possible for an individual to redirect the path of an organization. It took a while longer (and me driving one or two people on my team to quit, to my great regret) to learn when to do these things and when to let things go. Still, this lesson from Drew on what's possible always stands out in my memory. Thank you, Drew.

business Business career Career
0 views
Phil Eaton 4 months ago

Transaction pooling for Postgres with pgcat

This is an external post of mine. Click here if you are not redirected.

database Database
0 views
Phil Eaton 5 months ago

In response to a developer asking about systems

Sometimes I get asked questions that would be more fun to answer in public. All letters are treated as anonymous unless permission is otherwise granted. Hey [Redacted]! It's great to hear from you. I'm very glad you joined the coffee club and met some good folks. :) You asked how to learn about systems. A great question! I think I need to start first with what I mean when I say systems. My definition of systems is all of the underlying software we developers use but are taught not to think about because they are so solid: our compilers and interpreters, our databases, our operating system, our browser, and so on. We think of them as basically not having bugs, we just count on them to be correct and fast enough so we can build the applications that really matter to users. But 1) some developers do actually have to work on these fundamental blocks (compilers, databases, operating systems, browsers, etc.) and 2) it's not thaaaat hard to get into this development professionally and 3) even if you don't get into it professionally, having a better understanding of these fundamental blocks will make you a better application developer. At least I think so. To get into systems I think it starts by you just questioning how each layer you build on works. Try building that layer yourself. For example you've probably used a web framework like Rails or Next.js. But you can just go and write that layer yourself too (for education). And you've probably used Postgres or SQLite or DynamoDB. But you can also just go and write that layer yourself (for education). It's this habit of thinking and digging into the next lower layer that will get you into systems. Basically, not being satisfied with the black box. I do not think there are many good books on programming in general, and very very few must-read ones, but one that I recommend to everybody is Designing Data Intensive Applications. I think it's best if you read it with a group of people. (My book club will read it in December when the 2nd edition comes out, you should join.) But this book is specific to data obviously and not interested in the fundamentals of other systems things like compilers or operating systems or browsers or so on. Also, I see getting into this as a long-term thing. Throughout my whole career (almost 11 years now) I definitely always tried to dig into compilers and interpreters, I wrote and blogged about toy implementations a lot. And then 5 years ago I started digging into databases and saw that there was more career potential there. But it still took 4 years until I got my first job as a developer working on a database (the job I currently have). Things take time to learn and that's ok! You have a long career to look forward to. And if you end up not wanting to dig into this stuff that's totally fine too. I think very few developers actually do. And they still have fine careers. Anyway, I hope this is at least mildly useful. I hope you join the Software Internals Discord and nycsystems.xyz as well and look forward to seeing you at future coffee clubs! Cheers, Phil I wrote a letter in response to a developer asking about how to learn systems. pic.twitter.com/2ILNpzl662

database Database javascript JavaScript ruby Ruby
0 views
Phil Eaton 5 months ago

A simple clustering and replication solution for Postgres

This is an external post of mine. Click here if you are not redirected.

database Database devops DevOps
1 views
Phil Eaton 5 months ago

Analytics query goes 6x faster with EDB Postgres Distributed's new analytics engine

This is an external post of mine. Click here if you are not redirected.

database Database performance Performance
0 views
Phil Eaton 6 months ago

Set up a single-node EDB Postgres Distributed cluster on Ubuntu

This is an external post of mine. Click here if you are not redirected.

tutorial Tutorial database Database
0 views
Phil Eaton 6 months ago

What even is distributed systems

Distributed systems is simply the study of interactions between processes. Every two interacting processes form a distributed system, whether they are on the same host or not. Distributed systems create new challenges (compared to single-process systems) in terms of correctness (i.e. consistency ), reliability, and performance (i.e. latency and throughput). The best way to learn about the principles and fundamentals of distributed systems is to 1) read Designing Data Intensive Applications and 2) read through the papers and follow the notes in the MIT Distributed Systems course . For Designing Data Intensive Applications (DDIA), I strongly encourage you to find buddies at work or online who will read it through with you. You can also always join the Software Internals Discord 's #distsys channel to ask questions as you go. But it's still best if you have some partners to go through the book with, even if they are as new to it as you. I also used to think that you might want to wait a few years into your career before reading DDIA but when you have friends to read it with I think you need not wait. If you have only skimmed the book you should definitely go back and give it a thorough read. I have read it three times already and I will read it again as part of the Software Internals Book Club next year after the 2nd Edition is published. Keep in mind that every chapter of DDIA provides references to papers you can keep reading should you end up memorizing DDIA itself. When you've read parts of DDIA or the MIT Distributed Systems course and you want practice, the Fly.io x Jepsen Distributed Systems Challenge is one guided option. Other options might include simply implementing (getting progressively more complex down the list): And if you get bored there you can see Alex Miller's Data Replication Design Spectrum for more ideas and variants. And if you want more people to follow, check out the Distributed Systems section of my favorite blogs page. If these projects and papers sound arcane or intimidating, know that you will see the problems these projects/papers solve whether or not you know and understand these solutions. Developers often end up reinventing hacky versions of these which are more likely to have subtle bugs. While instead you can recognize and use one of these well-known building blocks. Or at least have the background to better reason about correctness should you be in a situation where you must work with a novel distributed system or you end up designing a new one yourself. And again, if you want folks to bounce ideas off of or ask questions to, I strongly encourage you to join the Software Internals Discord and ask there! I wrote a short post on learning the fundamentals of distributed systems, with a few suggested resources to read and a few suggested projects to try. pic.twitter.com/b0EhDP8K0t two-phase commit three-phase commit single-decree Paxos highly available key-value store on top of a 3rd-party consensus library chain replication (or CRAQ), using a 3rd-party consensus library

programming Programming backend Backend database Database
0 views
Phil Eaton 7 months ago

Stack traces for Postgres errors with backtrace_functions

This is an external post of mine. Click here if you are not redirected.

database Database
0 views
Phil Eaton 8 months ago

Want to meet people, try charging them for it?

I have been blogging consistently since 2017. And one of my goals in speaking publicly was always to connect with like-minded people. I always left my email and hoped people would get in touch. Even while my blog and twitter became popular, passing 1M views and 20k followers, I basically never had people get in touch to chat or meet up. So it felt kind of ridiculous when last November I started charging people $100 to chat . I mean, who am I? But people started showing up fairly immediately. Now granted the money did not go to me. It went to an education non-profit and I merely received the receipt. And at this point I've met a number of interesting people, from VCs to business professors to undergraduate students to founders and everyone in between. People wanting to talk about trends in databases, about how to succeed as a programmer, about marketing for developers, and so on. Women and men thoughout North America, Europe, Africa, New Zealand, India, Nepal, and so on. And I've raised nearly $6000 for educational non-profits. How is it that you go from giving away your time for free and getting no hits to charging and almost immediately getting results? For one, every person responded very positively to it being a fundraiser. It also helps me be entirely shameless about sharing on social media every single time someone donates; because it's such a positive thing. But also I think that in "charging" for my time it helps people feel more comfortable about actually taking my time, especially when we have never met. It gives you a reasonable excuse to take time from an internet rando. On the other hand, a lot of people come for advice and I think giving advice is pretty dangerous, especially since my background is not super conventional. I try to always frame things as just sharing my opinion and my perspective and that they should talk with many others and not take my suggestions without consideration. And there's also the problem that by charging everyone for my time now, I'm no longer available to people who could maybe use it the most. I do mention on my page that I will still take calls from people who don't donate, as my schedule allows. But to be honest I feel less incentivized to spend time when people do not donate. So I guess this is an issue with the program. But I mitigated even this slightly, and significantly jump-started the program, during my 30th birthday when I took calls with any person who donated at least $30. Anyway, I picked this path because I have wanted to get involved with helping students figure out their lives and careers. But without a degree I am literally unqualified for many volunteering programs. And I always found the time commitments for non-profits painful. So until starting this I figured it wouldn't be until I retire that I find some way to make a difference. But ultimately I kept meeting people who were starting their own non-profits now or donated significantly to help students. Peer pressure. I wanted to do my part now. And 30 minutes of my time in return for a donation receipt has been an easy trade. While only raising a humble $6,000 to date, the Chat for Education program has been more successful than I imagined. I've met many amazing people through it. And it's something that should be easy to keep up indefinitely. I hope to meet you through it too! I wrote about trying to meet like-minded people and fundraising for educational non-profits. pic.twitter.com/UJ9U6DIHGU

marketing Marketing business Business career Career
0 views
Phil Eaton 8 months ago

Debugging memory leaks in Postgres, jemalloc edition

This is an external post of mine. Click here if you are not redirected.

database Database performance Performance
0 views
Phil Eaton 9 months ago

Cheerleading

At work we're so absorbed in the difficulties we face that it becomes easy to forget what we appreciate and what we value in our coworkers. On social media we can be so focused on our own work that we forget to interact with others'. I have heard so many times from founders that they switch between heads down building, disappearing from the community, and then pop up to talk about what they've built. Engineers often say the exact same thing. This kills me. Though I probably still do it to a degree too. The easiest way for no one to give a shit about what you've done is for you to only interact at periodic intervals that are only convenient for you. With networking in general the most important time to interact and build relationships is when you do not need anything. By the time you want someone's support or you want someone to care about what you've done it is way too late to start building a relationship. One of the cheapest and most effective avenues to build genuine relationships is to be the biggest cheerleader you can be. At work this means when someone writes a fantastic blog post you call it out in a public channel. It means when a coworker's work is featured somewhere you call this out in a public channel. At least, this is what I do at work. I don't care that I have no leadership role or haven't met these people before. If I see something amazing that I'd like to see more of, I mention it in public and praise the person. Or when someone shares work they've done in a public channel, you hit all the emojis and you reply "that's awesome". On social media this means engaging seriously and (more) deeply with what people post. If the default behavior on social media is to passively observe, engaging more deeply can be liking a post. If you already show support through liking, engaging more deeply can be commenting or asking questions or (kindly) pointing out mistakes. Engagement is a spectrum. Genuine engagement consistently over time builds genuine relationships. I would not suggest that you do this without feeling it, or I'd rather only encourage you to respond to the degree that you feel. But I personally do feel so strongly when I cheer people on. So much of life and work is drudgery such that when you see something positive, someone taking initiative, someone with talent or potential doing something with their skills, how can you not feel an overwhelming urge to cheer them on and hope to see more of it? Hope to see it develop? What's more, I want to be around people who are trying new things and improving themselves. I want to be around people who celebrate. So I in turn try new things and work to improve myself and I celebrate the people around me. This energy is infectious. And I genuinely think even a single person in a group celebrating publicly changes the group dynamic. And I don't expect people to reciprocate in the same way or to the same degree that I show support. I'm particularly weird and confident and vocal. But if my cheerleading for some person seems like a complete sink then I'll not continue it and invest my time and energy elsewhere. Not everybody needs my support and that's ok! It is better that it's spent where it's most needed. There were people who used to cheer me on who don't much anymore and that's totally fine, I still have positive feelings for them for the time they did cheer me on. Cheerleading electrifies the work atmosphere. It is the proper use of social media. Cheerleading is the imperative of caring individuals as they become more experienced, gain confidence, command a wider audience, and want to continue their own growth through the development of genuine and supportive networks. And an ambitious, high-achieving, celebratory culture is simply the most fun to be a part of. I wrote a post on being a cheerleader. pic.twitter.com/mESNjSjQ0n

career Career
0 views
Phil Eaton 9 months ago

Debugging memory leaks in Postgres, heaptrack edition

This is an external post of mine. Click here if you are not redirected.

performance Performance database Database
0 views
Phil Eaton 10 months ago

Burn your title

I've been a developer, a manager, a cofounder, and now I'm a developer again. I ran away from each position until being a founder because I felt like I was limited by what I was allowed to do. But I reached an enlightment of sorts during my career progression: that everyone around me was dying for someone to pick things up, for employees to show engagement and agency. We think of our titles as our limits. We're quick to say and believe, "that isn't my job". While in reality titles reflect the minimum expected of us, not the maximum that is open to us. Trying to figure out what (new minimum) you must do to get promoted seems kind of backwards to me, reinforcing our sense of our own limits. Instead, at every stage in your career, focus on doing the intersection of: And this is the path to promotion and a successful and interesting career. Burn your title. Burn your job description. I mean, keep your boss happy for sure. Keep your teammates happy by supporting them and building them up and communicating well. But don't wait to be officially made a lead or given a new title to do what otherwise fits into that intersection above. And if after doing this for some time, demonstrating this level of agency, you are not promoted, it just means you're not at the right company or right organization within your company and you should look elsewhere. What's more, this work you did (at a company that doesn't appreciate your agency, if that happens to be the case) merely makes the case stronger for your successful interview at the next company. There's no downside. The cynical, and perhaps realistic, alternative to this is to do politics to get promoted. Or to not do politics but to do things that don't align with your long-term goals. I'm not personally interested in either path so I'm not covering them here. I'm interested in the intersection of things that move me in the direction I want, things that are useful to the company, and things that I am capable of doing (in addition to whatever minimum work I must actually do). Here's a peek at what this looks like for me as an individual contributor, a programmer, at EnterpriseDB. I started the EDB Engineering Newsletter because it seemed like we needed to do a better job telling the world the awesome things our engineering team is doing. (You know we're one of the biggest contributors to Postgres? Bruce Momjian, Robert Haas, Peter Eisentraut, etc. work here? The guy who implemented the WAL and MVCC in Postgres is my teammate?) Nobody asked me to do that. I started publishing blog views for the entire company once a month internally. Nobody asked me to do that. I wrote a number of internal docs and tutorials on the product because we were just obviously missing them. Nobody asked me to do that. I started a fortnightly incident review meeting for my team because it seemed like we were missing chances to update docs and teach each other. Nobody asked me to do that. I write a odd posts for the company blog on what I've learned. Nobody asked me to do that. These are just a few of the random things that seemed like a good idea for me to do on top of my Actual Work as a developer, which I think I do a decent job of on its own. Don't burn out. Don't do things you aren't asked for and don't find rewarding. Or that won't pave the way toward the career you want. I'm trying to be very careful not to advocate anything along those lines. But also don't wait to be asked to do something. Do what is interesting and obvious and rewarding to you. Interesting opportunities seem to come most reliably when you make them for yourself. Burn your title pic.twitter.com/4bQRPMX4EZ what you see needs to be done (that isn't being done) what you are capable of doing what you have the desire/energy (or would find fulfillment) doing

career Career
0 views
Phil Eaton 10 months ago

Transactions are a protocol

Transactions are not an intrinsic part of a storage system. Any storage system can be made transactional: Redis, S3, the filesystem, etc. Delta Lake and Orleans demonstrated techniques to make S3 (or cloud storage in general) transactional. Epoxy demonstrated techniques to make Redis (and any other system) transactional. And of course there's always good old Two-Phase Commit . If you don't want to read those papers, I wrote about a simplified implementation of Delta Lake and also wrote about a simplified MVCC implementation over a generic key-value storage layer. It is both the beauty and the burden of transactions that they are not intrinsic to a storage system. Postgres and MySQL and SQLite have transactions. But you don't need to use them. It isn't possible to require you to use transactions. Many developers, myself a few years ago included, do not know why you should use them. (Hint: read Designing Data Intensive Applications .) And you can take it even further by ignoring the transaction layer of an existing transactional database and implement your own transaction layer as Convex has done (the Epoxy paper above also does this). It isn't entirely clear that you have a lot to lose by implementing your own transaction layer since the indexes you'd want on the version field of a value would only be as expensive or slow as any other secondary index in a transactional database. Though why you'd do this isn't entirely clear (I will like to read about this from Convex some time). It's useful to see transaction protocols as another tool in your system design tool chest when you care about consistency, atomicity, and isolation. Especially as you build systems that span data systems. Maybe, as Ben Hindman hinted at the last NYC Systems , even proprietary APIs will eventually provide something like two-phase commit so physical systems outside our control can become transactional too. Transactions are a protocol short new post pic.twitter.com/nTj5LZUpUr

database Database
0 views