Posts in Toml (7 found)
Evan Schwartz 2 weeks ago

Your Clippy Config Should Be Stricter

“If it compiles, it works.” This feeling is one of the main things Rust engineers love most about Rust, and a reason why using it with coding agents is especially nice. After debugging some code that compiled but mysteriously stopped in production, I realized that it’s useful to enable more Clippy lints to catch bugs that the compiler won't prevent by itself. It's especially useful as guardrails for coding agents, but stricter linting can make your code safer, whether or not you’re coding with LLMs. Scour is the personalized content feed that I work on. Every Friday, Scour sends an email digest to each user with the top posts that matched their interests. On a recent Friday, the email sending job mysteriously stopped. This was puzzling because I had already put in place multiple type system-level safeguards and tests to ensure that it would continue with a log on all types of errors. After digging into the logs, I found the culprit to be . A function naively truncated article summaries without checking for UTF-8 character boundaries, which caused a panic and stopped the Tokio worker thread running the email sending loop. The solution for this particular bug was a safer method for truncating article summaries that respects UTF-8 character boundaries. However, this problem was reminiscent enough of the 2025 Cloudflare bug that "broke the internet" that I wanted some more general solution. Rust's compiler prevents many types of bugs but there are still production problems it can't catch. Panics will either crash your program or quietly kill Tokio worker threads. Deadlocks and dropped futures can make work silently stop. And plenty of numeric operations can silently cause incorrect behavior. We can stave off many of these types of bugs by making Clippy even stricter than it already is. This is especially relevant in the age of coding agents. A seasoned Rust engineer might naturally avoid patterns that could cause problems. An agent or a junior colleague might not. Stricter Clippy rules make it easier to rely on code you didn't personally write. Also, enabling new lints on an existing codebase is tedious, and exactly the kind of task that is good to hand to a coding agent. Clippy ships with hundreds of lints that are disabled by default. Some are disabled because they might have false positives and some are style choices which you might reasonably not want. Which lints should we enable to help us get back the "if it compiles [and passes Clippy], it works" feeling? Clippy's lints are grouped into categories : Correctness, Suspicious, Complexity, Perf, Style, Pedantic, Restriction, Cargo, Nursery, and Deprecated. Unfortunately, none of these categories cleanly map onto "don't let this panic or do the wrong thing in production". In fact, the Clippy docs say that "The category should, emphatically , not be enabled as a whole." Clippy even includes a dedicated lint, , to discourage you from enabling this category. While the category includes many useful lints, it also includes some that directly contradict one another. For example, it contains lints to enforce both and . The docs say "Lints should be considered on a case-by-case basis before enabling". Of course, you can enable whole categories like and and then specific ones you want to disable, but I'm outlining a selective opt-in here. Even if you don't use a certain pattern in your code base today, it's not bad to enable the lint anyway. Inapplicable lints serve as cheap tripwires in case the given pattern is ever added later, whether by you, a colleague, or a coding agent. Every project is different and you should look through the available lints to see which ones make sense for your project. Also, check when lints landed in stable if your Minimum Supported Rust Version predates 1.95, as some of these may have been added after your MSRV. With those caveats out of the way, here are the lints I enabled, roughly categorized by what kind of behavior they prevent. You can skip to the bottom if you just want to copy my config . This group prevents panics from unwraps and unsafe slicing or indexing into arrays and strings. Note that some of these, like and may produce many warnings throughout your code base. That may be annoying to fix. However, using safe methods like and iterators instead of slicing prevents pretty severe footguns, so I would argue that it's worth it. You might or might not want to enable . Calling on an or can result in a panic. However, the message you pass to should already document why that thing shouldn't happen. Enabling the lint and then selectively disabling it throughout your code with may end up duplicating the same rationale for using it in the first place. Another lint that is a real judgement call is . This can prevent overflows and division by zero. However, it will cause Clippy to warn you about every place you use math operators: , , , , , and . I tried enabling it in my code base and would estimate that around 15% of the warnings caught real issues and 85% was just noise. These prevent various concurrency bugs and deadlocks: The lints , , effectively force you to document invariants when doing lossy casts between numeric types. You might or might not find that useful. These two are especially useful if you're using a coding agent. Instead of letting the agent write , it should provide a reason wherever it's disabling a lint. If you're using a Cargo workspace, you'll want to enable these lints in the workspace Cargo.toml. Unfortunately, each workspace crate needs to opt in to inheriting lints with , rather than inheriting the lints by default. On nightly, there's a lint that specifically checks for this. If you're using stable Rust, you can use or a simple shell script run on CI to make sure you don't forget to make a workspace crate inherit the lints. When enabling lints, you can either set Clippy to or them. Either works but I personally prefer setting these to and running Clippy with before committing and on CI. This makes local iteration marginally easier because you can compile your code initially without fixing all the lints right away. Ultimately, as Clippy's docs say, "You can choose how much Clippy is supposed to annoy help you." But especially in the age of coding agents, I think it's worth tightening the guardrails so you end up with even fewer mysterious bugs in production and more code where you can say "if it compiles and lints, it should work." Discuss on r/rust , Lobsters , or Hacker News . - on (UTF-8 boundary panic). This would have caught my initial bug. / / - placeholder-panic macros - inside functions that return a - panics if the second is larger - / inside a function that returns a - drops without awaiting - swallows errors - silently drops - loses source error - discards the error message (only relevant if you're using an earlier edition than 2024) - deadlock pattern. The scoping was fixed in the 2024 edition so this is no longer an issue. - a that is too large can cause a stack overflow - every needs a comment - one unsafe op per block (one comment per op) / - only document safety where it belongs - on floats - stricter, also flags comparisons against constants - silently-rounded float literals ( ) - wraps to - always false - ( is single-threaded) - differs in debug vs release - method named returning non- - manual impl that disagrees with - impl whose error is should be - calls should be removed after debugging - every becomes - every requires a reason

rust Rust shell Shell programming Programming Toml
0 views
Fernando Borretti 3 months ago

Some Data Should Be Code

I write a lot of Makefiles . I use it not as a command runner but as an ad-hoc build system for small projects, typically for compiling Markdown documents and their dependencies. Like so: And the above graph was generated by this very simple Makefile: (I could never remember the automatic variable syntax until I made flashcards for them.) It works for simple projects, when you can mostly hand-write the rules. But the abstraction ceiling is very low. If you have a bunch of almost identical rules, e.g.: You can use pattern-matching to them into a “rule schema”, by analogy to axiom schemata: Which works backwards: when something in the build graph depends on a target matching , Make synthesizes a rule instance with a dependency on the corresponding file. But pattern matching is still very limited. Lately I’ve been building my own plain-text accounting solution using some Python scripts. One of the tasks is to read a CSV of bank transactions from 2019–2024 and split it into TOML files for each year-month, to make subsequent processing parallelizable. So the rules might be something like: I had to write a Python script to generate the complete Makefile. Makefiles look like code, but are data: they are a container format for tiny fragments of shell that are run on-demand by the Make engine. And because Make doesn’t scale, for complex tasks you have to bring out a real programming language to generate the Makefile. I wish I could, instead, write a file with something like this: Fortunately this exists: it’s called doit , but it’s not widely known. A lot of things are like Makefiles: data that should be lifted one level up to become code. Consider CloudFormation . Nobody likes writing those massive YAML files by hand, so AWS introduced CDK , which is literally just a library 1 of classes that represent AWS resources. Running a CDK program emits CloudFormation YAML as though it were an assembly language for infrastructure. And so you get type safety, modularity, abstraction, conditionals and loops, all for free. Consider GitHub Actions . How much better off would we be if, instead of writing the workflow-job-step tree by hand, we could just have a single Python script, executed on push, whose output is the GitHub Actions YAML-as-assembly? So you might write: Actions here would simply be ordinary Python libraries the CI script depends on. Again: conditions, loops, abstraction, type safety, we get all of those for free by virtue of using a language that was designed to be a language, rather than a data exchange language that slowly grows into a poorly-designed DSL. Why do we repeatedly end up here? Static data has better safety/static analysis properties than code, but I don’t think that’s foremost in mind when people design these systems. Besides, using code to emit data (as CDK does) gives you those exact same properties. Rather, I think some people think it’s cute and clever to build tiny DSLs in a data format. They’re proud that they can get away with a “simple”, static solution rather than a dynamic one. If you’re building a new CI system/IaC platform/Make replacement: please just let me write code to dynamically create the workflow/infrastructure/build graph. Or rather, a polyglot collection of libraries, one per language, like Pulumi .  ↩ Or rather, a polyglot collection of libraries, one per language, like Pulumi .  ↩

python Python shell Shell Toml programming Programming devops DevOps
0 views
Robin Moffatt 4 months ago

Alternatives to MinIO for single-node local S3

In late 2025 the company behind MinIO decided to abandon it to pursue other commercial interests. As well as upsetting a bunch of folk, it also put the cat amongst the pigeons of many software demos that relied on MinIO to emulate S3 storage locally, not to mention build pipelines that used it for validating S3 compatibility. In this blog post I’m going to look at some alternatives to MinIO. Whilst MinIO is a lot more than 'just' a glorified tool for emulating S3 when building demos, my focus here is going to be on what is the simplest replacement. In practice that means the following: Must have a Docker image. So many demos are shipped as Docker Compose, and no-one likes brewing their own Docker images unless really necessary. Must provide S3 compatibility. The whole point of MinIO in these demos is to stand-in for writing to actual S3. Must be free to use, with a strong preference for Open Source (per OSI definition ) licence e.g. Apache 2.0. Should be simple to use for a single-node deployment Should have a clear and active community and/or commercial backer. Any fule can vibe-code some abandon-ware slop, or fork a project in a fit of enthusiasm—but MinIO stood the test of time until now and we don’t want to be repeating this exercise in six months' time. Bonus points for excellent developer experience (DX), smooth configuration, good docs, etc. What I’m not looking at is, for example, multi-node deployments, distributed storage, production support costs, GUI capabilities, and so on. That is, this blog post is not aimed at folk who were using MinIO as self-managed S3 in production. Feel free to leave a comment below though if you have useful things to add in this respect :) My starting point for this is a very simple Docker Compose stack: DuckDB to read and write Iceberg data that’s stored on S3, provided by MinIO to start with. You can find the code here . The Docker Compose is pretty straightforward: DuckDB, obviously, along with Iceberg REST Catalog MinIO (S3 local storage) , which is a MinIO CLI and used to automagically create a bucket for the data. When I insert data into DuckDB: it ends up in Iceberg format on S3, here in MinIO: In each of the samples I’ve built you can run the to verify it. Let’s now explore the different alternatives to MinIO, and how easy they are to switch MinIO out for. I’ve taken the above project and tried to implement it with as few changes to use the replacement for MinIO. I’ve left the MinIO S3 client, in place since that’s no big deal to replace if you want to rip out MinIO completely (s3cmd, CLI, etc etc). 💾 Example Docker Compose Version tested: ✅ Docker image (5M+ pulls) ✅ Licence: Apache 2.0 ✅ S3 compatibility Ease of config: 👍👍 Very easy to implement, and seems like a nice lightweight option. 💾 Example Docker Compose Version tested: Ease of config: ✅✅ ✅ Docker image (100k+ pulls) ✅ Licence: Apache 2.0 ✅ S3 compatibility RustFS also includes a GUI: 💾 Example Docker Compose Version tested: ✅ Docker image (5M+ pulls) ✅ Licence: Apache 2.0 ✅ S3 compatibility Ease of config: 👍 This quickstart is useful for getting bare-minimum S3 functionality working. (That said, I still just got Claude to do the implementation…). Overall there’s not too much to change here; a fairly straightforward switchout of Docker images, but the auth does need its own config file (which as with Garage, I inlined in the Docker Compose). SeaweedFS comes with its own basic UI which is handy: The SeaweedFS website is surprisingly sparse and at a glance you’d be forgiven for missing that it’s an OSS project, since there’s a "pricing" option and the title of the front page is "SeaweedFS Enterprise" (and no GitHub link that I could find!). But an OSS project it is, and a long-established one: SeaweedFS has been around with S3 support since its 0.91 release in 2018 . You can also learn more about SeaweedFS from these slides , including a comparison chart with MinIO . 💾 Example Docker Compose Version tested: ✅ Docker image (also outdated ones on Docker Hub with 5M+ pulls) ✅ Licence: Apache 2.0 ✅ S3 compatibility Ease of config: 👍 Formerly known as S3 Server, CloudServer is part of a toolset called Zenko, published by Scality. It drops in to replace MinIO pretty easily, but I did find it slightly tricky at first to disentangle the set of names (cloudserver/zenko/scality) and what the actual software I needed to run was. There’s also a slightly odd feel that the docs link to an outdated Docker image. 💾 Example Docker Compose Ease of config: 😵 Version tested: ✅ Docker image (1M+ pulls) ✅ Licence: AGPL ✅ S3 compatibility I had to get a friend to help me with this one. As well as the container, I needed another to do the initial configuration, as well as a TOML config file which I’ve inlined in the Docker Compose to keep things concise. Could I have sat down and RTFM’d to figure it out myself? Yes. Do I have better things to do with my time? Also, yes. So, Garage does work, but gosh…it is not just a drop-in replacement in terms of code changes. It requires different plumbing for initialisation, and it’s not simple at that either. A simple example: . Excellent for production hygiene…overkill for local demos, and in fact somewhat of a hindrance TBH. 💾 Example Docker Compose Version tested: ✅ Docker images (1M+ pulls) ✅ Licence: Apache 2.0 ✅ S3 compatibility Ozone was spun out of Apache Hadoop (remember that?) in 2020 , having been initially created as part of the HDFS project back in 2015. Ease of config: 😵 It does work as a replacement for MinIO, but it is not a lightweight alternative; neither I nor Claude could figure out how to deploy it with any fewer than four nodes. It gives heavy Hadoop vibes, and I wouldn’t be rushing to adopt it for my use case here. I took one look at the installation instructions and noped right out of this one! Ozone (above) is heavyweight enough; I’m sure both are great at what they do, but they are not a lightweight container to slot into my Docker Compose stack for local demos. Everyone loves a bake-off chart, right? gaul/s3proxy ( Git repo ) Single contributor ( Andrew Gaul ) ( Git repo ) Fancy website but not much detail about the company ( Git repo ) Single contributor ( Chris Lu ), Enterprise option available Zenko CloudServer ( Git repo ) Scality (commercial company) 5M+ (outdated version) ( Git repo ) NGI/NLnet grants Apache Ozone ( Git repo ) Apache Software Foundation 1 Docker pulls is a useful signal but not an absolute one given that a small number of downstream projects using the image in a frequently-run CI/CD pipeline could easily distort this figure. I got side-tracked into writing this blog because I wanted to update a demo in which currently MinIO was used. So, having tried them out, which of the options will I actually use? SeaweedFS - yes. S3Proxy - yes. RustFS - maybe, but very new project & alpha release. CloudServer - yes, maybe? Honestly, put off by it being part of a suite and worrying I’d need to understand other bits of it to use it—probably unfounded though. Garage - no, config too complex for what I need. Apache Ozone - lol no. I mean to cast no shade on those options against which I’ve not recorded a ; they’re probably excellent projects, but just not focussed on my primary use case (simple & easy to configure single-node local S3). A few parting considerations to bear in mind when choosing a replacement for MinIO: Governance . Whilst all the projects are OSS, only Ozone is owned by a foundation (ASF). All the others could, in theory , change their licence at the drop of a hat (just like MinIO did). Community health . What’s the "bus factor"? A couple of the projects above have a very long and healthy history—but from a single contributor. If they were to abandon the project, would someone in the community fork and continue to actively develop it? Must have a Docker image. So many demos are shipped as Docker Compose, and no-one likes brewing their own Docker images unless really necessary. Must provide S3 compatibility. The whole point of MinIO in these demos is to stand-in for writing to actual S3. Must be free to use, with a strong preference for Open Source (per OSI definition ) licence e.g. Apache 2.0. Should be simple to use for a single-node deployment Should have a clear and active community and/or commercial backer. Any fule can vibe-code some abandon-ware slop, or fork a project in a fit of enthusiasm—but MinIO stood the test of time until now and we don’t want to be repeating this exercise in six months' time. Bonus points for excellent developer experience (DX), smooth configuration, good docs, etc. DuckDB, obviously, along with Iceberg REST Catalog MinIO (S3 local storage) , which is a MinIO CLI and used to automagically create a bucket for the data. ✅ Docker image (5M+ pulls) ✅ Licence: Apache 2.0 ✅ S3 compatibility ✅ Docker image (100k+ pulls) ✅ Licence: Apache 2.0 ✅ S3 compatibility ✅ Docker image (5M+ pulls) ✅ Licence: Apache 2.0 ✅ S3 compatibility ✅ Docker image (also outdated ones on Docker Hub with 5M+ pulls) ✅ Licence: Apache 2.0 ✅ S3 compatibility ✅ Docker image (1M+ pulls) ✅ Licence: AGPL ✅ S3 compatibility ✅ Docker images (1M+ pulls) ✅ Licence: Apache 2.0 ✅ S3 compatibility SeaweedFS - yes. S3Proxy - yes. RustFS - maybe, but very new project & alpha release. CloudServer - yes, maybe? Honestly, put off by it being part of a suite and worrying I’d need to understand other bits of it to use it—probably unfounded though. Garage - no, config too complex for what I need. Apache Ozone - lol no. Governance . Whilst all the projects are OSS, only Ozone is owned by a foundation (ASF). All the others could, in theory , change their licence at the drop of a hat (just like MinIO did). Community health . What’s the "bus factor"? A couple of the projects above have a very long and healthy history—but from a single contributor. If they were to abandon the project, would someone in the community fork and continue to actively develop it?

open-source Open Source Toml devops DevOps cloud Cloud
0 views
Schneems 5 months ago

Disallow code usage with a custom `clippy.toml`

I recently discovered that adding a file to the root of a Rust project gives the ability to disallow a method or a type when running . This has been really useful. I want to share two quick ways that I’ve used it: Enhancing calls via and protecting CWD threadsafety in tests. Update: you can also use this technique to disallow unwrap() ! There’s also which you use by adding to your . I use the fs_err crate in my projects, which provides the same filesystem API as but with one crucial difference: error messages it produces have the name of the file you’re trying to modify. Recently, while I was skimming the issues, someone mentioned using clippy.toml to deny usage . I thought the idea was neat, so I tried it in my projects, and it worked like a charm. With this in the file: Someone running will get an error: Running will now automatically update the code. Neat! Why was I skimming issues in the first place? I suggested adding a feature to allow enhancing errors with debugging information , so instead of: The message could contain a lot more info: To implement that functionality, I wrote path_facts , a library that provides facts about your filesystem (for debugging purposes). And since the core value of the library is around producing good-looking output, I wanted snapshot tests that covered all my main branches. This includes content from both relative and absolute paths. A naive implementation might look like this: In the above code, the test changes the current working directory to a temp dir where it is then free to make modifications on disk. But, since Rust uses a multi-threaded test runner and affects the whole process, this approach is not safe ☠️. There are a lot of different ways to approach the fix, like using cargo-nextest , which executes all tests in their own process (where changing the CWD is safe). Though this doesn’t prevent someone from running accidentally. There are other crates that use macros to force non-concurrent test execution, but they require you to remember to tag the appropriate tests . I wanted something lightweight that was hard to mess up, so I turned to to fail if anyone used for any reason: Then I wrote a custom type that used a mutex to guarantee that only one test body was executing at a time: You might call my end solution hacky (this hedge statement brought to you by too many years of being ONLINE), but it prevents anyone (including future-me) from writing an accidentally thread-unsafe test: Those are only two quick examples showing how to use clippy.toml to enhance a common API, and how to safeguard against incorrect usage. There’s plenty more you can do with that file, including: You wouldn’t want to use this technique of annotating your project with if the thing you’re trying to prevent would be actively malicious for the system if it executes, since rules won’t block your . You’ll also need to make sure to run in your CI so some usage doesn’t accidentally slip through. And that clippy lint work has paid off, my latest PR to was merged and deployed in version , and you can use it to speed up your development debugging by turning on the feature: Clip cautiously, my friends.

testing Testing Toml rust Rust
0 views
マリウス 9 months ago

📨🚕

Disclaimer: This service is in alpha state, see below . Please do not share this post, or 📨🚕 on link aggregation platforms just yet. The server infrastructure will not be able to handle a sudden traffic spike and besides, 📨🚕 is way too undercooked for the average HN user/lobster/redditor/etc. atm. Thank you for your understanding. For a long time, I’ve been using Pushover for various tasks, such as infrastructure monitoring, and I even contributed to its ecosystem . However, as I transitioned towards more freedom-respecting software , my needs changed, and Pushover no longer met them. I ended up building my own near drop-in replacement and migrated to it. This allowed me to use the clients I preferred to receive notifications, rather than being locked into a closed-source app that doesn’t work outside of Apple’s and Google’s ( GSF ) walled gardens. I’ve been running Overpush for myself for quite some time now, and it’s grown into a reliable open-source tool that anyone can easily set up on their own server. One challenge, though, has been the integration of target platforms. For example, if someone wanted Overpush to send notifications to XMPP, they’d have to find a reliable host, create a dedicated account, configure it in their file, and set up their target user account. While not overly complicated, it’s a tedious process that becomes more cumbersome with multiple platforms. Since I’ve been successfully hosting Overpush for myself, I thought I could turn it into a service for others. The service is based on a single, relatively lightweight Go binary, which doesn’t require heavy hardware. However, to support new users and dynamically configure their applications (where an application defines an endpoint-to-target-service route), I needed to extend Overpush to use a database, replacing the simple TOML file. I also had to build a web interface for user sign-ups and account management. Ideally, I wanted a clean landing page with essential documentation to help people get started. A few months ago, I began working on these updates, and I’m excited to announce that the first version of the hosted Overpush service is now live! Although I originally named the open-source project Overpush , I wanted to move away from its legacy of being a mere replacement for Pushover and focus on the future. That’s why I’m calling the hosted service 📨🚕, or MSG.TAXI , which is now available at the domain: https://msg.taxi As described in the introductory post on the 📨🚕 blog , MSG.TAXI is a multi-protocol push notification router. You send data to it via a webhook URL, and it routes that data to your configured targets (e.g. XMPP, Matrix, Telegram, and more coming soon). It’s the missing link between your code and your notification channels, whether that’s your smart home, CI pipeline, RPG guild’s Matrix room, or just your phone at 3 AM when your server crashes (again). 📨🚕 is push notifications from anything, to anything . :-) You can check out Overpush for a detailed breakdown of how the service works, but in short, 📨🚕 lets you sign up and create what are called applications . Each application is assigned a custom webhook URL and can route the data it receives to a single target platform (for example, XMPP) at the moment. Depending on the target platform you choose for an application , you’ll need to configure a destination , typically your user ID or username on that platform. For XMPP, this would be your JID . Once that’s set up, you can start posting webhooks to the application -specific URL, and 📨🚕 will process and forward them to your target. As of right now 📨🚕 supports the following target platforms: I’m working to make more target platforms available on 📨🚕. If you’re interested to use the service let me know which platforms you need ! One thing that’s particularly important to me is privacy , which is often overlooked in hosted services. With 📨🚕, I’m striving to do better. Currently, none of the target platforms support native E2EE implementations. For example, notifications routed via XMPP won’t be encrypted with OMEMO. There are a few reasons for this: Even if I were to implement native E2EE for each platform, it would be a half-hearted solution. The content of your notifications would be sent in plain text within 📨🚕 and encrypted only when it leaves the service. This would mean you’d have to trust 📨🚕 to not inspect your data. I don’t want you to have to trust 📨🚕. So instead of dealing with this overhead, my focus is on the “ends” in “end-to-end” . My long-term goal is to offer E2EE that’s modern and independent of the target platform. To achieve this, I’m developing a lightweight and portable 📨🚕 CLI client that allows you to post encrypted webhooks to the service. On the receiving end, the challenge is greater, as a client would need to be available on various platforms, including mobile. For now, there’s a manual method for implementing E2EE, at least on Android. I briefly documented it here . Essentially, you would use GPG to encrypt the content before submitting it to 📨🚕, and then use OpenKeychain on your Android device to decrypt the messages. You’d forward the encrypted message from e.g. Conversations , Element , Telegram , or whichever target platform’s client you’re using, via the system’s share dialog. (usually by long-tapping the received message, tapping “Share with” and choosing OpenKeychain ) I recommend using this method for confidential information. If for some reason you can’t encrypt the message at the source, 📨🚕 natively integrates age encryption. However, this isn’t true E2EE, as encryption happens after the webhook is accepted by 📨🚕 and processed by an internal worker . One future improvement is to move this encryption step to the moment the post hits the webhook, so the content remains encrypted throughout 📨🚕’s internal systems. I will also be adding integrated GPG support soon. Another key privacy concern is data storage. 📨🚕 does not store copies of the notifications you route through the service, with one exception: Each application allows you to debug the payload it receives via webhooks. Think of it like a debug output for the notifications. If you enable this debugging feature, 📨🚕 will store a copy of the latest webhook content in its database so you can view it in the web console. However, once a new webhook hits the endpoint, the previously stored content is overwritten. 📨🚕 does not maintain a historical timeline of your webhooks. That said, if you keep debugging enabled, it’s important to note that backups are taken periodically to prevent data loss in case of server failure. These backups will contain the latest webhook received at the time of backup. This means that if debugging is left on, the data might end up being stored in backups, even if it’s not part of the active database. In summary, if privacy is important to you, I recommend disabling webhook logging once you’ve verified that your application is working as expected. As mentioned in the blog post, the current version of 📨🚕 is more like an alpha release. While it should work smoothly, occasional hiccups are possible. I’ll do my best to ensure a seamless experience, but please bear with me if things don’t work perfectly right away. If you run into any issues, feel free to reach out . If you think 📨🚕 could be useful to your workflow, I invite you to give it a try. You’re also welcome to share it with anyone who might find it useful, but I kindly ask that you do not share this post, or 📨🚕 on link aggregation platforms just yet. The server infrastructure will not be able to handle a sudden traffic spike and besides, 📨🚕 is way too undercooked for the average HN user/lobster/Redditor atm. I’d prefer the platform to grow slowly and organically so I can actively engage with users, resolve issues, fix bugs, and eventually make it a really great service. I’m looking forward to hearing from you if you decide to give 📨🚕 a try!

open-source Open Source Toml devops DevOps
1 views
Kaushik Gopal 10 months ago

🦊 How to Firefox

Chrome finally pulled the trigger on the web’s best ad-blocker, uBlock Origin . Now that Chrome has hobbled uBO, Firefox — my beloved — 1 is surging again. I want to do my part to convince you to switch to Firefox and show you how I use it. Let’s get through the important talking points, in case you need a quick copy paste to convince a friend. This section can be quick. Here’s a github link to the source code of the Firefox browser. You can clone the repo, pop open your favorite AI code assistant and start asking questions about your browser - the most important app you use. What libraries does their Android app use? libs.versions.toml boom! Also 8.11.1 on android gradle plugin? not bad Firefox. Their license allows you to fork and distribute alternative versions . Vibe code a whole new browser. Most of the web today is enshittified with a cesspool of ads, popups, cookie notices, and tracking scripts. Our primary defense has been ad-blockers, with the most powerful being uBlock Origin. uBO relies on community-curated filter lists that play a cat-and-mouse game to zap known ads, trackers, and other digital sludge. But with Chrome controlling the web, Google followed through on its promise to kneecap uBO with Manifest V3, effectively blocking the full version from its extension store. Sure, there’s uBlock Origin “Lite” now, which does the same thing, right? Nope : uBlock Origin Lite != uBlock Origin Filter lists update only when the extension updates no fetching up to date lists from servers (this is a big one!) No custom filters so no element picker which allows you to point and zap Many filters are dropped at conversion time due to MV3’s limited filter syntax No strict-blocked pages No per-site switches No dynamic filtering No importing external lists Did you know, uBlock Origin works best on Firefox . Why not just use the real thing then? My browsing experience is beautiful because I have most of the shit-bits blocked away. On my Pixel too. With Firefox for Android, you get seamless sync of tabs, bookmarks, passwords between browser and phone 2 . Let’s face it, Safari between Mac and iPhone is a sublime experience. We can get that with Firefox. Here’s something the iPhone isn’t getting anytime soon: honest-to-god browser extensions that you use on your desktop, also on your phone. Which means… you can run uBlock Origin on Android, completely unnerfed . Safari has extensions, but they still require an App Store review for distribution on Apple platforms. They also just got a version of the uBO “Lite” extension. But… Firefox doesn’t look as clean and minimal as Safari. You can claw the vertical tabs out of my cold dead Arc hands! This is what my Firefox browser looks like: It only takes about five minutes and a browser restart to get this look. I’ll walk you through my setup now, from essential add-ons to privacy tweaks and a few “nice-to-have” extras. Nerd Alert This is my setup. I’m a nerd, so I find joy in tinkering. You don’t need to do all of this, but a few small tweaks can give you a massively better browser. Think of uBO as a powerful, wide-spectrum filter for the web. It uses community-maintained lists to block ads, trackers, cookie notices, and other digital sludge before it ever loads. Your browser stays faster and cleaner. It can be confusing to know which filter lists to enable. I follow the advice of a uBO wizard on Reddit , and these settings alone make the web 90% better. Check the same boxes, and you’re good to go. Custom Filters are an exclusive uBO superpower The “My filters” tab is where you can write your own rules to block nearly anything, from annoying widgets to entire domains. For the truly privacy-conscious, uBO can block all outgoing traffic to specific domains, like Facebook. 3 In the past, Firefox’s Facebook Container add-on helped by isolating your Facebook activity. But if any other site embedded a Facebook widget or tracker, your data could still leak to Meta’s servers, fingerprinting you even if you never visit Facebook directly. With a custom uBO rule, you can sever that connection entirely from non-Facebook sites. This is a level of control other browsers don’t offer. The other line you see there? That one-liner blocks all those “Sign in with Google?” pop-ups. This granular control is only possible with the full uBlock Origin, not the “Lite” version found on other browsers. If you want to go deeper, this video is a great showcase of its advanced capabilities. Firefox now includes Total Cookie Protection (TCP) by default. This automatically isolates cookies to the site that created them, giving each site its own “cookie jar”. Importantly it means sites aren’t allowed to read cookies from other site’s jars. This stops trackers from following you across the web. Firefox first piloted this feature as the Multi-Account Containers (MAC) add-on. But with TCP, the containers feature is somewhat redundant for basic anti-tracking . However, the container technology is still incredibly useful if you want to seamlessly manage multiple online identities. Instead of juggling separate browser profiles, you can use “Containers” to stay logged into two different Gmail accounts (e.g., “Work” and “Personal”) in the same browser window, with zero overlap. The old MAC add-on made this possible, but it was really clunky to setup. It’s actually much easier to do this today without installing an additional add-on. That’s it. This works without the extra MAC add-on because the Container concept is baked natively into Firefox. So with the above config tweaks, you enable the default built-in containers and whenever you click the new tab button, you can choose which container to open. You can now open gmail in these containers and it’s as if you’re opening an entirely new browser. But what about links? A work link (like Datadog or Sentry) clicked from your email in a Work container, might open in the default container and use the wrong Google account. You could right click the link and say “Open in Container >” but that gets old fast. Wouldn’t it be nice if you could give Firefox a set of rules and say, always open these websites or URLs in these containers? and everything just worked magically? I created the firefox add-on Container Traffic Control to help with that exact requirement. It’s open source and pretty well documented + tested. This combination of the native config tweak and my add-on provides a simple, but more powerful multi-profile setup (than even MAC). These are also not essential, but they add a nice layer of polish. Firefox is famously customizable via . Besides the container tweak, I only use one other: I’m collecting in this section, some of the cooler Firefox features that’ll make you wonder why every browser doesn’t have them: The web can still be beautiful. You just need the right tools to see it. Go download Firefox and make your web beautiful again. If you try this setup or have suggestion, let me know in the comments. No, goddammit, AI didn’t write this post.  ↩︎ For the few who have reached this point of the article and furiously questioned why I don’t just use Zen browser or Libre.  ↩︎ You can of course send outgoing traffic from Meta owned websites so Threads etc. still work.  ↩︎ Seriously, try the apostrophe trick. It’s a game-changer for keyboard navigation.  ↩︎ 100% open-source Un-enshittify the web Android users rejoice Customize to your heart’s content Filter lists update only when the extension updates no fetching up to date lists from servers (this is a big one!) No custom filters so no element picker which allows you to point and zap Many filters are dropped at conversion time due to MV3’s limited filter syntax No strict-blocked pages No per-site switches No dynamic filtering No importing external lists Dark Reader : For a consistent, customizable dark mode on every site. Stylus : To apply custom CSS. I use it to force my on code blocks. Return YouTube Dislike : Does what it says on the tin. Obsidian Web Clipper : To save notes and clippings directly to Obsidian, from desktop or mobile. Auto Tab Discard : Suspends background tabs to save RAM. A holdover from my RAM-strapped MacBook days, but it still does its job silently. New tabs open next to your current tab, not at the end. You catch that Mr.Gruber ? Type and start typing for quick find (vs ⌘F). But dig this, and Firefox will only match text for hyper links 4 If you have an obnoxious site disable right click, just hold Shift and Firefox will bypass and show it to you. No add-one required. URL bar search shortcuts: for bookmarks, for open tabs, for history No, goddammit, AI didn’t write this post.  ↩︎ For the few who have reached this point of the article and furiously questioned why I don’t just use Zen browser or Libre.  ↩︎ You can of course send outgoing traffic from Meta owned websites so Threads etc. still work.  ↩︎ Seriously, try the apostrophe trick. It’s a game-changer for keyboard navigation.  ↩︎

css CSS Toml web-development Web Development java Java open-source Open Source
0 views
Matthias Endler 1 years ago

The Best Programmers I Know

I have met a lot of developers in my life. Lately, I asked myself: “What does it take to be one of the best? What do they all have in common?” In the hope that this will be an inspiration to someone out there, I wrote down the traits I observed in the most exceptional people in our craft. I wish I had that list when I was starting out. Had I followed this path, it would have saved me a lot of time. Read the Reference If there was one thing that I should have done as a young programmer, it would have been to read the reference of the thing I was using. I.e. read the Apache Webserver Documentation , the Python Standard Library , or the TOML spec . Don’t go to Stack Overflow, don’t ask the LLM, don’t guess , just go straight to the source . Oftentimes, it’s surprisingly accessible and well-written. Know Your Tools Really Well Great devs understand the technologies they use on a fundamental level . It’s one thing to be able to use a tool and a whole other thing to truly grok (understand) it. A mere user will fumble around, get confused easily, hold it wrong and not optimize the config. An expert goes in (after reading the reference!) and sits down to write a config for the tool of which they understand every single line and can explain it to a colleague. That leaves no room for doubt! To know a tool well, you have to know: For example, if you are a backend engineer and you make heavy use of Kafka, I expect you to know a lot about Kafka – not just things you read on Reddit. At least that’s what I expect if you want to be one of the best engineers. Read The Error Message As in Really Read the Error Message and Try to Understand What’s Written . Turns out, if you just sit and meditate about the error message, it starts to speak to you. The best engineers can infer a ton of information from very little context. Just by reading the error message, you can fix most of the problems on your own. It also feels like a superpower if you help someone who doesn’t have that skill. Like “reading from a cup” or so. Break Down Problems Everyone gets stuck at times. The best know how to get unstuck. They simplify problems until they become digestible. That’s a hard skill to learn and requires a ton of experience. Alternatively, you just have awesome problem-solving skills, e.g., you’re clever. If not, you can train it, but there is no way around breaking down hard problems. There are problems in this world that are too hard to solve at once for anyone involved. If you work as a professional developer, that is the bulk of the work you get paid to do: breaking down problems. If you do it right, it will feel like cheating: you just solve simple problems until you’re done. Don’t Be Afraid To Get Your Hands Dirty The best devs I know read a lot of code and they are not afraid to touch it. They never say “that’s not for me” or “I can’t help you here.” Instead, they just start and learn. Code is just code . They can just pick up any skill that is required with time and effort. Before you know it, they become the go-to person in the team for whatever they touched. Mostly because they were the only ones who were not afraid to touch it in the first place. Always Help Others A related point. Great engineers are in high demand and are always busy, but they always try to help. That’s because they are naturally curious and their supportive mind is what made them great engineers in the first place. It’s a sheer joy to have them on your team, because they are problem solvers. Write Most awesome engineers are well-spoken and happy to share knowledge. The best have some outlet for their thoughts: blogs, talks, open source, or a combination of those. I think there is a strong correlation between writing skills and programming. All the best engineers I know have good command over at least one human language – often more. Mastering the way you write is mastering the way you think and vice versa. A person’s writing style says so much about the way they think. If it’s confusing and lacks structure, their coding style will be too. If it’s concise, educational, well-structured, and witty at times, their code will be too. Excellent programmers find joy in playing with words. Never Stop Learning Some of the best devs I know are 60+ years old. They can run circles around me. Part of the reason is that they keep learning. If there is a new tool they haven’t tried or a language they like, they will learn it. This way, they always stay on top of things without much effort. That is not to be taken for granted: a lot of people stop learning really quickly after they graduate from University or start in their first job. They get stuck thinking that what they got taught in school is the “right” way to do things. Everything new is bad and not worth their time. So there are 25-year-olds who are “mentally retired” and 68-year-olds who are still fresh in their mind. I try to one day belong to the latter group. Somewhat related, the best engineers don’t follow trends, but they will always carefully evaluate the benefits of new technology. If they dismiss it, they can tell you exactly why , when the technology would be a good choice, and what the alternatives are. Status Doesn’t Matter The best devs talk to principal engineers and junior devs alike. There is no hierarchy. They try to learn from everyone, young and old. The newcomers often aren’t entrenched in office politics yet and still have a fresh mind. They don’t know why things are hard and so they propose creative solutions. Maybe the obstacles from the past are no more, which makes these people a great source of inspiration. Build a Reputation You can be a solid engineer if you do good work, but you can only be one of the best if you’re known for your good work; at least within a (larger) organization. There are many ways to build a reputation for yourself: Why do I think it is important to be known for your work? All of the above are ways to extend your radius of impact in the community. Famous developers impact way more people than non-famous developers. There’s only so much code you can write. If you want to “scale” your impact, you have to become a thought leader. Building a reputation is a long-term goal. It doesn’t happen overnight, nor does it have to. And it won’t happen by accident. You show up every day and do the work. Over time, the work will speak for itself. More people will trust you and your work and they will want to work with you. You will work on more prestigious projects and the circle will grow. I once heard about this idea that your latest work should overshadow everything you did before. That’s a good sign that you are on the right track. Have Patience You need patience with computers and humans. Especially with yourself. Not everything will work right away and people take time to learn. It’s not that people around you are stupid; they just have incomplete information. Without patience, it will feel like the world is against you and everyone around you is just incompetent. That’s a miserable place to be. You’re too clever for your own good. To be one of the best, you need an incredible amount of patience, focus, and dedication. You can’t afford to get distracted easily if you want to solve hard problems. You have to return to the keyboard to get over it. You have to put in the work to push a project over the finishing line. And if you can do so while not being an arrogant prick, that’s even better. That’s what separates the best from the rest. Never Blame the Computer Most developers blame the software, other people, their dog, or the weather for flaky, seemingly “random” bugs. The best devs don’t. No matter how erratic or mischievous the behavior of a computer seems, there is always a logical explanation: you just haven’t found it yet! The best keep digging until they find the reason. They might not find the reason immediately, they might never find it, but they never blame external circumstances. With this attitude, they are able to make incredible progress and learn things that others fail to. When you mistake bugs for incomprehensible magic, magic is what it will always be. Don’t Be Afraid to Say “I Don’t Know” In job interviews, I pushed candidates hard to at least say “I don’t know” once. The reason was not that I wanted to look superior (although some people certainly had that impression). No, I wanted to reach the boundary of their knowledge. I wanted to stand with them on the edge of what they thought they knew. Often, I myself didn’t know the answer. And to be honest, I didn’t care about the answer. What I cared about was when people bullshitted their way through the interview. The best candidates said “Huh, I don’t know, but that’s an interesting question! If I had to guess, I would say…” and then they would proceed to deduce the answer. That’s a sign that you have the potential to be a great engineer. If you are afraid to say “I don’t know”, you come from a position of hubris or defensiveness. I don’t like bullshitters on my team. Better to acknowledge that you can’t know everything. Once you accept that, you allow yourself to learn. “The important thing is that you don’t stop asking questions,” said Albert Einstein. Don’t Guess “In the Face of Ambiguity, Refuse the Temptation to Guess” That is one of my favorite rules in PEP 20 – The Zen of Python . And it’s so, so tempting to guess! I’ve been there many times and I failed with my own ambition. When you guess, two things can happen: Again, resist the urge to guess. Ask questions, read the reference, use a debugger, be thorough. Do what it takes to get the answer. Keep It Simple Clever engineers write clever code. Exceptional engineers write simple code. That’s because most of the time, simple is enough. And simple is more maintainable than complex. Sometimes it does matter to get things right, but knowing the difference is what separates the best from the rest. You can achieve a whole lot by keeping it simple. Focus on the right things. Final Thoughts The above is not a checklist or a competition; and great engineering is not a race. Just don’t trick yourself into thinking that you can skip the hard work. There is no shortcut. Good luck with your journey. its history: who created it? Why? To solve which problem? its present: who maintains it? Where do they work? On what? its limitations: when is the tool not a good fit? When does it break? its ecosystem: what libraries exist? Who uses it? What plugins? You built and shipped a critical service for a (larger) org. You wrote a famous tool You contribute to a popular open source tool You wrote a book that is often mentioned In the best case you’re wrong and your incorrect assumptions lead to a bug. In the worst case you are right… and you’ll never stop and second guess yourself. You build up your mental model based on the wrong assumptions. This can haunt you for a long time.

programming Programming Toml python Python go Go career Career
2 views