Latest Posts (3 found)
flak 1 months ago

backporting go on openbsd

The OpenBSD ports tree generally tracks current, but sometimes backports (and stable packages) are made for more serious issues. As was the case for git 2.50.1. However, the go port has not seen a backport in quite some time. The OpenBSD release schedule aligns with the go schedule such that we always get the latest release, but not minor revisions. For instance, OpenBSD 7.7 shipped with go 1.24.1, but there’s a few minor revisions after that. We maybe don’t care about many of these backports, but issue 73570 is a backported fix for a bug specific to OpenBSD, so let’s say we want that. I always forget the procedures for building ports from scratch and waste a bunch of time running and cancelling and rerunning commands. So here’s a recipe that worked. If we don’t have the ports tree, we need to get that. If we don’t have bash, we need to install that. (There’s a magic formula to have ports install packages, but this is simpler.) We need to update the go port to a suitable revision. The port is currently on 1.25, but I’d rather stick with 1.24, so we go back a little ways. The OpenBSD port was never updated for 1.24.7, but those changes don’t look very exciting. Maybe next time I’ll try a custom update to a new version. We build the port. The bootstrap flavor is important, or we’ll end up building it twice. Tick, tock, ding, ding. Running will build and install a package. Check. Looks good. redux What if we want a version that’s not in ports? I figured this post would be pretty boring, but go just released 1.24.8, which includes security fixes I’d like, so now we definitely need to try building a new version. Let’s edit the Makefile . Now tell the ports system to download the new version and update the checksum. This downloads the new version and prints it’s checksum. Okay? Well, the go downloads page shows checksums in hex, but we can redo it to check. Looks good. Now run and again. Uh oh. Fucking FIPS, every fucking time. I don’t want to think too much about what this is doing, but the file has been renamed, so we need to update the pkg/PLIST file. Hopefully this is an aberration, as the go team is usually conservative about backporting changes, but one never knows what to expect. Now the package builds and installs correctly. And then rebuild everything that uses go.

go Go open-source Open Source Bash
0 views
flak 2 months ago

async dns

curl experimented with using pthread_cancel to timeout async DNS requests and it blew up. What else can we do? Out of curiosity, I decided to review some alternatives and see how they work. My personal priorities are control over events; no background threads or signals or secret mechanisms. getaddrinfo The tried and true classic technique is to call getaddrinfo in a thread. Probably with more than one thread so you don’t get stuck behind a single slow request, but probably not boundless either. You can also use a separate process if you don’t use threads. This is probably good enough for many uses. getaddrinfo_a glibc provides getaddrinfo_a which basically does the thread dance for you. Some of it. It comes with some caveats, and it’s distinctly non portable, and probably doesn’t mesh with your idea of an event loop. Passing. c-ares c-ares is a standalone DNS library. It supports async queries via a threaded backend or an event driven system. I think the thread backend has the same issues, in that it uses a callback and then you need to push the results back into your application. Alas, the event system uses lots of callbacks as well. This also includes some dire warnings in the documentation. “When the associated callback is called, it is called with a channel lock so care must be taken to ensure any processing is minimal to prevent DNS channel stalls.” Everyone knows the ideal callback just sets a flag, etc., but also everyone is inevitably tempted to do just one more thing, and hey look, it works fine, wait, why did it break. And thus I have a strong preference for library interfaces where you call into it, get some results, but any time you’re in your own code, you’re free to do what you want. But worth a try. Based on the sample code I wrote the quickest dirtiest demo I could. c-ares code It’s okay, but the callbacks are annoying. Notifying me which descriptors need watching means I’m required to pack up my poll structure so I can access it in the callbacks, etc. Everything gets bound just a little bit tighter. wadns Among the alternatives the c-ares project helpfully lists, is dns.c. This sounds enticing. On the downside, it’s not clear where the demo code stops and the functional code begins. As in, there’s a getaddrinfo sample, but it incorporates a lot of other code that doesn’t seem to be public. The public header doesn’t actually expose a means to interface with an event loop. The code is meant to be integrated into a project, which is understandable and even advantageous, but it means no demo today. asr The asr code was written for smtpd in OpenBSD. It doesn’t use threads and requires the caller to push events. Unfortunately, a portable version currently only exists in the OpenSMTPD repo. On the plus side, it’s used as the basis for the libc resolver in OpenBSD, which means the “sample” code to replace getaddrinfo literally is getaddrinfo.c. I rewrote the c-ares demo to use asr. It comes out quite a bit shorter, and I think clearer as well. asr code I like this API. It’s very much like read or write in that it either gives you an answer, or tells you to come back later, and then it’s up to you to decide when that is.

programming Programming performance Performance
0 views
flak 2 months ago

slice tails don't grow forever

In a garbage collected language, one needs to be cautious about leaks caused by errant references, but caution can turn to paranoia. In go, I like to make little fifo queues by appending to the tail of a slice. But does this grow forever? One theory is that this will eventually consume all memory, as somehow the slice grows and grows. The garbage collector is unable to reduce its size because the head of the slice is still out there somewhere. Another theory is no, the slice gets reallocated when it needs to expand, but only the referenced section gets copied. We can test this! This runs basically forever, but uses very little memory. Changing grow to true, it reliably explodes. I knew this, and was already relying on it, but also in most cases a slow leak may take weeks to develop and go unnoticed. Thirty seconds to check.

performance Performance go Go
0 views