Pat Shaughnessy 1 weeks ago

Compiling Ruby To Machine Language

I've started working on a new edition of Ruby Under a Microscope that covers Ruby 3.x. I'm working on this in my spare time, so it will take a while. Leave a comment or drop me a line and I'll email you when it's finished. Here’s an excerpt from the completely new content for Chapter 4, about YJIT and ZJIT. I’m still finishing this up… so this content is fresh off the page! It’s been a lot of fun for me to learn about how JIT compilers work and to brush up on my Rust skills as well. And it’s very exciting to see all the impressive work the Ruby team at Shopify and other contributors have done to improve Ruby’s runtime performance. To find hot spots, YJIT counts how many times your program calls each function or block. When this count reaches a certain threshold, YJIT stops your program and converts that section of code into machine language. Later Ruby will execute the machine language version instead of the original YARV instructions. To keep track of these counts, YJIT saves an internal counter nearby the YARV instruction sequence for each function or block. Figure 4-5 shows the YARV instruction sequence the main Ruby compiler created for the sum += i block at (3) in Listing 4-1. At the top, above the YARV instructions, Figure 4-5 shows two YJIT related values: jit_entry and jit_entry_calls . As we’ll see in a moment, jit_entry starts as a null value but will later hold a pointer to the machine language instructions YJIT produces for this Ruby block. Below jit_entry , Figure 4-5 also shows jit_entry_calls , YJIT’s internal counter. Each time the program in Listing 4-1 calls this block, YJIT increments the value of jit_entry_calls . Since the range at (1) in Listing 4-1 spans from 1 through 40, this counter will start at zero and increase by 1 each time Range#each calls the block at (3). When the jit_entry_calls reaches a particular threshold, YJIT will compile the YARV instructions into machine language. By default for small Ruby programs YJIT in Ruby 3.5 uses a threshold of 30. Larger programs, like Ruby on Rails web applications, will use a larger threshold value of 120. (You can also change the threshold by passing —yjit-call-threshold when you run your Ruby program.) While compiling your Ruby program, YJIT saves the machine language instructions it creates into YJIT blocks . YJIT blocks, which are distinct from Ruby blocks, each contain a sequence of machine language instructions for a range of corresponding YARV instructions. By grouping YARV instructions and compiling each group into a YJIT block, YJIT can produce more optimized code that is tailored to your program’s behavior and avoid compiling code that your program doesn’t need. As we’ll see next, a single YJIT block doesn’t correspond to a Ruby function or block. YJIT blocks instead represent smaller sections of code: individual YARV instructions or a small range of YARV instructions. Each Ruby function or block typically consists of several YJIT blocks. Let’s see how this works for our example. After the program in Listing 4-1 executes the Ruby block at (3) 29 times, YJIT will increment the jit_entry_calls counter again, just before Ruby runs the block for the 30th time. Since jit_entry_calls reaches the threshold value of 30, YJIT triggers the compilation process. YJIT compiles the first YARV instruction getlocal_WC_1 and saves machine language instructions that perform the same work as getlocal_WC_1 into a new YJIT block: On the left side, Figure 4-6 shows the YARV instructions for the sum += i Ruby block. On the right, Figure 4-6 shows the new YJIT block corresponding to getlocal_WC_1 . Next, the YJIT compiler continues and compiles the second YARV instruction from the left side of Figure 4-7: getlocal_WC_0 at index 2. On the left side, Figure 4-7 shows the same YARV instructions for the sum += i Ruby block that we saw above in Figure 4-6. But now the two dotted arrows indicate that the YJIT block on the right contains the machine language instructions equivalent to both getlocal_WC_1 and getlocal_WC_0 . Let’s take a look inside this new block. YJIT compiles or translates the Ruby YARV instructions into machine language instructions. In this example, running on my Mac laptop, YJIT writes the following machine language instructions into this new block: Figure 4-8 shows a closer view of the new YJIT block that appeared on the right side of Figures 4-6 and 4-7. Inside the block, Figure 4-8 shows the assembly language acronyms corresponding to the ARM64 machine language instructions that YJIT generated for the two YARV instructions shown on the left. The YARV instructions on the left are: getlocal_WC_1 , which loads a value from a local variable located in the previous stack frame and saves it on the YARV stack, and getlocal_WC_0 , which loads a local variable from the current stack from and also saves it on the YARV stack. The machine language instructions on the right side of Figure 4-8 perform the same task, loading these values into registers on my M1 microprocessor: x1 and x9 . If you’re curious and would like to learn more about what the machine language instructions mean and how they work, the section “Adding Two Integers Using Machine Language” discusses the instructions for this example in more detail. Next, YJIT continues down the sequence of YARV instructions and compiles the opt_plus YARV instruction at index 4 in Figures 4-6 and 4-7. But this time, YJIT runs into a problem: It doesn’t know the type of the addition arguments. That is, will opt_plus add two integers? Or two strings, floating point numbers, or some other types? Machine language is very specific. To add two 64-bit integers on an M1 microprocessor, YJIT could use the adds assembly language instruction. But adding two floating pointer numbers would require different instructions. And, of course, adding or concatenating two strings is an entirely different operation altogether. In order for YJIT to know which machine language instructions to save into the YJIT block for opt_plus , YJIT needs to know exactly what type of values the Ruby program might ever add at (3) in Listing 4-1. You and I can tell by reading Listing 4-1 that the Ruby code is adding integers. We know right away that the sum += 1 block at (3) is always adding one integer to another. But YJIT doesn’t know this. YJIT uses a clever trick to solve this problem. Instead of analyzing the entire program ahead of time to determine all of the possible types of values the opt_plus YARV instruction might ever need to add, YJIT simply waits until the block runs and observes which types the program actually passes in. YJIT uses branch stubs to achieve this wait-and-see compile behavior, as shown in Figure 4-9. Figure 4-9 shows the YARV instructions on the left, and the YJIT block for indexes 0000-0002 on the right. But note the bottom right corner of Figure 4-7, which shows an arrow pointing down from the block to a box labeled stub. This arrow represents a YJIT branch. Since this new branch doesn’t point to a block yet, YJIT sets up the branch to point to a branch stub instead.

Performance Programming Rust Ruby

(think) 2 weeks ago

Burst-driven Development: My Approach to OSS Projects Maintenance

I’ve been working on OSS projects for almost 15 years now. Things are simple in the beginning - you’ve got a single project, no users to worry about and all the time and the focus in world. Things changed quite a bit for me over the years and today I’m the maintainer of a couple of dozen OSS projects in the realms of Emacs, Clojure and Ruby mostly. People often ask me how I manage to work on so many projects, besides having a day job, that obviously takes up most of my time. My recipe is quite simple and I refer to it as “burst-driven development”. Long ago I’ve realized that it’s totally unsustainable for me to work effectively in parallel on several quite different projects. That’s why I normally keep a closer eye on my bigger projects (e.g. RuboCop, CIDER, Projectile and nREPL), where I try to respond quickly to tickets and PRs, while I typically do (focused) development only on 1-2 projects at a time. There are often (long) periods when I barely check a project, only to suddenly decide to revisit it and hack vigorously on it for several days or weeks. I guess that’s not ideal for the end users, as some of them might feel that I “undermaintain” some (smaller) projects much of the time, but this approach has worked for me very well for quite a while. The time I’ve spent develop OSS projects has taught me that: To illustrate all of the above with some example, let me tell you a bit about copilot.el 0.3 . I became the primary maintainer of about 9 months ago. Initially there were many things about the project that were frustrating to me that I wanted to fix and improve. After a month of relatively focused work I had mostly achieved my initial goals and I’ve put the project on the backburner for a while, although I kept reviewing PRs and thinking about it in the background. Today I remembered I hadn’t done a release there in quite a while and 0.3 was born. Tomorrow I might remember about some features in Projectile that have been in the back of my mind for ages and finally implement them. Or not. I don’t have any planned order in which I revisit my projects - I just go wherever my inspiration (or current problems related the projects) take me. And that’s a wrap. Nothing novel here, but I hope some of you will find it useful to know how do I approach the topic of multi-project maintenance overall. The “job” of the maintainers is sometimes fun, sometimes tiresome and boring, and occasionally it’s quite frustrating. That’s why it’s essential to have a game plan for dealing with it that doesn’t take a heavy toll on you and make you eventually hate the projects that you lovingly developed in the past. Keep hacking! few problems require some immediate action you can’t always have good ideas for how to improve a project sometimes a project is simply mostly done and that’s OK less is more “hammock time” is important

Open Source Ruby Clojure

Pat Shaughnessy 2 weeks ago

YARV’s Internal Stack and Your Ruby Stack

I've started working on a new edition of Ruby Under a Microscope that covers Ruby 3.x. I'm working on this in my spare time, so it will take a while. Leave a comment or drop me a line and I'll email you when it's finished. The content of Chapter 3, about the YARV virtual machine, hasn't changed much since 2014. However, I did update all of the diagrams to account for some new values YARV now saves inside of each stack frame. And some of the common YARV instructions were renamed as well. I also moved some content that was previously part of Chapter 4 here into Chapter 3. Right now I'm rewriting Chapter 4 from scratch, describing Ruby's new JIT compilers. As we’ll see in a moment, YARV uses a stack internally to track intermediate values, arguments, and return values. YARV is a stack-oriented virtual machine. In addition to its own internal stack, YARV keeps track of your Ruby program’s call stack , recording which methods call which other methods, functions, blocks, lambdas, and so on. In fact, YARV is not just a stack machine—it’s a double-stack machine! It has to track the arguments and return values not only for its own internal instructions but also for your Ruby program. Figure 3-1 shows YARV’s basic registers and internal stack. YARV’s internal stack is on the left. The SP label is the stack pointer, or the location of the top of the stack. On the right are the instructions that YARV is executing. PC is the program counter, or the location of the current instruction. You can see the YARV instructions that Ruby compiled from the puts 2+2 example on the right side of Figure 3-1. YARV stores both the SP and PC registers in a C structure called rb_control_frame_t , along with the current value of Ruby’s self variable and some other values not shown here. At the same time, YARV maintains another stack of these rb_control_frame_t structures, as shown in Figure 3-2. This second stack of rb_control_frame_t structures represents the path that YARV has taken through your Ruby program, and YARV’s current location. In other words, this is your Ruby call stack—what you would see if you ran puts caller . The CFP pointer indicates the current frame pointer. Each stack frame in your Ruby program stack contains, in turn, a different value for the self, PC, and SP registers, as shown in Figure 3-1. Ruby also keeps track of type of code running at each level in your Ruby call stack, indicated by the “[BLOCK]”, “[METHOD]” notation in Figure 3-2. In order to help you understand this a bit better, here are a couple of examples. I’ll begin with the simple 2+2 example from Chapters 1 and 2, shown again in Listing 3-1. This one-line Ruby script doesn’t have a Ruby call stack, so I’ll focus on the internal YARV stack for now. Figure 3-3 shows how YARV will execute this script, beginning with the first instruction, putself . As you can see in Figure 3-3, YARV starts the program counter (PC) at the first instruction, and initially the stack is empty. Now YARV executes the putself instruction, and pushes the current value of self onto the stack, as shown in Figure 3-4. Because this simple script contains no Ruby objects or classes, the self pointer is set to the default top self object. This is an instance of the Object class that Ruby automatically creates when YARV starts. It serves as the receiver for method calls and the container for instance variables in the top-level scope. The top self object contains a single, predefined to_s method, which returns the string “main.” You can call this method by running the following command in the console: YARV will use this self value on the stack when it executes the opt_send_without_block instruction: self is the receiver of the puts method because I didn’t specify a receiver for this method call. Next, YARV executes putobject 2 . It pushes the numeric value 2 onto the stack and increments the PC again, as shown in Figure 3-5. This is the first step of the receiver (arguments) operation pattern described in “How Ruby Compiles a Simple Script” on page 34. First, Ruby pushes the receiver onto the internal YARV stack. In this example, the Fixnum object 2 is the receiver of the message/method + , which takes a single argument, also a 2. Next, Ruby pushes the argument 2, as shown in Figure 3-6. Finally, Ruby executes the + operation. In this case, opt_plus is an optimized instruction that will add two values: the receiver and the argument, as shown in Figure 3-7. As you can see in Figure 3-7, the opt_plus instruction leaves the result, 4, at the top of the stack. Now Ruby is perfectly positioned to execute the puts function call: The receiver self is first on the stack, and the single argument, 4, is at the top of the stack. (I’ll describe how method lookup works in Chapter 6.) Next, Figure 3-8 shows what happens when Ruby executes the puts method call. As you can see, the opt_send_without_block instruction leaves the return value, nil , at the top of the stack. Finally, Ruby executes the last instruction, leave , which finishes the execution of our simple, one-line Ruby program. Of course, when Ruby executes the puts call, the C code implementing the puts function will actually display the value 4 in the console output.

Programming Tutorial Ruby C

Pat Shaughnessy 3 weeks ago

Compiling a Call to a Block

I've started working on a new edition of Ruby Under a Microscope that covers Ruby 3.x. I'm working on this in my spare time, so it will take a while. Leave a comment or drop me a line and I'll email you when it's finished. This week's excerpt is from Chapter 2, about Ruby's compiler. Whenever I think about it, I'm always suprised that Ruby has a compiler like C, Java or any other programming language. The only difference is that we don't normally interact with Ruby's compiler directly. The developers who contributed Ruby's new parser, Prism, also had to rewrite the Ruby compiler because Prism now produces a completely different, redesigned abstract syntax tree (AST). Chapter 2's outline is more or less the same as it was in 2014, but I redrew all of the diagrams and updated much of the text to match the new AST nodes and other changes for Prism. Next, let’s compile my 10.times do example from Listing 1-1 in Chapter 1 (see Listing 2-2). Notice that this example contains a block parameter to the times method. This is interesting because it will give us a chance to see how the Ruby compiler handles blocks. Figure 2-13 shows the AST for the 10.times do example again. The left side of Figure 2-13 shows the AST for the 10.times function call: the call node and the receiver 10, represented by integer node. On the right, Figure 2-13 shows the beginning of the AST for the block: do |n| puts n end , represented by the block node. You can see Ruby has added a scope node on both sides, since there are two lexical scopes in Listing 2-2: the top level and the block. Let’s break down how Ruby compiles the main portion of the script shown on the left of Figure 2-13. As before, Ruby starts with the first PM_NODE_SCOPE and creates a new snippet of YARV instructions, as shown in Figure 2-14. Next, Ruby steps down the AST nodes to PM_CALL_NODE, as shown in Figure 2-15. At this point, there is still no code generated, but notice in Figure 2-13 that two arrows lead from PM_CALL_NODE : one to PM_INTEGER_NODE , which represents the 10 in the 10.times call, and another to the inner block. Ruby will first continue down the AST to the integer node and compile the 10.times method call. The resulting YARV code, following the same receiver-arguments-message pattern we saw in Figures 2-7 through 2-11, is shown in Figure 2-16. Notice that the new YARV instructions shown in Figure 2-16 push the receiver (the integer object 10) onto the stack first, after which Ruby generates an instruction to execute the times method call. But notice, too, the block in <main> argument in the send instruction. This indicates that the method call also contains a block argument: do |n| puts n end . In this example, the arrow from PM_CALL_NODE to the second PM_SCOPE_NODE has caused the Ruby compiler to include this block argument. Ruby continues by compiling the inner block, beginning with the second PM_CALL_NODE shown at right in Figure 2-13. Figure 2-17 shows what the AST for that inner block looks like. Notice Ruby inserted a scope node at the top of this branch of the AST also. Figure 2-17 shows the scope node contains two values: argc=1 and locals: [n] . These values were empty in the parent scope node, but Ruby set them here to indicate the presence of the block parameter n . From a relatively high level, Figure 2-18 shows how Ruby compiles the inner block. You can see the parent PM_NODE_SCOPE at the top, along with the YARV code from Figure 2-16. And below that Figure 2-18 shows the the inner scope node for the block, along with the YARV instructions for the block’s call to puts n . Later in this chapter we’ll learn how Ruby handles parameters and local variables, like n in this example; why Ruby generates these instructions for puts n . The key point for now is that Ruby compiles each distinct scope in your Ruby program—methods, blocks, classes, or modules, for example—into a separate snippet of YARV instructions.

Programming Java Ruby

Farid Zakaria 1 months ago

Nix derivation madness

I’ve written a bit about Nix and I still face moments where foundational aspects of the package system confounds and surprises me. Recently I hit an issue that stumped me as it break some basic comprehension I had on how Nix works. I wanted to produce the build and runtime graph for the Ruby interpreter. I have Ruby but I don’t seem to have the derivation, , file present on my machine. No worries, I think I can it and download it from the NixOS cache. I guess the NixOS cache doesn’t seem to have it. 🤷 This was actually perplexing me at this moment. In fact there are multiple discourse posts about it. My mental model however of Nix though is that I must have first evaluated the derivation (drv) in order to determine the output path to even substitute. How could the NixOS cache not have it present? Is this derivation wrong somehow? Nope. This is the derivation Nix believes that produced this Ruby binary from the database. 🤨 What does the binary cache itself say? Even the cache itself thinks this particular derivation, , produced this particular Ruby output. What if I try a different command? So I seem to have a completely different derivation, , that resulted in the same output which is not what the binary cache announces. WTF? 🫠 Thinking back to a previous post, I remember touching on modulo fixed-output derivations . Is that what’s going on? Let’s investigate from first principles. 🤓 Let’s first create which is our fixed-output derivation . ☝️ Since this is a fixed-output derivation (FOD) the produced path will not be affected to changes to the derivation beyond the contents of . Now we will create a derivation that uses this FOD. The for the output for this derivation will change on changes to the derivation except if the derivation path for the FOD changes. This is in fact what makes it “modulo” the fixed-output derivations. Let’s test this all out by changing our derivation. Let’s do this by just adding some garbage attribute to the derivation. What happens now? The path of the derivation itself, , has changed but the output path remains consistent. What about the derivation that leverages it? It also got a new derivation path but the output path remained unchanged. 😮 That means changes to fixed-output-derivations didn’t cause new outputs in either derivation but it did create a complete new tree of files. 🤯 That means in nixpkgs changes to fixed-output derivations can cause them to have new store paths for their but result in dependent derivations to have the same output path. If the output path had already been stored in the NixOS cache, then we lose the link between the new and this output path. 💥 The amount of churn that we are creating in derivations was unbeknownst to me. It can get even weirder! This example came from @ericson2314 . We will duplicate the to another file whose only difference is the value of the garbage. Let’s now use both of these in our derivation. We can now instantiate and build this as normal. What is weird about that? Well, let’s take the JSON representation of the derivation and remove one of the inputs. We can do this because although there are two input derivations, we know they both produce the same output! Let’s load this modified derivation back into our and build it again! We got the same output . Not only do we have a trait for our output paths to derivations but we can also take certain derivations and completely change them by removing inputs and still get the same output! 😹 The road to Nix enlightenment is no joke and full of dragons.

Open Source Ruby DevOps JSON

Pat Shaughnessy 1 months ago

Parsing: How Ruby Understands Your Code

I've started working on a new edition of Ruby Under a Microscope that covers Ruby 3.x. I'm working on this in my spare time, so it will take a while. Leave a comment or drop me a line and I'll email you when it's finished. Update : I’ve made a lot of progress so far this year. I had time to completely rewrite Chapters 1 and 2, which cover Ruby’s new Prism parser and the Ruby compiler which now handles the Prism AST. I also updated Chapter 3 about YARV and right now I’m working on rewriting Chapter 4 which will cover YJIT and possibly other Ruby JIT compilers. Here’s an excerpt from the new version of Chapter 1. Many thanks to Kevin Newton, who reviewed the content about Prism and had a number of corrections and great suggestions. Also thanks to Douglas Eichelberger who had some great feedback as well. I’ll post more excerpts from Chapters 2, 3 and 4 in the coming weeks. Thanks for everyone’s interest in Ruby Under a Microscope! Once Ruby converts your code into a series of tokens, what does it do next? How does it actually understand and run your program? Does Ruby simply step through the tokens and execute each one in order? No. Your code still has a long way to go before Ruby can run it. The next step on its journey through Ruby is called parsing , where words or tokens are grouped into sentences or phrases that make sense to Ruby. When parsing, Ruby takes into account the order of operations, methods, blocks, and other larger code structures. Ruby’s parsing engine defines Ruby’s syntax rules. Reading in tokens, Ruby matches the token types and the order the tokens appear with a large series of patterns. These patterns, indeed, are the heart and soul of the Ruby language. How we write a function call, how we define a method using the def keyword, how we write classes and modules - the patterns Ruby looks for define the language. Ruby’s parse algorithm has three high level steps: Let’s break down these ideas further, by following Ruby through the “Hello World” program. Afterwards, we’ll look at a second, slightly more complicated example. As we saw in the previous section, Ruby first converts the text in this code file into tokens. For Hello World, Ruby’s tokenizer produces these five tokens: To make the following diagrams simpler, let’s redraw these tokens in a more compact format: Using a single gray line of text, Figure 1-15 shows the five tokens from Figure 1-14 in a more compact format. First, PM_TOKEN_IDENTIFIER represents the word “puts” from the beginning of the program. Next, three tokens make up the string literal value: PM_TOKEN_STRING_BEGIN for the first double quote, followed by PM_TOKEN_STRING_VALUE for the words Hello and World, and PM_TOKEN_STRING_END represents the second quote. Finally, the program ends with PM_TOKEN_EOF to mark the end of the source code file. Now let’s follow Ruby as it processes the Hello World example using the three steps: identify, recurse and compare. First, identify . How does Ruby understand what the first token, PM_TOKEN_IDENTIFIER , means? Figure 1-16 represents the state of Ruby’s parser when it starts to parse this code. At this moment, Ruby is just getting started by inspecting the puts identifier. One of the patterns Ruby looks for matches the identifier; but what does this identifier mean? Ruby knows puts could be a local variable, or it could be the name of a function to call. Since there are no local variables defined in this program, Ruby determines that the puts identifier represents a function the program is calling. (It’s also possible that the program is about to create a new local variable like this: puts = "Hello World" . If that were the case, Ruby would see the assignment operator next and parse things differently.) What happens next? After matching the token to the function call pattern, Ruby records this match in a data structure called an abstract syntax tree (AST). Ruby and most other programming languages use ASTs to record the results of parsing tokens like this. As we’ll see, the AST’s tree structure is well suited for holding the nested, recursive structure of computer programs. Figure 1-17 shows the first node Ruby saves in the AST tree. In a moment, Ruby will begin to add more nodes to the AST. Before proceeding to the next token, let’s imagine the syntax pattern for a function call: Although in Ruby the parentheses are optional, so this pattern also applies: NOTE The original version of the Ruby parser used patterns or grammar rules like this directly with a tool called a parser generator. However, starting with Ruby 3.3, Ruby uses a new parser called Prism, which detects these patterns directly using hand written C code. After parsing the first token, Ruby inspects the second token. According to the function call pattern, Ruby knows the second token might represent the first argument to the function call. But, how many arguments are there? And what is each argument? The program in Listing 1-11 is very simple, but it could have instead printed a very complex expression - the arguments to puts could have run on for many lines and used hundreds of tokens. Second, recurse . To parse each of the arguments to puts, Ruby has to call itself. Figure 1-18 shows two levels of the Ruby parser’s call stack; the top line shows Ruby parsing the puts identifier token, and matching the function call pattern. The second line shows how Ruby called itself to parse the second token, PM_TOKEN_STRING_BEGIN , the leading quote of the string literal. Think of these lines as the backtrace of the Ruby parser. Figure 1-18 also shows a value 14 on the right side. While calling itself recursively, Ruby passes in a numeric value called the binding power . We’ll return to this later. Now that Ruby has called itself, Ruby starts the 3-step process all over again: identify, recurse and compare. This time, Ruby has to identify what the PM_TOKEN_STRING_BEGIN token means. This token always indicates the start of a string value. In this example PM_TOKEN_STRING_BEGIN represents the double quote that appears after puts . But the same token might represent a single quote or one of the other ways you can write a string in Ruby, for example using %Q or %q . Ruby’s new parser, Prism, next parses the string contents directly by processing the following two tokens: In this example, Ruby’s parser is done after finding the PM_TOKEN_STRING_END token and can continue to the next step. More complex strings - strings that contain interpolated values using #{} for example - might have required Ruby to call itself yet again to process more nested expressions. But for the simple "Hello World" string Ruby is done. To record the string value, Ruby creates a new AST node called PM_STRING_NODE . Figure 1-20 shows two AST nodes Ruby has created so far: the call node created earlier, and now a new string node. Ruby’s parser is a recursive descent parser . This Computer Science term describes parsers that resemble the grammar or syntax rules of the programs they parse, and call themselves recursively in a top-down manner as they process nested structures. Many modern programming languages today use this general approach. Identify : First, Ruby identifies what the next token represents. Ruby does this by comparing the token’s type - and possibly the types of the following tokens - with a large series of patterns. If one pattern matches, Ruby understands what your code means. If not, Ruby emits a syntax error. Recurse : Secondly, Ruby calls itself. Each value in one of the syntax patterns can itself be a subexpression - a smaller program that Ruby needs to parse. To do this, Ruby calls itself recursively. Compare : Third, Ruby compares the current token with the next token to determine which has a higher precedence. This comparison leads Ruby down a specific path, processing the tokens in a certain order.

Programming C Ruby

André Arko 1 months ago

We want to move Ruby forward

On September 9, without warning, Ruby Central kicked out the maintainers who have cared for Bundler and RubyGems for over a decade. Ruby Central made these changes against the established project policies , while ignoring all objections from the maintainers’ team . At the time, Ruby Central claimed these changes were “temporary". However, While we know that Ruby Central had no right to act the way they did, it is nevertheless clear to us that the Ruby community will be better off if the codebase, maintenance, and legal rights to RubyGems and Bundler are all together in the same place. To bring this about, we are prepared to transfer our interests in RubyGems and Bundler to Matz , end the dispute over the GitHub enterprise account, 2 GitHub organizations, and 70 repositories, and hand over all rights in the Bundler logo and Bundler name, including the trademark applications in the US, EU, and Japan. Once we have entered into a legal agreement to settle any legal claims with Ruby Central and transfer all rights to Matz, the former maintainers will step back entirely from the RubyGems and Bundler projects, leaving them fully and completely to Matz, and by extension to the entire Ruby community. Although Ruby Central’s actions were not legitimate, our commitment to the Ruby community remains strong. We’re choosing to focus our energy on projects to improve Ruby for everyone, including rv , Ruby Butler , jim , and gem.coop . Signed, The former maintainers: André , David , Ellen , Josef , Martin , and Samuel None of the “temporary” changes made by Ruby Central have been undone, more than six weeks later. Ruby Central still has not communicated with the removed maintainers about restoring any permissions. Ruby Central still has not offered “operator agreements” or “contributor agreements” to any of the removed maintainers. The Ruby Together merger agreement plainly states that it is the maintainers who will decide what is best for their projects, not Ruby Central. Last week, Matz stepped in to assume control of RubyGems and Bundler himself. His announcement states that the Ruby core team will assume control and responsibility for the primary RubyGems and Bundler GitHub repository. Ruby Central did not communicate with any removed maintainers before transferring control of the rubygems/rubygems GitHub repo to the Ruby core team. On October 24th, Shan publicly confirmed she does not believe the maintainers need to be told why they were removed .

Ruby Open Source

DHH 1 months ago

Success always spawns haters

As Omarchy was taking off this summer, and thousands of happy users started expressing their delight with the system, I kept waiting for the universe to balance the scales of passion. Nothing of note in this world is allowed to succeed without spawning a counteracting force of haters. And now they're finally here. The same happened twenty years ago with Ruby on Rails, but back then I still thought you could argue your way to understanding. That if you just made a logical case to counter whatever objections were raised, you'd be able to persuade most haters to change their perspective. How naive. It was Kathy Sierra who changed my perspective on this. From being annoyed by straw men and non sequiturs to accepting them and the haters as a natural consequence of success. That if you want people to love your creation, you have to accept the opposing force. Yin and yang. Here's how Kathy presented the choice: It's safe there in the gray middle. Nobody is mad at you, nobody is making any bad-faith arguments, but also, nobody cares. Lots of work exists in this zone. And that's fine. We don't need every project to reach the moon! But when escape velocity is achieved, you can't avoid drawing energy from both sides. All this isn't to say that all objections, skepticism, or criticisms come from haters. Far from it. But once sufficient success is secured, a large portion will. It's just that kind of planet, as Jim Rohn would say. The trick is to see this in aggregate as a necessary milestone. One that's even worth celebrating! Have you even made something worth cheering for, if there isn't a contingent there to boo at it too? Probably not. So embrace the boos as you embrace the cheers. They come as a pair.

Business Career Ruby

Kix Panganiban 1 months ago

Dumb Cursor is the best Cursor

I previously wrote about how I believe Cursor peaked with Cursor Compose , and that since introducing Agent mode and letting the LLM make decisions, its user experience and quality of output has subjectively gotten worse. So I tried to force Cursor to go back to its roots -- just a simple "dumb" LLM tool that can do edits and nothing else. Enter dumb mode: Essentially -- a new custom "agent" that has no access to any tools apart from edit and delete. This completely prevents it from going off the rails and diving through random areas of your codebase (or godforbid, the web) and wasting tokens and time. Using it feels like the natural extension of Cmd + K -- I choose what context to give it by manually specifying which files to edit/look at each time (just like Cursor Compose!), and because it's just Auto mode, it's quick to run and very controlled. Works exactly like the surgical tool that I was looking for. Amazing for quick or tedious edits that require no thinking or decision making -- just something that takes natural language in and code out based on what exists on the files you expose it to.

AI Programming Ruby

Chris Coyier 1 months ago

Everything is Broken

Over in the ol’ ShopTalk Discord (that’s what our Patreon thingy unlocks) our editor Chris Enns was venting about some streaming gear woes. And I said: Nothing Ever Works Chris ultimately blogged the situation and used my reply as part of the title of the blog post. Then shortly after, Jason Rodriguez’s post made the rounds in my circles: Why doesn’t anything work anymore? I’ve officially reached “old man yells at cloud” age. Same, Jason. I feel like this should be one of those viral blog post topics! Like the “good newsletters” one that went around or “why I started blogging” before that or whatever those were. Let’s make it happen people. Here’s my list from the last week or so. I was trying to log into Paramount+ on my AppleTV, but was getting some kind of unclear error. I wasn’t even sure if I had an account or not, so I tried the signup flow from my laptop. Another unclear error. Tried a different browser and the same. I just wanted to watch the Packer game and this service I either already pay or wanted to pay just wouldn’t let me. And it wasn’t the only reason I was annoyed at Paramount+ that day. I bought a Gandalf costume for Halloween for like $50. The picture has a guy, ya know, dressed up as Gandalf on the cover of the package and it looks fine. Big grey beard. Small text on the package: beard not included . What the what. I use TablePlus (which I get through SetApp ) for local database spelunking. I had some data I was trying to get at that I knew was going to be a fairly complicated query to write. It was a count of entries on a column that wasn’t the index but then I needed the index to join onto another table while having where filter and also filtering on that final count as well. I could probably reason it out, but it would probably take me an hour. So I was like: AI! Turns out TablePlus does have an AI assistant built in, so I tossed in my OpenAI API key and… You exceeded your current quota, please check your plan and billing details. Fair enough. Figure out where I can put a few bucks into my account and… I get some “unknown” error. WHY WILL NOBODY TAKE MY MONEY. I tried another browser and another credit card and turned off any “blocker” extensions I had in case of interference, but nothing worked. So I tried to use the Anthropic integration instead, and it was behaving the same. (In retrospect, it was probably the us-east-1 downtime period.) I tried the Gemini integration last, and it worked and I got my API key properly. I got my prompt together explaining exactly what I needed to do and… I am sorry, I cannot fulfill this request. The available tools lack the ability to query data or cross-reference tables. I can only retrieve metadata such as database lists, schema lists, table lists, and table creation statements. What in the what. The AI tool built into TablePlus can’t… query data? Like, wouldn’t that be the entire point of an AI assistant in a tool like this? I tried using the built-in tool rather than just going to an AI tool because I figured it would be all extra-smart, having access to the actual local database structure and stuff to use as context. I get that it might be a saftey concern (you don’t want a tool like this sending actual data over to an LLM) but that wasn’t a concern here and I didn’t need that anyway, I just needed a query that I’d run myself. Anyway I just Zoomed Marie and she helped me write the query in like 2 minutes. We brain coded it. I bought a little cheap remote control car the other day from Fred Meyer, for me and Ruby to drive around and torment her new puppy. The car took 6 AA batteries. The package came with 4 AA batteries. What in the what. Can you imagine being in the meeting where this is decided? Everybody at that table was either stupid or mean. I can’t even say greedy because someone greedy would just advocate for no batteries at all which at least is understandable. (As a consumer you’d just assume they adjust the price accordingly and you don’t have to worry about cheap junk batteries that have lost their power after sitting in a warehouse for 3 years. How far we’ve fallen.) I needed one of those like 4-cup measuring cups the other day, so grabbed a GoodCook brand one from the grocery store. After one usage and trip through the dishwasher, the markings on the side are unreadable. What in the what. Just complete garbage. Not sure why I would forsaken Pyrex , I just assumed the competition would have caught up, but apparently they have not.

AI Technology Culture Ruby

Steve Klabnik 1 months ago

I see a future in jj

In December of 2012, I was home for Christmas, reading Hacker News. And that’s when I saw “ Rust 0.5 released ."" I’m a big fan of programming languages, so I decided to check it out. At the time, I was working on Ruby and Rails, but in college, I had wanted to focus on compilers, and my friends were all very much into systems stuff. So I decided to give Rust a try. And I liked it! But, for other reasons I won’t get into here, I was thinking about a lot of things in that moment. I was looking to shake things up a bit. So I asked myself: is Rust going to be A Thing? So, I thought about it. What does a programming language need to be successful? It needs some sort of market fit. It needs to have people willing to work on it, as bringing a new language into the world is a lot of work. And it needs users. When I considered all of these things, here’s what I saw with Rust: Market fit: there was basically no credible alternatives to C and C++. I had been involved in the D community a bit, but it was clear that it wasn’t going to take off. Go was a few years old, and hit 1.0 earlier that year, but for the kinds of work that C and C++ are uniquely able to do, I saw the same problem that I did with D: garbage collection. This doesn’t mean Go isn’t a good language, or that it’s not popular, but I didn’t see it as being able to credibly challenge C and C++ in their strongholds. Rust, on the other hand, had a novel approach to these problems: memory safety without garbage collection. Now, I also need to mention that Rust back in those days was much closer to Go than it even is today, but again, I had just learned about it for a few hours, I didn’t really have a deep understanding of it yet. If I had, I actually might have also dismissed it as well, as it wasn’t really GC that was the issue, but a significant runtime. But again: I hadn’t really come to that understanding yet. Point is: low-level programming was a space where there hadn’t been much innovation in a very long time, and I thought that meant that Rust had a chance. Check. For a team: well, Mozilla was backing it. This is a big deal. It meant that there were folks whose job it was to work on the language. There’s so much that you need to do to make a new language, and that means a ton of work, which means that if you’re going to be able to get it done in a reasonable amount of time, having paid folks working on it is certainly better than the alternative. Check. And finally, how does this translate into users? Well, Mozilla was planning on using it in Firefox. This is huge. Firefox is a major project, and if they could manage to use Rust in it, that would prove that Rust was capable of doing real work. And, more importantly, it would mean that there would be a lot of folks who would need to learn Rust to work on Firefox. This would create a base of users, which would help the language grow. Check. Finally, even though it wasn’t part of my initial assessment, I just really liked the Rust folks. I had joined IRC and chatted with people, and unlike many IRC rooms, they were actually really nice. I wanted to be around them more. And if I did, other people probably would too. So that was also a plus. So, I started learning Rust. I decided to write a tutorial for it, “Rust for Rubyists,” because I’m a sucker for alliteration. And I eventually joined the team, co-authored The Book, and if you’re reading this post, you probably know the rest of the story. For some background, jj is a new version control system (VCS), not a programming language. It is written in Rust though! While I talked about how I decided to get involved with Rust above, my approach here generalizes to other kinds of software projects, not just programming languages. I have a rule of thumb: if Rain likes something, I will probably like that thing, as we have similar technical tastes. So when I heard her talk about jj, I put that on my list of things to spend some time with at some point. I was especially intrigued because Rain had worked at Meta on their source control team. So if she’s recommending something related to source control, that’s a huge green flag. It took me a while, but one Saturday morning, I woke up a bit early, and thought to myself, “I have nothing to do today. Let’s take a look at jj.” So I did . You’ll note that link goes to a commit starting a book about jj. Since it worked for me with Rust, it probably would work for me for jj as well. Writing about something really helps clarify my thinking about it, and what better time to write something for a beginner than when you’re also a beginner? Anyway, people seem to really like my tutorial, and I’m thankful for that. So, what do I see in jj? Well, a lot of it kind of eerily mirrors what I saw in Rust: a good market fit, a solid team, and a potential user base. But the market fit is interesting. Git has clearly won, it has all of the mindshare, but since you can use jj to work on Git repositories, it can be adopted incrementally. At Oxide , Rain started using jj, and more of us did, and now we’ve got a chat channel dedicated to it. This is, in my opinion, the only viable way to introduce a new VCS: it has to be able to be partially adopted. Google is using jj, and so that is a bit different than Mozilla, but the same basic idea. I have more to say about Google’s relationship to jj, but that’s going to be a follow-up blog post. What I will say in this post is that at the first ever jj conference a few weeks ago, Martin (the creator of jj) said that internal adoption is going really well. I’m burying the lede a bit here, because the video isn’t up yet, and I don’t want to get the details of some of the more exciting news incorrect in this post. I also don’t mean to imply that everyone at Google is using jj, but the contingent feels significant to me, given how hard it is to introduce a new VCS inside a company of that size. Well, in this case, it’s using Piper as the backend, so you could argue about some of the details here, but the point is: jj is being used in projects as small as individual developers and as large as one of the largest monorepos in the world. That’s a big deal. It can show the social proof needed for others to give jj a chance. Outside of Google, a lot of people say that there’s a bit of a learning curve, but once you get over that, people really like it. Sound familiar? I think jj is different from Rust in this regard in that it’s also very easy to learn if you aren’t someone who really knows a ton about Git. It’s folks that really know Git internals and have put time and care into their workflows that can struggle a bit with jj, because jj is different. But for people who just want to get work done, jj is really easy to pick up. And when people do, they often tend to like it. jj has developed a bit of a reputation for having a passionate fanbase. People are adopting it in a skunkworks way. This is a great sign for a new tool. And finally, the team. Martin is very dedicated to jj, and has been working on it for a long time. There’s also a small group of folks working on it with him. It recently moved out from his personal GitHub account to its own organization, and has started a more formal governance. The team is full of people who have a deep history of working on source control tools, and they know what they’re doing. The burgeoning jj community reminds me of that early Rust community: a bunch of nice folks who are excited about something and eager to help it grow. Basically, to me, jj’s future looks very bright. It reminds me of Rust in all of the best ways. Speaking of burying the lede… I’ve decided to leave Oxide. Oxide is the best job I’ve ever had, and I love the people I work with. I was employee 17. I think the business will do fantastic in the future, and honestly it’s a bit of a strange time to decide to leave, since things are going so well. But at the same time, some of my friends have started a new company, ERSC , which is going to be building a new platform for developer collaboration on top of jj. Don’t worry, “errssk” isn’t going to be the name of the product. It’s kind of like how GitHub was incorporated as Logical Awesome , but nobody calls it that. This won’t be happening until next month, I have some stuff to wrap up at Oxide, and I’m going to take a week off before starting. But as sad as I am to be leaving Oxide, I’m also really excited to be able to spend more time working in the jj community, and helping build out this new platform. For those of you who’ve been asking me to finish my tutorial, well, now I’ll have the time to actually do that! I’m sorry it’s taken so long! You’ll see me talking about jj even more, spending even more time in the Discord , and generally being more involved in the community. And I’ll be writing more posts about it here as well, of course. I’m really excited about this next chapter. 2025 has been a very good year for me so far, for a number of reasons, and I am grateful to be able to take a chance on something that I’m truly passionate about. Here’s my post about this post on BlueSky: I see a future in #jj-vcs : steveklabnik.com/writing/i-se... I see a future in jj Blog post: I see a future in jj by Steve Klabnik

Ruby Rust Programming C++

ptrchm 1 months ago

How to Accept Crypto Payments in Rails

I wanted to see what it would take to implement crypto payments in PixelPeeper . With Coinbase Commerce, it’s surprisingly easy. To accept crypto, you’ll need an ETH wallet address and a Coinbase Commerce account. Coinbase provides a nice hosted checkout page, where users have multiple options to pay (the funds will be converted to USDC). Upon successful payment, Coinbase will collect a small fee and the funds will be sent to your ETH address (there’s a caveat, see the last section). How does it work? Create a Charge Rails Integration The Woes of Crypto UX

Crypto Tutorial Ruby

Joel Drapper 1 months ago

The journey to the perfect type signature in Ruby

Ruby Programming Tutorial

DHH 1 months ago

A petabyte worth of Omarchy in a month

Omarchy didn't even exist before this summer. I did much of the pre-release work during the downtime between sessions at the 24 Hours of Le Mans in June. And now, just a few months later, we've delivered a petabyte of ISOs in the past thirty days alone. That's about 150,000 installs of the Omarchy Linux distribution! I've been involved with a lot of successful open-source projects in the past quarter of a century or so. Ruby on Rails, first and foremost. But nothing, not even Rails, grew as quickly as Omarchy has been growing in the first few months of its life. It's rather remarkable. This is what product-market fit looks like. Doesn't matter if the product is free or not. The fit is obvious. The stream of people who don't just enjoy Omarchy but love it is seemingly endless. The passion is palpable. But why? And why now? As per usual, there are a lot of contributing factors, but key is how Apple and Microsoft have been fumbling their relationship with people who love computers in general and developers in particular. Microsoft is killing off Windows 10, which in turn cuts off a whole slew of perfectly fine computers made prior to around 2017–2018. They also seem intent on shoving AI into everything, and wavering on whether that might be optional or not. Oh, and Windows is still Windows: decades of patching cracks in a foundation that just never was all that solid to begin with. Apple too has turned a ton of people off with macOS 26 Tahoe, liquid glass, and faltering software quality. They're also cutting off all Intel-based Macs from future updates. A Mac Mini sold as recently as 2023 is now end-of-life! This is before we even talk about how poorly the company has been treating developers depending on the App Store bureaucracy. Meanwhile, Linux has never looked better. Hyprland, the tiling window manager at the heart of Omarchy, is a sensation. It's brought an incredible level of finesse, detail, and style to the tiling window management space: superb animations, lightning-fast execution, and super-light resource consumption. The historic gap in native GUI apps has never mattered less either. The web has conquered all as the dominant computing platform. In the past, missing, say, Photoshop was a big deal. Now it's Figma — a web app! — that's driving designers. Same too with tools like Microsoft Office or Outlook, which are all available on the web. I'm not saying there aren't specialized apps that some people simply can't do without, that keep them trapped on Windows or Mac. But I am saying that they've never been fewer. Almost everything has a great web alternative. And for developers, the fact is that Linux was always a superior platform in terms of performance and tooling for most programming environments. With 95% of the web running on Linux servers, all optimization and tuning needed to get the most out of the hardware was done with Linux in mind. This is why even a $500 Beelink Mini PC is competitive with an M4 Max machine costing thousands of dollars for things like our HEY test suite, which runs on Ruby and MySQL. Linux is just really efficient and really fast. Finally, I think the argument that owning your computer, fully and deeply, is starting to resonate. The Free Software crowd has been making the argument since the 90s, if not before, but it's taken Apple's and Microsoft's recent tightening of the reins on our everyday operating systems to make it relevant for most. Omarchy is a beautiful, modern, and opinionated Linux distribution, but it's also yours. Everything is preconfigured, sure, but every configuration is also changeable. Don't like how something works? Change it. Don't like the apps I use? Change them. Don't like how something looks? Redesign it. The level of agency is off the charts. Turns out that plenty of people were starved for just this. All it took was someone to actually put all the pieces together, ignore the Linux neckbeards who insist you aren't worthy to run Arch or Hyprland without spending a hundred hours setting it up from scratch, and invite everyone to the party!

Open Source Linux Ruby

André Arko 1 months ago

Announcing <code>rv</code> 0.2

With the help of many new contributors, and after many late nights wrestling with make, we are happy to (slightly belatedly) announce the 0.2 release of rv ! This version dramatically expands support for Rubies, shells, and architectures. Rubies: we have added Ruby 3.3, as well as re-compiled all Ruby 3.3 and 3.4 versions with YJIT. On Linux, YJIT increases our glibc minimum version to 2.35 or higher. That means most distro releases from 2022 or later should work, but please let us know if you run into any problems. Shells: we have added support for bash, fish, and nushell in addition to zsh. Architectures: we have added Ruby compiled for macOS on x86, in addition to Apple Silicon, and added Ruby compiled for Linux on ARM, in addition to x86. Special thanks to newest member of the maintainers’ team @adamchalmers for improving code and tests, adding code coverage and fuzzing, heroic amounts of issue triage, and nushell support. Additional thanks are due to all the new contributors in version 0.2, including @Thomascountz , @lgarron , @coezbek , and @renatolond . To upgrade, run , or check the release notes for other options.

Ruby Programming Open Source Bash

André Arko 1 months ago

The RubyGems “security incident”

Ruby Central posted an extremely concerning “ Incident Response Timeline ” today, in which they make a number of exaggerated or purely misleading claims. Here’s my effort to set the record straight. First, and most importantly: I was a primary operator of RubyGems.org, securely and successfully, for over ten years. Ruby Central does not accuse me of any harms or damages in their post, in fact stating “we have no evidence to indicate that any RubyGems.org data was copied or retained by unauthorized parties, including Mr. Arko.” The actions I took during a time of great confusion and uncertainty (created by Ruby Central!) were careful, specific, and aimed to defend both Ruby Central the organization and RubyGems.org the service from potential threats. The majority of the team, including developers in the middle of paid full-time work for Ruby Central, had just had all of their permissions on GitHub revoked. And then restored six days later. And then revoked again the next day. Even after the second mass-deletion of team permissions, Marty Haught sent an email to the team within minutes, at 12:47pm PDT, saying he was (direct quote) “terribly sorry” and “I messed up”. Update : Added email timestamp. The erratic and contradictory communication supplied by Marty Haught, and the complete silence from Shan and the board, made it impossible to tell exactly who had been authorized to take what actions. As this situation occurred, I was the primary on-call. My contractual, paid responsibility to Ruby Central was to defend the RubyGems.org service against potential threats. Marty’s final email clearly stated “I’ll follow up more on this and engage with the governance rfc in good faith.”. Just a few minutes after that email, at 1:01pm PDT, Marty also posted a public GitHub comment , where he agreed to participate in the proposed governance process and stated “I’m committed to find the right governance model that works for us all. More to come.” Update : screenshot of comment removed and replaced with link, since the comment appears to still be visible (at least to logged out users) on GitHub. Given Marty’s claims, the sudden permission deletions made no sense. Worried about the possibility of hacked accounts or some sort of social engineering, I took action as the primary on-call engineer to lock down the AWS account and prevent any actions by possible attackers. I did not change the email addresses on any accounts, leaving them all owned by a team-shared email at rubycentral.org, to ensure the organization retained overall control of the accounts, even if individuals were somehow taking unauthorized actions. Within a couple of days, Ruby Central made an (unsigned) public statement, and various board members agreed to talk directly to maintainers. At that point, I realized that what I thought might have been a malicious takeover was both legitimate and deliberate, and Marty would never “fix the permissions structure”, or “follow up more” as he said. Once I understood the situation, I backed off to let Ruby Central take care of their “security audit”. I left all accounts in a state where they could recover access. I did not alter, or try to alter, anything in the Ruby Central systems or GitHub repository after that. I was confident, at the time, that Ruby Central’s security experts would quickly remove all outside access. My confidence was sorely misplaced. Almost two weeks later, someone asked if I still had access and I discovered (to my great alarm), that Ruby Central’s “security audit” had failed. Ruby Central also had not removed me as an “owner” of the Ruby Central GitHub Organization. They also had not rotated any of the credentials shared across the operational team using the RubyGems 1Password account. I believe Ruby Central confused themselves into thinking the “Ruby Central” 1Password account was used by operators, and they did revoke my access there. However, that 1Password account was not used by the open source team of RubyGems.org service operators. Instead, we used the “RubyGems” 1Password account, which was full of operational credentials. Ruby Central did not remove me from the “RubyGems” 1Password account, even as of today. Aware that I needed to disclose this surprising access, but also aware that it was impossible for anyone except former operators to exploit this security failure, I immediately wrote an email to Ruby Central to disclose the problem. Here is a copy of my disclosure email, in full. Ruby Central did not reply to this email for over three days. When they finally did reply, they seem to have developed some sort of theory that I was interested in “access to PII”, which is entirely false. I have no interest in any PII, commercially or otherwise . As my private email published by Ruby Central demonstrates, my entire proposal was based solely on company-level information, with no information about individuals included in any way. Here’s their response, over three days later. In addition to ignoring the (huge) question of how Ruby Central failed to secure their AWS Root credentials for almost two weeks, and appearing to only be aware of it because I reported it to them , their reply also failed to ask whether any other shared credentials might still be valid. There were more. Unbeknownst to me, while I was answering Marty’s email in good faith, Ruby Central’s attorney was sending my lawyer a letter alleging I had committed a federal crime, on the theory that I had “hacked” Ruby Central’s AWS account. On the contrary, my actions were taken in defense of the service that Ruby Central was paying me to support and defend. With my side of the story told, I’ll leave it to you to decide whether you think it’s true that “Ruby Central remains committed to transparent, responsible stewardship of the RubyGems infrastructure and to maintaining the security and trust that the Ruby ecosystem depends on.”

Security Ruby

マリウス 1 months ago

Alpine Linux on a Bare Metal Server

When I began work on 📨🚕 ( MSG.TAXI ) , I kept things deliberately low-key, since I didn’t want it turning into a playground for architecture astronauts . For the web console’s tech stack, I went with the most boring yet easy-to-master CRUD stack I could find , that doesn’t depend on JavaScript . And while deploying Rails in a sane way (without resorting to cOnTaInErS ) is a massive PITA, thanks to the Rails author’s cargo-cult mentality and his followers latching onto every half-baked wannabe-revolutionary idea, like Kamal and more recently Omarchy , as if it were a philosopher’s stone, from a development perspective it’s still the most effective getting-shit-done framework I’ve used to date. Best of all, it doesn’t rely on JavaScript (aside from actions, which can be avoided with a little extra effort). Similarly, on the infrastructure side, I wanted a foundation that was as lightweight as possible and wouldn’t get in my way. And while I’m absolutely the type of person who would run a Gentoo server, I ultimately went with Alpine Linux due to its easier installation, relatively sane defaults (with a few exceptions, more on that later ), and its preference for straightforward, no-nonsense tooling that doesn’t try to hide magic behind the scenes. “Why not NixOS?” you might ask. Since I’m deploying a lightweight, home-brewed Ruby/Rails setup alongside a few other components, I didn’t see the point of wrapping everything as Nix packages just to gain the theoretical benefits of NixOS. In particular, the CI would have taken significantly longer, while the actual payoff in my case would have been negligible. Since I’m paying for 📨🚕 out of my own pocket, I wanted infrastructure that’s cheap yet reliable. With plenty of people on the internet praising Hetzner , I ended up renting AMD hardware in one of their Finnish datacenters. Hetzner doesn’t offer as many Linux deployment options as cloud providers like Vultr , so I had to set up Alpine myself, which was pretty straightforward. To kickstart an Alpine installation on a Hetzner system, you just need to access the server’s iPXE console, either by renting a Hetzner KVM for an hour or by using their free vKVM feature. From there, you can launch the Alpine Linux by initializing the network interface and chain-loading the file: From that point on setup should be easy thanks to Alpine’s installer routine. If you’re using Hetzner’s vKVM feature to install Alpine, this chapter is for you. Otherwise, feel free to skip ahead. vKVM is a somewhat hacky yet ingenious trick Hetzner came up with, and it deserves a round of applause. If you’re curious about how it works under the hood, rent a real KVM once and reboot your server into vKVM mode. What you’ll see is that after enabling vKVM in Hetzner’s Robot , iPXE loads a network image, which boots a custom Linux OS. Within that OS, Hetzner launches a QEMU VM that uses your server’s drives to boot whatever you have installed. It’s basically Inception at the OS level. As long as vKVM is active (meaning the iPXE image stays loaded), your server is actually running inside this virtualized environment, with display output forwarded to your browser. Run while in vKVM mode and you’ll see, for example, your NIC showing up as a VirtIO device. Here’s the catch: When you install Alpine through this virtualized KVM environment, it won’t generate the your physical server actually needs. For instance, if your server uses an NVMe drive, you may discover that doesn’t include the module, causing the OS to fail on boot. Hetzner’s documentation doesn’t mention this, and it can easily bite you later. Tl;dr: If you installed your system via vKVM , make sure your includes all necessary modules. After updating , regenerate the . There are several ways to do this, but I prefer . Always double-check that the regenerated really contains everything you need. Unfortunately Alpine doesn’t provide tools for this, so here’s a .tar.gz with Debian’s and . Extract it into , and note that you may need to for them to work properly, due to Alpine’s somewhat annoying defaults (more on that later ). Finally, after rebooting, make sure you’ve actually left the vKVM session. You can double check by running . If the session is still active (default: 1h), your system may have just booted back into the VM, which you can identify by its Virt-devices. As soon as your Alpine Linux system is up and running there are a couple of things that I found important to change right off the bat. Alpine’s default boot timeout is just 1 second, set in ( ). If you ever need to debug a boot-related issue over a high-latency KVM connection, you will dread that 1-second window. I recommend increasing it to 5 seconds and running to apply the change. In practice, you hopefully won’t be rebooting the server that often, so the extra four seconds won’t matter day-to-day. Alpine uses the classic to configure network settings. On Hetzner’s dedicated servers, you can either continue using DHCP for IPv4 or set the assigned IP address statically. For IPv6, you’ll be given a subnet from which you can choose your own address. Keep in mind that the first usable IPv6 on Hetzner’s dedicated servers is : Amongst the first things you do should be disabling root login and password authentication via SSH: Apart from that you might want to limit the type of key exchange methods and algorithms that your SSH server allows, depending on the type of keys that you’re using. Security by obscurity: Move your SSH server from its default port (22) to something higher up and more random to make it harder for port-scanners to hit it. Finicky but more secure: Implement port knocking and use a handy client to open the SSH port for you only, for a limited time only. Secure: Set up a small cloud instance to act as Wireguard peer and configure your server’s SSH port to only accept connections from the cloud instance using a firewall rule . Use Tailscale if a dedicated Wireguard instance is beyond your expertise. You will likely want to have proper (GNU) tools around, over the defaults that Alpine comes with ( see below ). Some of the obvious choices include the following: In addition, I also like to keep a handful of convenience tools around: This is a tricky part because everyone’s monitoring setup looks different. However, there are a few things that make sense in general. Regardless what you do with your logs it’s generally a good idea to switch from BusyBox to something that allows for more advanced configurations, like syslog-ng : You probably should have an overview of how your hardware is doing. Depending on what type of hard drives your server has, you might want to install the or packages. UFW is generally considered an uncomplicated way to implement firewalling without having to complete a CCNP Security certification beforehand: Depending on your SSH setup and whether you are running any other services that could benefit from it, installing Fail2Ban might make sense: The configuration files are located at and you should normally only create/edit the files. The easiest way to backup all the changes that you’ve made to the general configuration is by using , the integrated Alpine local backup solution that was originally intended as a tool to manage diskless mode installations. I would, however, recommend to manually back up installed packages ( ) and use Restic for the rest of the system, including configuration files and important data, e.g.: However, backups depend on the data that your system produces and your desired backup target. If you’re looking for an easy to use, hosted but not-too-expensive one-off option, then Tarsnap might be for you. You should as well look into topics like local mail delivery, system integrity checks (e.g. AIDE ) and intrusion detection/prevention (e.g. CrowdSec ). Also, if you would like to get notified for various server events, check 📨🚕 ( MSG.TAXI ) ! :-) One of the biggest annoyances with Alpine is BusyBox : You need SSH? That’s BusyBox. The logs? Yeah, BusyBox. Mail? That’s BusyBox, too. You want to untar an archive? BusyBox. What? It’s gzipped? Guess what, you son of a gun, gzip is also BusyBox. I understand why Alpine chose BusyBox for pretty much everything, given the context that Alpine is most often used in ( cOnTaInErS ). Unfortunately, most BusyBox implementations are incomplete or incompatible with their full GNU counterparts, leaving you wondering why something that worked flawlessly on your desktop Linux fails on the Alpine box. By the time I finished setting up the server, there was barely any BusyBox tooling left. However, I occasionally had to resort to some odd trickery to get things working. You now have a good basis to set up whatever it is that you’re planning to use the machine for. Have fun! Footnote: The artwork was generated using AI and further botched by me using the greatest image manipulation program .

JavaScript Linux Ruby DevOps

ptrchm 1 months ago

Event-driven Modular Monolith

The main Rails app I currently work on has just turned eight. It’s not a huge app. It doesn’t deal with web-scale traffic or large volumes of data. Only six people working on it now. But eight years of pushing new code adds up. This is a quick overview of some of the strategies we use to keep the codebase maintainable. After the first few years, our codebase suffered from typical ailments: tight coupling between domains, complex database queries spread across various parts of the app, overgrown models, a maze of side effects triggered by ActiveRecord callbacks , endlessly chained associations (e.g. ) – with an all-encompassing model sitting on top of the pile. Modular Monolith Pub/Sub (Events) Patterns Service Objects Repositories for Database Queries Slim and Dumb Models Bonus: A Separate Frontend App How Do I Start?

UI AI Ruby

André Arko 1 months ago

Announcing gem.coop, a community gem server

The team behind the last ten years of rubygems.org, including @deivid-rodriguez, @duckinator, @martinemde, @segiddins, @simi, and myself, is very pleased to announce a new gem server for the Ruby community: gem.coop . The new server’s governance policies are being prepared in coordination with Mike McQuaid of Homebrew, and will be released later this week. The current versions of RubyGems and Bundler work with this new server already, and any Ruby developer is welcome to switch to using this new server immediately. We have exciting plans to add new features and functionality in the coming days. Join us!

Open Source Ruby

Robin Moffatt 1 months ago

Stumbling into AI: Part 5—Agents

A short series of notes for myself as I learn more about the AI ecosystem as of Autumn [Fall] 2025. The driver for all this is understanding more about Apache Flink’s Flink Agents project, and Confluent’s Streaming Agents . I started off this series —somewhat randomly, with hindsight—looking at Model Context Protocol ( MCP ) . It’s a helper technology to make things easier to use and provide a richer experience. Next I tried to wrap my head around Models —mostly LLMs, but also with an addendum discussing other types of model too. Along the lines of MCP, Retrieval Augmented Generation ( RAG ) is another helper technology that on its own doesn’t do anything but combined with an LLM gives it added smarts. I took a brief moment in part 4 to try and build a clearer understanding of the difference between ML and AI . So whilst RAG and MCP combined make for a bunch of nice capabilities beyond models such as LLMs alone, what I’m really circling around here is what we can do when we combine all these things: Agents ! But…what is an Agent, both conceptually and in practice? Let’s try and figure it out. Let’s begin with Wikipedia’s definition : In computer science, a software agent is a computer program that acts for a user or another program in a relationship of agency . We can get more specialised if we look at Wikipedia’s entry for an Intelligent Agent : In artificial intelligence, an intelligent agent is an entity that perceives its environment, takes actions autonomously to achieve goals , and may improve its performance through machine learning or by acquiring knowledge . Citing Wikipedia is perhaps the laziest ever blog author’s trick, but I offer no apologies 😜. Behind all the noise and fuss, this is what we’re talking about: a bit of software that’s going to go and do something for you (or your company) autonomously . LangChain have their own definition of an Agent, explicitly identifying the use of an LLM: An AI agent is a system that uses an LLM to decide the control flow of an application. The blog post from LangChain as a whole gives more useful grounding in this area and is worth a read. In fact, if you want to really get into it, the LangChain Academy is free and the Introduction to LangGraph course gives a really good primer on Agents and more. Meanwhile, the Anthropic team have a chat about their definition of an Agent . In a blog post Anthropic differentiates between Workflows (that use LLMs) and Agents: Workflows are systems where LLMs and tools are orchestrated through predefined code paths. Agents, on the other hand, are systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks. Independent researcher Simon Willison also uses the LLM word in his definition: An LLM agent runs tools in a loop to achieve a goal. He explores the definition in a recent blog post: I think “agent” may finally have a widely enough agreed upon definition to be useful jargon now , in which Josh Bickett’s meme demonstrates how much of a journey this definition has been on: That there’s still discussion and ambiguity nearly two years after this meme was created is telling. My colleague Sean Falconer knows a lot more this than I do. He was a guest on a recent podcast episode in which he spells things out: [Agentic AI] involves AI systems that can reason, dynamically choose tasks, gather information, and perform actions as a more complete software system. [ 1 ] [Agents] are software that can dynamically decide its own control flow: choosing tasks, workflows, and gathering context as needed. Realistically, current enterprise agents have limited agency[…]. They’re mostly workflow automations rather than fully autonomous systems . [ 2 ] In many ways […] an agent [is] just a microservice . [ 3 ] A straightforward software Agent might do something like: Order more biscuits when there are only two left The pseudo-code looks like this: We take this code, stick it on a server and leave it to run. One happy Agent, done. An AI Agent could look more like this: Other examples of AI Agents include: Coding Agents . Everyone’s favourite tool (when used right). It can reason about code, it can write code, it can review PRs. One of the trends that I’ve noticed recently (October 2025) is the use of Agents to help with some of the up-front jobs in software engineering (such as data modelling and writing tests ), rather than full-blown code that’s going to ship to production. That’s not to say that coding Agents aren’t being used for that, but by using AI to accelerate certain tasks whilst retaining human oversight (a.k.a. HITL ) it makes it easier to review the output rather than just trusting to luck that reams and reams of code are correct. There’s a good talk from Uber on how they’re using AI in the development process, including code conversion, and testing. Travel booking . Perhaps you tell it when you want to go, the kind of vacation you like, and what your budget is; it then goes and finds where it’s nice at that time of year, figures out travel plans within your budget, and either proposes an itinerary or even books it for you. Another variation could be you tell it where , and then it integrates with your calendar to figure out the when . This is a canonical example that is oft-cited; I’d be interested if anyone can point me to an actual implementation of it, even if just a toy one . I saw this in a blog post from Simon Willison that made me wince, but am leaving the above in anyway just to serve as an example of the confusion/hype that exists in this space: comes from plus , the latter meaning of, relating to, or characterised by . So is simply AI that is characterised by an Agent, or Agency. Contrast that to AI that’s you sat at the ChatGPT prompt asking it to draw pictures of a duck dressed as a clown . Nothing Agentic about that—just a human-led and human-driven interaction. "AI Agents" becomes a bit of a mouthful with the qualifier, so much of the current industry noise is simply around "Agents". That said, "Agentic AI" sounds cool, so gets used as the marketing term in place of "AI" alone. So we’ve muddled our way through to some kind of understanding of what an Agent is, and what we mean by Agentic AI. But how do we actually build one? All we need is an LLM (such as access to the API for OpenAI or Claude ), something to call that API (there are worse choices than !), and a way to call external services (e.g. MCP servers) if the LLM determines that it needs to use them. So in theory we could build an Agent with some lines of bash, some API calls, and a bunch of sticky-backed plastic . This is a grossly oversimplified example (and is missing elements such as memory)—but it hopefully illustrates what we’re building at the core of an Agent. On top of this goes all the general software engineering requirements of any system that gets built (suitable programming language and framework, error handling, LLM output validation, guard rails, observability, tests, etc etc). The other nuance that I’ve noticed is that whilst the above simplistic diagram is 100% driven by an LLM (it decides what tools to call, it decides when to iterate) there are plenty of cases where an Agent is to some degree rules-driven. So perhaps the LLM does some of the autonomous work, but then there’s a bunch of good ol' statements in there too. This is also borne out by the notion of "Workflows" when people talk about Agents. An Agent doesn’t wake up in the morning and set out on its day serving only to fulfill its own goals and enrichment. More often than not an Agent is going to be tightly bound into a pre-defined path with a limited range of autonomy. What if you want to actually build this kind of thing for real? That’s where tools like LangGraph and LangChain come in. Here’s a notebook with an example of an actual Agent built with these tools. LlamaIndex is another framework, with details of building an Agent in their docs. As we build up from the so-simple-it-is-laughable strawman example of an Agent above, one of the features we’ll soon encounter is the concept of memory. The difference between a crappy response and a holy-shit-that’s-magic response from an LLM is often down to context . The richer the context, the better a chance it has at generating a more accurate output. So if an Agent can look back on what it did previously, determining what worked well and what didn’t, perhaps even taking into account human feedback, it can then generate a more successful response the next time. You can read a lot more about memory in this chapter of Agentic Design Patterns by Antonio Gulli . This blog post from "The BIG DATA guy" is also useful: Agentic AI, Agent Memory, & Context Engineering This diagram from Generative Agents: Interactive Simulacra of Human Behavior (J.S. Park, J.C. O’Brien, C.J. Cai, M.R. Morris, P. Liang, M.S. Bernstein) gives a good overview of a much richer definition of an Agent’s implementation. The additional concepts include memory (discussed briefly above), planning, and reflection: Also check out Paul Iusztin’s talk from QCon London 2025 on The Data Backbone of LLM Systems . Around the 35-minute mark he goes into some depth around Agent architectures. Just as you can build computer systems as monoliths (everything done in one place) or microservices (multiple programs, each responsible for a discrete operation or domain), you can also have one big Agent trying to do everything (probably not such a good idea) or individual Agents each good at their particular thing that are then hooked together into what’s known as a Multi-Agent System (MAS). Sean Falconer’s family meal planning demo is a good example of a MAS. One Agent plans the kids' meals, one the adults' meals, another combines the two into a single plan, and so on. This is a term you’ll come across referring to the fact that Agents might be pretty good, but they’re not infallible. In the travel booking example above, do we really trust the Agent to book the best holiday for us? Almost certainly we’d want—at a minimum—the option to sign off on the booking before it goes ahead and sinks £10k on an all-inclusive trip to Bognor Regis. Then again, we’re probably happy enough for an Agent to access our calendars without asking permission, and as to whether they need permission or not to create a meeting is up to us and how much we trust them. When it comes to coding, having an Agent write code, test it, fix the broken tests, compare it to a spec, and iterate is really neat. On the other hand, letting it decide to run …less so 😅. Every time an Agent requires HITL, it reduces its autonomy and/or responsiveness to situations. As well as simply using smarter models that make fewer mistakes, there are other things that an Agent can do to reduce the need for HITL such as using guardrails to define acceptable parameters. For example, an Agent is allowed to book travel but only up to a defined threshold. That way the user gets to trade off convenience (no HITL) with risk (unintended first-class flight to Hawaii). 📃 Generative Agents: Interactive Simulacra of Human Behavior 🎥 Paul Iusztin - The Data Backbone of LLM Systems - QCon London 2025 📖 Antonio Gulli - Agentic Design Patterns 📖 Sean Falconer - https://seanfalconer.medium.com/ Coding Agents . Everyone’s favourite tool (when used right). It can reason about code, it can write code, it can review PRs. One of the trends that I’ve noticed recently (October 2025) is the use of Agents to help with some of the up-front jobs in software engineering (such as data modelling and writing tests ), rather than full-blown code that’s going to ship to production. That’s not to say that coding Agents aren’t being used for that, but by using AI to accelerate certain tasks whilst retaining human oversight (a.k.a. HITL ) it makes it easier to review the output rather than just trusting to luck that reams and reams of code are correct. There’s a good talk from Uber on how they’re using AI in the development process, including code conversion, and testing. Travel booking . Perhaps you tell it when you want to go, the kind of vacation you like, and what your budget is; it then goes and finds where it’s nice at that time of year, figures out travel plans within your budget, and either proposes an itinerary or even books it for you. Another variation could be you tell it where , and then it integrates with your calendar to figure out the when . This is a canonical example that is oft-cited; I’d be interested if anyone can point me to an actual implementation of it, even if just a toy one . I saw this in a blog post from Simon Willison that made me wince, but am leaving the above in anyway just to serve as an example of the confusion/hype that exists in this space: 📃 Generative Agents: Interactive Simulacra of Human Behavior 🎥 Paul Iusztin - The Data Backbone of LLM Systems - QCon London 2025 📖 Antonio Gulli - Agentic Design Patterns 📖 Sean Falconer - https://seanfalconer.medium.com/

AI Bash Ruby