Posts in Data-visualization (20 found)
Stone Tools 3 days ago

Lotus Improv on the NeXT

It is spoken of in reverent tones, invoked like a magic spell, whispered about as one of the mythical "roads not taken" in computing history. Its proponents write glowing reports and fawning YouTube reviews, lamenting its failure in the historical marketplace, each offering some justification for it not having set the world ablaze. It is safe to say, the legend of Lotus Improv persists to this day. If we put ourselves in Steve Jobs's shoes circa 1988, it's easy to see why he would have been smitten by an early preview of what we now call a "pivot table." NeXT was struggling to find its market, and exclusives which justified a system purchase were sorely needed. "Maybe Improv can do for NeXT what VisiCalc did for the Apple II," must surely have crossed Jobs's mind. In making it a NeXT exclusive, Improv seemed to be a clear-eyed, forward-looking answer to the question dogging the company, posed by the inaugural issue of NeXTWORLD Magazine , "Why NeXT?" Both NeXT and Improv brought innovative solutions to their respective computing questions of the day. NeXT made UNIX palatable to those scared of the terminal, debuted Display Postscript on a mega-pixel display, and made development for the system a snap. Improv rethought multi-dimensional data modeling, offered English-like formulas, and made it easy to rearrange data views on a whim. The bones of both remain to this day. NeXT's operating system, NeXTSTEP / OPENSTEP, morphed into macOS, and by extension iOS. Lotus abandoned Improv fairly quickly, but a quick survey of today's spreadsheets shows a universal embrace of pivot tables. Even today, it feels like nothing else, and still does data tricks no other spreadsheet can do as elegantly. Given its mythos, I feel a kind of nerd obligation to love it. Events have converged to make this the right time to finally talk about Lotus Improv . First, it is time once again for "International Spreadsheet Day." Second, as Improv was initially a NeXT exclusive, it seems only appropriate to take a look at that machine here on the 15th (!) anniversary of the passing of Steve Jobs. Third, and not to toot my own horn too loudly, it is the 1st anniversary of this very blog. Lotus Improv was one of four pieces of software I had in mind when deciding to start Stone Tools. In fact, it was quite literally the software that first triggered the thought, "I wish I had a place to talk about stuff like this." Deluxe Paint , the first post, was also one of those four. There are two yet to go; place your bets. The trouble was, while I knew Deluxe Paint intimately and could jump right in, I simply had no real experience with other categories of productivity software. It would have been impossible to judge Improv fairly without first getting better acquainted with the landscape into which it was born. So, I bided my time. Boy, howdy, do I want to impress upon you the superhuman restraint I've exhibited in waiting a full year to examine Improv . I think we all understand Improv, being a spreadsheet , knows how to calculate sums and transcendentals and other mathematical goodness. "It does math!" is not a particularly compelling justification for its existence, nor to support its claim as the "NeXT generation of spreadsheets." Instead, let's get to know it by studying examples which show off its unique strengths, and with the suggestion of the book Improv 2.1 Revealed! we'll model up some good ole' American baseball, a game I love and understand completely. Industry historians are far more capable of discussing the exile and homecoming of one Mr. Steven Paul Jobs back to Apple. I touched on the TL;DR of it in the timeline, and if you're really clamoring for juicy tidbits, a book was published this very year that delivers on that front. While NeXT hardware was well regarded, it never really caught on. In the long run, the better result of NeXT engineering was NeXTSTEP and how neatly it wrapped UNIX in a clean, user-friendly, graphical interface. It's quite an accomplishment. For me, it was the technology behind NeXTSTEP/OPENSTEP developer tools, Objective-C and Interface Builder, that kickstarted my programming career when those became the OS X development environment (hat tip to Aaron Hillegass's Cocoa Bootcamp at the Big Nerd Ranch). Even today, years after the advent of iOS and Swift, NeXT technology remains in the Foundation layer of Apple software, as denoted by -prefixed classes (a name-spacing trick from the Objective-C days). For this investigation, I'm using NeXTSTEP 3.3, the last version before transitioning to OPENSTEP. NeXTSTEP looks and feels like a modern system, with some choice differences. Familiarity with any modern operating system, especially macOS, will inoculate you from breaking out into hives using it. Considering the eye-candy Jobs seemed excited to add to OS X after the Apple merger, NeXTSTEP is positively staid; nothing here screams " lickable ." If you've ever wondered what Susan Kare could have achieved if she'd had a larger canvas and palette (NeXTSTEP is 2-bit) than the original Mac, look no further. I like it. The dock is present, though more rigid than the playful fluidity it acquired when transitioning to OS X. No "genie effects" nor dynamic magnification; it is ever-present and doesn't hide itself when not in use. Some applications change the dock icon when running (like Mail ), while other applications do not (like Edit ). One odd quirk is how the dock indicates running apps, by the absence of three dots on an icon tile. That's a strange choice, decorating the inactive icons. File browsing adds a little complexity to system usage. The file browser contains its own mini-dock for favorites, the path to the currently selected item rendered as giant icons, and the file browser proper. Dragging files and folders around can only be done with an item's full-size icon. If you're browsing in list view, you can't drag list items into a folder or the dock, for example. Bit of a head-scratcher, that one. Menuing for all applications is handled via a user-positionable, vertical menu. Sticking the menu somewhere sets its position for the active application, and every application can have its own position. You could theoretically have the menu jumping around all over the place as you hop between apps. Thankfully, it can also be called up at the current mouse position with a right-click; shades of RISC OS. Also, good news for those who love the spinning beach ball on macOS! You'll feel that little ASMR tingle when it appears, especially if you run a color workstation. It's funny the things that can make something feel like "home." Honestly, my biggest issue in adjusting to NeXTSTEP is the mouse. By default, it feels weird to use, with a dual-level acceleration curve, one for slow movement, one for fast. Crossing the threshold between "slow" and "fast" jumps the cursor across the screen when the alternate acceleration curve kicks in. I could never find my cursor except when moving the mouse verrrryyyy slooowwllllyyyy. Luckily, the emulator Previous has tools to help compensate ( check "Sharpening the Stone" ). Overall, you'll only need to adjust your thinking a little here and there, forget some modern habits, and you should be comfortable with NeXTSTEP in short order. 35 years later, it still looks and feels much like any modern operating system. Almost depressingly so? Maybe that's a topic for another day. Question: what is interesting about the following screenshot from Improv ? Answer: it contains all information necessary to build a copy for yourself. All data, formulas, dimensions, and relationships are exposed and visible. I note this not to suggest that screenshots are a great way to share Improv worksheets with others, though you could; the sheet above was rebuilt from a screenshot, in fact. Rather, I want to illustrate how "graspable" data can be in Improv's environment. Its formulas read naturally, and value references match the item names in the table. The work does what it claims. Let's interrogate that a little more thoroughly. To understand Improv is to forget what we know about spreadsheets. Slash commands? Never heard of him. B1? J93? K12 = int(A14 * lookup(B2..B7))? The ramblings of a madman. Rather than generic alphanumeric column/row identifiers, we define our rows and columns with "categories of items" specific to our domain. Put another way, instead of asserting, "Columns A to L represent the months, and rows 1 through 31 represent the days," we will create a category called "months" and define twelve items for it named "January," "February," etc. and we will create a category called "days" and define 31 of those. Take a look at these comparative videos which illustrate the difference. First, I'll use LibreOffice Calc to create a months/days grid, apply a value to January 1, and calculate December 31 as a multiple of January 1. Here's how that same table is created in Improv . There are three things to note: how sequential items are generated, how categories are named (not strictly necessary for this simple example), and how formulas are created/applied. There's a lot to absorb, I know, starting with how a new, blank worksheet consists of a single cell, not an infinite table of empty cells. Improv works with only and precisely the data we define. Where in the LibreOffice example we attached meaning to the columns by adding labels to the top of each column, that was for our benefit not Calc's . I could, just to feel the warm, snuggly blanket of classic column/row naming, make an Improv category called "columns" and name my items A, B, C, etc. and a category called rows, with names 1, 2, 3, etc. Were we to take such a pedestrian approach to categories and items, we would diminish the magic of Improv's formula builder. The bottom panel holds all formulas, which are applied at the granularity defined, in the order shown. In the example, I specified but could just as easily have targeted all of December by specifying . That single formula would apply to every list item in December without needing to copy formulas through to each cell, nor set absolute/relative markers for cell references. Here's a simple expansion of the previous table. Every day in January increases by 10%, and its corresponding day in December doubles the January value. The two formulas shown are the only two formulas for the entire worksheet. I want to stress this; there are no other formulas lurking inside cells, hidden from view, waiting to be revealed by a selection or editing process. There are no duplicated formulas per cell, each increasing a relative cell reference by . We simply refer to categories (in this case "months" or "days"), items (as "January" or "27"), or specific intersections (as "April:19"), including relativity signifiers , , and . You might be thinking of ways to trick the system; to confuse and befuddle it. Why would you want to bully an old app like that? What did Improv do to deserve such shabby treatment? "What if two items have the same name? What if two formulas target the same cell?" Fret not, all shall be revealed and you will feel proper shame at trying to confuse this old program. In the above example I have renamed one of the Days as "March," to create ambiguity when referencing "March." Does "March" mean the Day or the Month now? We can be explicit, referring to . Note though that if we aren't explicit, Improv will be on to our tricks and call us out, as in formula 5. In formula 2, I set to the static value . In formula 3, I also set it to be calculated as "double its corresponding January value." Note the "overlap" icon (two intersecting squares) on those formulas and the text in the cell. Clicking the overlap icon gives us an opportunity to resolve ambiguity by explicitly selecting which formula should win for that cell. Cleanly handling formula ambiguity was originally seen by the prototype team as an intractable engineering problem. From "Improv: The Inside Story", by Simson L. Garfinkel, NeXTWORLD , Fall 1991: Personally, I like Improv's solution. I honestly think more software should be very dumb, and not try to intuit or guess what I want. Tell me the problem in clear language, and provide the simplest tool that gives me the most control over problem resolution. Be smart about helping me, don't try to help me by being smart. Before we get in too deep, I want to make sure I note Improv's beautiful help system, replete with index and search. Considering the difficulty in tracking down a manual for the NeXT release , having this available during my exploration proved invaluable. It's neatly illustrated, with terse (sometimes overly terse) descriptions of the tools, hyperlinked to other pages of the documentation, as warranted. Improv felt it could do even better than that and made its help contextual . The Help system, when invoked, defaults to displaying documentation for whatever is currently selected in the user interface. That's very useful, but becomes supercharged with "Point for Help," as demonstrated in the video above. This turns the cursor into a symbol, and whatever we click will be revealed in the Help window. This is a great idea, because I don't always know what Lotus wants to call some little interface doohickey, so I don't know what to search for. Now I know: the "Calc" button is a "Status Indicator" and the triangle icon that sometimes shows up is the "Formula Error Indicator." Pork chop sandwiches. Looking through Usenet archives, I saw this called "toilet paper" by at least one person. Lotus called it the "item dispenser." It's an interesting first attempt to do something we take for granted today and only exists in the NeXT version of Improv . It's cute, but smacks of someone being overly excited with new development toys. If we need to generate a list of items to populate an axis of your data, this tool generates sequential list items for you. It does its best to kind of "guess" our intentions, so if we type in "Sunday," then pull down on the dispenser, days of the week will be auto-generated in a list. We then "tear off" the list, which loads the cursor with the items, which we attach to an existing item, appending that category with the loaded list items. That means we need to have at least one item in a category onto which the list can be attached. This is annoying, because the existing item likely has no relationship to the generated list. I'm always having to make a list, attach it to an existing item, then delete the original, unrelated item. These days, we can type directly into a cell and extend that cell intelligently across adjacent cells. In fact, Lotus would switch to that methodology when migrating Improv over to Windows for version 2. Hold ALT and drag an existing item to get the same effect. The original dispenser truly is a feature that could only have been thought up within the NeXT environment. I don't hold it against the devs for getting excited by the GUI flexibility Interface Builder afforded. I, too, did the same back in the early OS X 10.2 days. It practically begged us to play around with GUI ideas, remixing elements into new configurations. "First Responder" and "bindings" stans, represent! One of the great features of that development environment was how easy it was to “say ‘No’ a thousand times” while trying ideas. Lotus embraced that, achieving quickly on NeXT what had blocked their progress on OS/2 and Microsoft's Presentation Manager. The "item dispenser" was singled out as a specific tool that benefitted from Interface Builder; there really wasn't anything similar to its rapid, iterative, GUI development at the time. In fact, I might go so far as to say there's nothing quite like it today, even from Apple . Joel Spolsky, in editing The Best Software Writing I , added a personal aside on Lotus Improv , blaming its failure thusly, " Improv required you to define strict n-dimensional hypercubes for your data. There was none of the flexibility of traditional spreadsheets." I suppose if one wanted to paint Improv in the most inhospitable light, Spolsky is correct. In practice, it doesn't feel as impenetrable as his description. However, I do think he's on to something that gets at the heart of why Improv never gained traction. For now, let's hold our collective nose and dive into the scary, foreboding waters of "n-dimensional hypercubes" and model up some baseball. Our first dimension will be . Let's add a dimension next. Next, we'll get more granular and add . Internally, Improv separates our work into three parts: data, views, and formulas. What we see on screen is just Improv's best-effort to represent our three-dimensional data flattened out into a two-dimensional view. Improv adds extra columns to the view, as necessary, to ensure we always see the full extent of our data relationships. Now, we'll add some win/loss data and the next thing you know we’ve built a 4-dimensional hypercube. Are we scared yet? Since I've only modeled at the "Division" level, I'll add together divisional wins inline to calculate the totals and... hmmm... "No dice," as they say in baseball (Presumably. Are dice involved in baseball?) Alas and alack, we can only enter concrete numbers, dates, or text into cells; any formula, no matter how trivial, must go into the Formula Panel (bottom). In this case, I could add a formula that targets a specific cell, as , but the whole point of a spreadsheet is to automate that kind of junk for me. I suppose the path of least resistance is to add a category to add up all team wins and, wait a second... Dang it, that petard I confidently told myself wouldn't hoist me has come around to do precisely that. Here we see the downside of dimensional modeling, at least for the layperson, and arguably for most typical spreadsheet users. It all looks like a neat little chart, but that word "dimension" describes and reveals the problem. To add a category, its items necessarily apply to all other categories. Teams don't do that; they are unique to each League and Division. But we just might have a way out. Lotus engineers clearly understood that categories alone weren't going to cut it. Categories impose rigidity to our data that does not always accommodate the messiness of real data. We need another way to organize our data. Groups are intra-category, named collections of sequential items. Make a category and add a bunch of individual items. Select a sequential group of some subset of items and group them, giving that group a unique name. Once everything is grouped, groups of groups can be grouped to group a group of ... wait, I got myself confused. Groups can be grouped, 'nuff said. With "Teams," we quickly run into a side-effect of groups, and how it can necessitate a cascade effect across multiple categories. I want to add teams and group them by "Division," but that forces me to also make "League" a group as well. What was three categories have now collapsed into one: "Teams" Here's what that looks like in practice. We still gain the benefit of name references in formulas, as in the above untethered-to-reality example. A reference to would apply to both American and National, unless we specify by as in and Groups can be subtotaled, using the function, as you can see in this more data-rich chart of the MLB stats from 2022 to 2026. Notice, however, we risk a conflict for in our field, where there are two formulas battling it out for dominance over the same cell calculation. The handy keyword will skip (i.e. "won't apply this formula to") groups, cells, items, and categories as designated. Groups helped bring granularity and specificity to our data structure, but the more categories we remove, the more we remove access to the very feature that has kept Improv in the back of nerd minds for decades. You know what I'm talking about. FINALLY we get to the point those in the know have been anxious to talk about, " Lotus Improv invented pivot tables. It says so right there in Wikipedia." At this stage of my knowledge of all things retro productivity, I think the most I'm willing to concede right now is that Lotus Improv likely invented pivot tables as we understand them today. But, like Yoda coyly teasing the existence of Luke's sister to a guy who should have already known about her, there is another program, a future Stone Tools subject, that may lay claim to the title of "first pivot table." I exercise my right, as "guy who runs the blog," to reserve final judgement on this point until then. For our purposes today, it's a true enough statement and the inventor even has a nice video on YouTube in which he shows it off to Lotus staff for the first time. Meet Pito Salas, inventor of the pivot table. Mike Wells (shown in the thumbnail) takes a question at 28:40 and reveals that NeXT has "Objective Fortran" and "Objective Lisp" ?!?! Salas, now retired , developed the idea while working for Lotus Development's Advanced Technology Group. He put together a fakey version of his vision, a kind of slideshow, showing off the things he wanted to be able to do: English-like formulas, multi-dimensional data, and rearrangeable views into the data. He was given the greenlight to proceed, "All those [ideas] I had described with hand waving,'' recalls Pito. "I had no clue how to build it." A year of nose-to-the-grindstone later, and a prototype dubbed "Back Bay" went on US tour, wowing financial modelers. Everyone who saw it, wanted it. Let's take a look at how Improv handles pivoting. To start, I'm leaving it empty of data, so we can focus on the dynamic view structure more easily. Then, I'll move categories around to show how the view flexes to present the data. Next, I'll put categories into the third axis, called a "Page Category," allowing us to paginate our data. Finally, I'll enter a formula and a little data to show off a trick pivot table contemporaries cannot perform. Now I'm going to try and replicate this in LibreOffice Calc v26.8. There are a few things to note, but let's first point and laugh at the fumbling attempts of a LibreOffice newbie (me). A few things jumped out at me while comparing these approaches. First, modeling up a relatively complex data set in Improv is as easy as thinking about it, and the presentation of the data matches my mental model. LibreOffice requires me to bridge a fairly wide conceptual gap between what I want and how I model it, due to the amount of redundant data I have to enter for every line item. Some items, like "League," translate cleanly, while "Record" does not. In fact, I'm still unclear what the best approach would be to convert my Improv computed item "Win Ratio." LibreOffice has the concept of a "computed field" inside pivot tables, but reaching across columns to pluck out related data to perform computations is complicated. In Apple Numbers, Microsoft Excel, Google Sheets, and LibreOffice , pivot table data is splintered off into a separate snapshot of the data, dedicated to the pivot table view. Pivot table data can typically be lightly manipulated, but does not affect the original data. The reverse can be done, but only after asking for an explicit update of the pivot table to reflect changes to the primary data table. This is a huge drop-off in ease-of-use from the Improv way of things where every view of the data directly manipulates the base data. Dragging around categories is just as easy in either approach, though Improv gives immediate drag-and-drop access to what LibreOffice calls a "Filter," i.e. a paged view, and generally "feel better," IMHO. LibreOffice requires us to open up a separate panel and drag things around in there. I will say, however, that being able to filter the filters (i.e. exclude/include specific subsets of data) is handy. Sheets , Numbers , and Excel (online version) are all strictly panel-based pivoting. Interestingly enough, the panel approach is precisely the interface Lotus rejected , at the behest of Steve Jobs. Once more, from the Garfinkel article. Having now directly compared Improv's approach to dynamic views against modern pivot tables, it is safe to say they are the same only so much as wagyu beef with gruyere cheese prepared by Gordon Ramsay and the sawdust at the bottom of a Happy Meal are both "cheeseburgers." Improv tastes much better. A Lotus invention which debuted in Lotus 1-2-3 , "named ranges" have attempted to bring order to the chaos of cell identifiers. In 1-2-3, will apply a name of choice to a defined cell range, after which that range can be referenced in formulas by that name. So, in 1-2-3 or any modern spreadsheet we can name range as and will work. Superficially, this looks similar to how Improv works with categories and items for multi-dimensional data. However, named ranges are for convenience only, and do not establish relationships between data values. In Improv , would represent a fundamental aspect of a piece of data. Getting the sum of all sales is the same, , but it being a dimension on the data gives us deeper options. For a specific year, will match against two axes at once. could further pinpoint our summation. Let's see how this would be done with modern spreadsheets and named ranges. If we want to mimic Improv's dimensional relationships, we must define relationships of interest within each formula, (and those formulas have to be copied over to every related cell) such that: Meaning "sum items in the named range called "Value" only if the Year is 2020, and the League is "American," and the Record is "Win," and the Division is "East") Of course we could substitute hardcoded filter criteria with values pulled from a lookup, or perhaps extract equated values via a statement. Maybe even set up a little sub-table to hold calculated values. Quickly this gets way more complicated than I'd have hoped for, so personally I'd resort to doing it the old-fashioned way and just hardcode in a bunch of cell references. To reiterate, the complete solution for an entire sheet in Improv , where win/loss information is its own dimension, is the single formula: The last of the big features, which takes advantage of NeXT's graphics prowess, is the charting functionality. Creating a chart launches an entirely separate application, Presentation Builder , which has grand plans on becoming PageMaker when it grows up. Generating a chart generates a full, blank page onto which a chart is placed. A suite of palettes complements the charting workflow, allowing for precision edits of the chart, its titles, colors, positioning, and more within the full page. Multiple charts can be generated and placed on the same page. Arbitrary text and shapes, lines pointing at interesting bits, and even sound annotations (a system-wide NeXT tool, called "Lip Service") can be placed at will. So it goes that while we have the tools to place arbitrary text on the page, yet lack tools to center that text, the sense that Lotus has overextended itself is strong. Sometimes it's enough to just generate the output a program excels at, and leave "page composition" to the professionals. Again, I need software to be more dumb . That's without even mentioning the plethora of formatting tools scattered amongst an unreasonable number of palettes, panels, and "inspectors." A benefit of these tools on the NeXT is that everything we do is Postscript by default, thanks to NeXT's Display Postscript screen rendering engine. "Print" a document as PostScript, and we can render the resultant .ps file on any modern system. I opened it directly in Affinity and received precisely what I saw in Improv , chopped off text and all, but like, high-quality chopped off text. Look, I try my best to craft a compelling narrative, moving from topic to topic, stitching together callbacks and observations to lead you, my dear reader, on a carefully considered ride. It's just you and me, side by side at the keyboard, chatting about software and learning together. But, sometimes there's junk I need to squeeze in that I can't find a natural segue for discussing. Here, then, is a brain dump of other things I want you to know about Improv . Speaking of version 2... In 1993, NeXT abandoned its hardware initiative and moved over to Intel, rebranding as NeXT Software, releasing OPENSTEP soon after. At the same time, Improv also moved over to Intel, but parted ways with NeXT. With version 2, Lotus developers achieved the success that eluded them on OS/2, porting Improv over to Windows 3.1. The feature set for version 2 is disappointingly similar to version 1, and by this time competing software was starting to embrace some of Improv's unique qualities. We do finally get undo, so thank Kapor for small favors. Scripting arrives in the form of LotusScript. A new Windows UI adds a bunch of buttons to the interface, and flushes the toilet paper dispenser for a more sensible list generator. That's basically it for the new stuff, but at least they kept the old problems as well. Cell reference relativity is still locked to a single dimension. Formulas cannot be freely rearranged, only cut-and-pasted into position. No 3D worksheets. Then, to throw a little extra wrinkle into the mix, there was Lotus's product strategy on the PC. If Improv wasn't your thing, maybe their other Windows spreadsheet, 1-2-3/Windows , would feel more comfortable. Or their other-other option, Lotus Symphony . Or their other-other-other option, Lotus Works . Or their other-other-other-other option, Lotus 1-2-3 for DOS Release 4 , which went on sale in 1994. Regardless, it was well-liked by the press, even if reviewers didn't seem comfortable giving full-throated endorsements. Take this Compute , July 1994, review for example. Or this PC World , February 1994, review. It would be hard to intuit, based on that review, that PC World had declared Improv the "Winner" in the Applications category for their 10th Annual Awards for Technical Excellence just a few months prior. 3 in spirit, if not in flesh; let's take a quick look at Quantrix , the product I clumsily foreshadowed earlier. From ex-Lighthouse CEO (also ex-Sun CEO) Jonathan Schwartz's blog , "In 1989, a few friends and I started a software company, Lighthouse Design, that devoted itself to the NeXT platform. Whether any of us admitted it at the time, Lighthouse was built by a group of people for whom Steve Jobs was the gravitational center of the universe." Their devotion to Jobs was returned in favor, by way of advice for their products to flourish on NeXT. Ultimately, according to Schwartz, the problem was less about finding a way for Lighthouse to flourish on NeXT than it was about getting NeXT to flourish in the first place. When looking for a buyout in Lighthouse's waning days, they had fingers crossed to be acquired by NeXT but Jobs didn't bite. They were bought by Sun, their products shoved in a vault never to see the light of day. With Microsoft unwilling to support the platform, Lighthouse made the closest thing to an Office suite for NeXT. They built programming tools, project management software, a presentation tool called Concurrence ( that Jobs used and which "inspired" Keynote ), and two spreadsheet programs: ParaSheet and Quantrix . ParaSheet took the more traditional approach, while Quantrix was just a flat-out clone of Improv , I don't know how else to say it. It has the same split screen worksheet/formulas, the same formula tools, the same draggable categories, it imports Improv documents, and more. According to one Hacker News reader , "The manuals were interchangeable." Lighthouse took the ideas behind Improv and ran with them, envisioning what a possible v3 could have been. Put simply, everything I've expressed frustration with in Improv is addressed in Quantrix . It has undo. It has drag-and-drop, reorderable formulas. It has scripting. It has cross-worksheet category synchronization. The "Item Dispenser" is replaced with an in-line widget; pull to expand like any modern spreadsheet does today. It has multi-dimensional cell relativity. It runs on OPENSTEP for Intel, as well as NeXTSTEP on Motorola 68K . It has color. It loses contextual help (can't win 'em all). And that morass of panels and palettes for editing the charts I showed earlier? Those are collapsed into a single, tidy "Inspector." Long story short? If you want to experience the best version of Lotus Improv today, use Lighthouse Quantrix v2.3. You'll get all of the original, cool tools built better, with a more sensible interface. Plus, Lighthouse gave away their NeXT applications, with license code, for free. How neat is that? The big twist, for those affording me a generous definition of "twist," is that Quantrix still exists . It is alive and being sold for "ask for quote," so I think I'll just put this credit card back in my wallet at this time. Today, it very explicitly targets the professional, high-end, financial modeling crowd, and not the general-purpose masses. You know, the crowd that Lotus first showed their Back Bay prototypes to back in the day (see timeline). Its self-introductory video pivots (get it?) many of its 35-year-old features as revolutionary. A back-handed indictment of the stagnation of an industry, or a product finally finding its niche? You be the judge! It's hard to suss out precisely what happened, but I found a little information floating around the Quantrix forums . If that is to be trusted, Lighthouse's Quantrix chief engineer, Pete Murray, founded the company Subx and rebuilt Quantrix as a Java application in 2003. Quantrix's website says it was acquired by IDBS in 2010. I'm still completely in the dark how Quantrix escaped Sun's containment field, as Sun had refused to sell off the Lighthouse apps to hungry buyers; that stuff could have been converted into OS X apps in the early days. At any rate, it is one of the more handsome JRE applications, while suffering from telltale Java rendering issues, especially with regard to text positioning and scaling. Otherwise it is clean and inoffensive. Sterile? Boring. That's the word I was looking for: boring. "It knows its target audience," is perhaps more charitable. So where does this leave my relationship with Lotus Improv ? Why did I dangle that "And yet..." at the start of the post? If I'm completely honest with myself, there is a vibrating tension between "wanting to love it" and "actually loving it." After all of these weeks, I think I'm in a pretty good position to suggest why it didn't take the world by storm: it's not a great spreadsheet program. Let me qualify that statement lest r/Improv4Life roasts me alive; the roasting can wait a moment. For what it does, it is amazing and even fun to use. What it doesn't do is provide a good, general purpose spreadsheet experience to the masses. It claimed to do that, positioning itself as the "NeXT generation of spreadsheets." Whatever Lotus Improv did or did not accomplish for the spreadsheet genre, one thing is for certain: it makes a fantastic tech demo. Seen in that light, I'd have been as sold on it as Jobs was, excitedly showing it to NeXT non-believers, "Look what MY computer can do!" Those moments of smug technological superiority amongst friends are fun, aren't they? Once you played with it for a week or two, and the initial excitement of pivoting dulled, you may have ashamedly harbored a bit of buyer's remorse. Adam Bosworth, in an article contemplating a talk he gave at ICSOC '04, was thinking about software complexity vs. simplicity, and posited: Paradoxically, Improv's promise to improve data comprehensibility wound up narrowing its general appeal. Want to quickly throw down a few lines of data, like a makeshift phone directory? You're gonna have to rethink your data as a matrix of abstract qualities about your friends. Despite its friendly English-like formulas and delightful GUI for pivoting data, Improv is in actuality purpose-built for far more rigorous data modeling than its playful tools suggest. It's the next generation of spreadsheets for a thin slice of the market. That said, there is a non-zero set of readers for whom their hardcore number crunching needs perfectly align with Improv's promises. If that's you, you're absolutely going to fall in love... ...with Quantrix . Ways to improve the experience, notable deficiencies, workarounds, and notes about incorporating the software into modern workflows (if possible). Improv does export to Lotus 1-2-3 , but you'll likely be disappointed with the results. Consider a table with formulas and calculated values in the cells. Everything you see in the cells will be exported, but there will be no formulas or relationships between anything. You will receive a snapshot of all current values, arranged to visually match the table structure you had when exporting. Improv's approach to shared formulas and multi-dimensional data simply don't have an elegant export solution to traditional spreadsheets. If you have Improv documents you want to import into Quantrix , two criteria must be met. Save your Improv project as an file (not the default ), and make sure you have the Lighthouse (available at the same site offering the Quantrix software link I mentioned under "Version 3") installed at . Exporting charts is simplicity, thanks to Display Postscript. The Postscript files NeXTSTEP generates open and render just fine on modern machines, using Ghostscript or other graphics programs. As mentioned previously, I actually recommend using Lighthouse Quantrix v2.3 instead of Lotus Improv , on the NeXT. The substantial features Improv lacks are numerous: In general, it's lacking the clay-like malleability of a traditional spreadsheet. If you're doing what a lot of us do, using spreadsheets like a kind of mini-database, you'll probably get frustrated. Here's the main event. Compare and contrast with his OS X announcement at MacWorld 2000 for bonus points. Previous v4.4 on Windows 11 NeXTstation Turbo 68040 CPU (variable clock rate; ~40 - 50Mhz) 2GB hard drive Monochrome "Megapixel" display NeXTSTEP 3.3 Lotus Improv Rel 1.0 (v108r0) There is no undo. This isn't the first program I've covered that lacked undo, and I'm not talking about software that can be forgiven for being born too early. The formula panel at the bottom applies formulas in the order shown. As order is important, it would make sense that you might want to rearrange them from time to time. There is no mechanism to do this, other than cut and paste. This has been more of a pain than the lack of undo. Improv uses special keywords to add cell relativity to formulas. So THIS value is the PREV value times 2, or whatever. But we're in a multi-dimensional sheet, so what do these keywords point to, precisely? Well, in a formula they must pick a category, and only that category can be referenced relatively. If a formula uses multi-dimensional data to calculate a value, like , those keywords won't work, though it would be completely natural to write such a formula and expect success. There is no scripting. Lotus 1-2-3 users would have been particularly put out in not being able to bring their carefully built automation over to Improv , which would be a huge shocker since they're made by the same company. I would even go so far as to say that that alone would have pushed a great number of potential converts away. Nobody has the time to rebuild their financial empires from scratch! There are no 3D sheets, meaning there is no linking between projects. A project can contain up to 100 worksheets, and those worksheets can reference one another, but only within the same project file. If a category in a worksheet exactly matches a category in another, item for item, those can match and data will flow across as expected. Unfortunately, there is no way to synchronize the shared categories. Change the items in one sheet and those will not be picked up in sheets using that linked category. This means we cannot make a "master" category inherited by other worksheets. Like a lot of things back in the early days of productivity software, user diligence plays a large role in keeping things running smoothly. Even version 2 on Windows didn't add this feature. If only someone would step up to the challenge. ( foreshadowing ) Previous never crashed on me, and has gotten a lot easier to set up since its early days. Version 4.4 has been great. For mouse movement, I found that setting (in Previous , not NeXTSTEP) "Slow movement" to "Fast" and "Fast movement" to "Slow" balanced things out to something close to consistent tracking; it worked for me, is all I can promise. There is a modern add-on to NeXTSTEP called "GKMouseScaler" that lets the user draw precise cursor acceleration curves, should you need better control. Personally, I never needed it. I found the 2-bit monochrome display to be more than sufficient; no color needed. This is version 1.0 of Improv and it did crash a couple of times, quite suddenly. There appeared to be no specific trigger, nor function, that led to the crashes. It just beachballed and died. NeXTSTEP itself felt stable to me. For installing software from modern repositories, three things to keep in mind Thanks to a convenience in Previous , at in NeXTSTEP you can access your host computer file system. You'll find a lot of .tar.gz files in the wild, so brush up on and Once you have a file, double-click and it should install itself. No undo in v1 on NeXT; v2 on Windows does. Can't reorder formulas except through cut/copy/paste. and only work along a single axis, even though it makes complete logical sense for them to work along multiple axes in the same formula. Presentation Builder is overly complex on the one hand, yet missing very basic tools on the other hand. Can't reference values in other projects, aka no 3D worksheets. No scripting options in v1 on NeXT; v2 on Windows does.

0 views
Andy Bell 2 weeks ago

In response to “The death of web development education”

I read Mat​hia⁠s S​chäf⁠er’s excellent post, titled The death of web development education . I was thinking about how I can contribute to the conversation and really struggled to articulate the problem. The most effective way I can contribute, in my opinion, is a crude sketch I just did of Piccalilli ‘s Stripe chart for all time sales: That’s a whopping 67.5% reduction since 2025… Pretty bleak, huh? Something really does have to change or you’ll miss publishers when they’re gone .

0 views
Simon Willison 1 months ago

Generating running routes with GPT-6 Astra and ChatGPT Work

Here's a neat thing I had ChatGPT Work with GPT-6 Astra (Max) do this morning: It worked for 27 minutes and produced exactly what I'd asked for, as both an embedded visualization and downloadable GPX file and GeoJSON files. Here's that 5K route: When I asked it how it had created the route, it replied: I used Nominatim to locate the address and Overpass to download local OpenStreetMap roads and trails , then calculated the loops locally. Frustratingly, the actual code it ran and exact details of what it did weren't visible to me in the ChatGPT UI. I see this lack of transparency is an anti-feature. By the time I thought to ask for a copy of the Python code it had used, ChatGPT was unable to provide it. This appears to be because the thread had been compacted. I think any LLM system that uses compaction needs to both preserve the pre-compacted text and make that text available via agent tool calls, to protect against this kind of problem. As for displaying the map to me, that used the visualize skill . It created a file called to embed directly into the ChatGPT UI. Here's a copy of that HTML , which starts like this: The element contains the full geometry needed to render both the running route and the map itself, using D3, which is loaded from an allow-listed CDN location described in this section of the visualize skill : You are only seeing the long-form articles from my blog. Subscribe to /atom/everything/ to get all of my posts, or take a look at my other subscription options . The CSP allows only , , , , , , and . Other origins are blocked and fail silently.

0 views
Unsung 1 months ago

“A great look at the destructive power of an uncaught Infinity value”

There are two distinct flavors of speedrunning: Watching the first one feels like watching the Olympics. The second one is absolutely incomprehensible without someone narrating over and explaining things. This video by Marblr is such narration and explanation of a particular glitch in Portal that significantly shortened a few speedrunning moments: = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/a-great-look-at-the-destructive-power-of-an-uncaught-infinity-value/yt1-play.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/a-great-look-at-the-destructive-power-of-an-uncaught-infinity-value/yt1-play.1600w.avif" type="image/avif"> It’s an hour-long video, and I think it’s a really well-made one, and one you can learn a lot from. The video has some nice history of strafing – similar in some way to Scroll Lock – and motor memory accommodations. It also shows the source code of Portal to explain things. This is fun on its own, but on top of that, Marblr layers some really cool visualizations – watching how one value “infected” by infinity proceeds to infects the others is riveting. Even the levels and their geometry are presented in a really beautiful and informative way, too: I’m still not fully onboard with this branch of speedrunning – if the entry point to your effort is changing mouse sensitivity to an abnormally high value, or if you have to pick a very specific version of the game that just happens not to have patched the glitch you care about, then perhaps things have gotten too esoteric and maybe even academic? But at the very least we get videos like this one – or one I linked to in May – going under the hood of software in some particularly enthralling ways. completing a game as fast as possible through feats of extraordinary mastery, timing, and precision, completing a game as fast as possible by finding all sorts of bugs and exploits in the game (commonly known as “glitches”) that allow the player to find eerie shortcuts between areas without having to play them.

0 views
Simon Willison 1 months ago

The Pelican comparison grid for Astra is pretty interesting

I got access to GPT-6 Astra this afternoon, so naturally I used it to generate SVGs of pelicans riding bicycles - at low, medium, high, xhigh and max reasoning levels (Astra doesn't support reasoning=none). Then I rendered those pelicans in a comparison grid with GPT-5.6 Sol, Terra, and Luna, and beyond being fun the result was surprisingly useful. See the grid for full quality images. Here's the transcript that created the GPT-6 Nova pelicans. There are a few interesting things that stand out from this grid. I wonder if Astra and Luna are more related to each other than OpenAI let on? You are only seeing the long-form articles from my blog. Subscribe to /atom/everything/ to get all of my posts, or take a look at my other subscription options . The Astra pelicans are much better . The very best GPT-5.6-Sol pelican (I liked xhigh better than max) is still pretty clearly a bunch of abstract shapes. Every single one of the Astra pelicans, from low to xhigh, looks better than that. The Astra max one is really good. Astra below max still doesn't reliably get the pelican legs on both sides of the frame. In terms of cost, Astra may be around twice the price of Sol ($10/million input, $50/million output, compared to $5/$30 for Sol), but it uses significantly less tokens at each of the levels, making the prices at the different levels closer than they might otherwise be. Astra low produces a better pelican than ANY of the GPT-5.6 Sol models at any level, for 9.55 cents. Spending 10 cents on any other model gets a much worse result. Look at the input token counts: Astra and Luna both used 16 input tokens, Sol and Terra used 26. That's interesting.

0 views
Jim Nielsen 1 months ago

A Calendar View For My Blog

807 blog posts across 14 years. That’s how much I’ve published on my blog at the time of this writing. And here’s the question I’ve been turning over in my mind: “How do I convey that kind of volume across time in a more interesting way than a mere reverse-chronological list?” I’m not hating on reverse-chronological lists. I love my list view . I use it all the time to find stuff I’ve written. But it’s only one way of navigating and digesting all my posts. “What would be another way?” Surely there are many answers to that question. And I’ll probably be exploring them more and more over time. But I had an idea for a new view that I built out and shipped: my calendar view. It’s also just a reverse-chronological view, but it’s meant to convey a sense of posting patterns through time more than it is meant to be a good browsing experience of content. The view is simple: a calendar view of days in each year, and if I posted on a day, it gets a circle (bonus: if a post hit hacker news, it gets an orange square instead). You can click on the dots to see the names of the posts from that day (and follow the link to them, but really this view is basically to scratch an itch of mine. Because you can just make stuff for yourself, and that’s what I’ve done here. And now I’m writing about it because that means I get another little circle for today! Good job, Jimbo. Check it out Reply via: Email · Mastodon · Bluesky

0 views
Giles's blog 1 months ago

Adding diagrams to my static site generator with D2

A lot of the time when I've been writing posts for this blog, I've felt that a diagram would really help. But they're a pain to produce well, and I think I underuse them as a result. I wanted to fix that, and wound up adding D2 support to my static site generator. I think it works pretty well! In the past, I've tried drawing my own diagrams in LibreOffice and exporting as SVG, but my complete lack of artistic skill doesn't help: Asking an AI to do it for me helped in simple cases: ...but with something less standard (there must be a million neural network diagrams in their training sets) it can be really fiddly to get something right. I did some investigations into the various diagram-generating tools out there, and decided to give D2 a go. It has a simple language for specifying what your diagram should show, and the output is pretty nice: Here's the source for that diagram: That looks pretty clear to me! So now, in the source for my blog posts, I have a directory. That contains subdirectories -- by convention, I create one for each post that needs diagrams -- and D2 files. These can be generated automatically when I publish: (Hat tip to Evan Hahn for the method on , which I wasn't aware of.) The flags on the command line took a little bit of fiddling; the just gets rid of the large margins that D2 puts around the diagram by default, but the others are to tell it to use the ELK layout package with particular formatting. Its default layout has curvy lines, and I prefer the closer-to-right-angle ones that ELK provides. Another awkward bit was in scaling; the file that is generated by that command comes out pretty large ( you can see it full-size here ). By default, I allow images inlined into my posts to be as wide as the text, but that would still be too large here. I use to convert the markdown source for my posts into HTML, and there isn't any way to tell it what size an image should be using markdown-ish syntax. So for now, instead of embedding images the normal markdown way, like this: ...for these D2-generated ones I'll just embed a normal tag like this: ...so that I can control the size. Perhaps more work needed there. At some point I may go back and update my old diagrams -- at least, the really ugly hand-drawn ones -- to use this. And a random thought: perhaps it might also make sense to include the D2 source somehow on the blog? I can imagine that it could help with accessibility in some situations, and perhaps also for any LLMs stopping by. Will have to ponder that a bit more. What do you think? Does the D2 diagram look good to you? Or is there a better diagramming package that might work better?

0 views
Maurycy 3 months ago

The Tadpole galaxy:

North is up (exact, mirrored). 0.53 "/pixel [18.8' x 7.4'] FWHM = 4.2" This galaxy has a massive (and rather bright) tidal tail, but I can't see an obvious companion galaxy. The general consensus is that there's a second galaxy behind it... although a nearby elliptical has a suspiciously similar redshift: Not my finest work due to some patchy clouds, but not terrible. Callibration (dark + flat) Stacking (average w/ outlier rejection) White balance and background subtraction (no gradient removal) Asinh stretch (color preserving) Rotation + crop /astro/arp188/stacked.fits.fz : Raw stacks https://ned.ipac.caltech.edu/level5/Arp/Figures/big_arp188.jpeg : Arp's image

0 views
Jason Tucker 3 months ago

From Plex to Jellyfin Part 7: Watching Habits—Tautulli to Streamystats

When I was running Plex, Tautulli sat quietly in the background doing one thing I actually cared about: telling me what I'd watched and how much time I'd spent on it. I had no use for its notification system or custom script execution. I just wanted the numbers. The graphs. The "oh, I watched 40 hours of this show last month" kind of accountability. Tautulli was good at that. You'd log in, see your homepage stats, filter by time range, check your user page, and walk away knowing exactly which shows were eating your free time. Then you migrate to Jellyfin. Tautulli stops working because it only speaks Plex. Jellyfin doesn't have a built-in equivalent. You're suddenly flying blind. Enter Streamystats . Tautulli and Streamystats both answer the same basic question: what have I been watching? But they arrive at the answer through completely different lenses. Tautulli is a Python-based monitoring tool that tracks what was watched, who watched it, when and where, with detailed statistics about each library and user. The interface is functional, organized by discrete views (Users, Libraries, History, Graphs). Everything is there. Nothing is flashy. Streamystats is built with modern frameworks and includes a dashboard with overview statistics, live sessions, recommendations, and an interactive AI chat interface. The data architecture is newer. The UI feels more intentional. You're not clicking through separate tabs to find what you want; you're asking the dashboard what you want to know. Here's the practical difference: Tautulli made you navigate. Streamystats makes you browse. The rows that matter depend on what you actually use. I skipped notifications and scripting entirely. The loss is immaterial to me. I gained better graphs, a nicer interface, and a completely new capability I didn't have before: AI-driven insights. Tautulli's homepage gave you four key boxes: your top items this week, your top users, recent activity, your watch time by duration. Clean. Minimal. You look, you go. Streamystats' dashboard is denser. It defaults to showing you live sessions (who's watching right now), recent activity, trending content, and recommendations. Scroll down and you get genre breakdowns, watch time graphs, and completion rates. It's a different philosophy. Instead of answering "what was I doing," it's showing you "what's happening now and what should you notice." The real shift comes in the user pages. Your own user page in Streamystats shows your personal watch history with filters (date range, library, media type), time spent watching by date, watch frequency, top items, and now, heatmaps showing when you watch and how much. You can drill into any show and see your episode progression. You can see which users share your watching taste. That last one didn't exist in Tautulli. Not in any meaningful way. Streamystats added four things that fundamentally change how you interact with your library. Watchlists. You can create custom lists in the interface. Mark items as watched or unwatched. Set them public or private. Flag them as "promoted" so they show up as sections in your Jellyfin clients. This wasn't Tautulli's job. This was Radarr/Sonarr territory. Streamystats treats it as a first-class feature. This isnt something that I use since I have other tools doing this but its interesting. Semantic search and AI chat. If you enable OpenAI-compatible API access (OpenAl,Together Al, Fireworks Al, Voyage Al, OpenRouter, Ollama (Local), LM Studio (Local), LocalAl (Local), Custom) Streamystats can embed your library and answer questions about it. "Show me all the sci-fi I've started but haven't finished." "What should I watch based on what I liked last month?" The backend uses vectorchord to store embeddings and performs fuzzy similarity search. It's genuinely useful if you have a large library and a fuzzy memory of what you own. I've messed with this a bit. Lots of graphs and charts. See visually when you and your household watch. Monday night at 8pm? There's the spike. Thursday afternoon when the kids are home? Another pattern. Saturday morning movie binges? It's all there in a grid. Year in Review. Streamystats generates an annual recap of your watching habits. Total hours watched, top shows, top movies, most-watched genres, viewing trends across the year. It's Spotify Wrapped for your media server. You get a moment at the end of the year where you actually see what you spent your time on, quantified and visualized. Tautulli never bothered with this. It was data for data's sake. Streamystats understands that sometimes you want to feel something about your numbers. These weren't problems Tautulli solved. They're problems you didn't know you had until the tool offered an answer. Streamystats runs via Docker with a setup wizard that walks you through connecting your Jellyfin server, and the first load can take a while depending on library size. I used the Unraid template " Streamystats—AIO, " which bundles the frontend and backend together. One template, one configuration, done. The Docker Compose setup is straightforward. Change your SESSION_SECRET (generate one with openssl rand -hex 64). Point it at your Jellyfin server. Wait for the initial scan to complete. That's it. The first load took about fifteen minutes with my library size. After that, it runs quiet. The database grows as you watch, but there's no plugin to manage on the Jellyfin side. Streamystats relies solely on the Jellyfin API for statistics, so the playback reporting plugin is no longer needed. Part 6 covered how Moonfin 's native collections support and Moonbase's row reordering let you build a curated home screen without relying on the Home Screen Sections plugin. Streamystats extends that toolkit, but differently depending on which client you use. If you're on Moonfin, Streamystats isn't directly part of your home screen setup. Instead, you use the Seerr integration, which lives in Moonfin's Integrations menu. From there, you enable and reorder Discovery Rows: Recent Requests, Recently Added, Trending, Popular Movies. These rows pull from your Seerr instance and surface directly on your home screen alongside your automated collections. No Streamystats watchlists involved. This is how you surface curated request activity without a separate collection build job. If you're on Streamyfin , promoted watchlists work differently. When you create a watchlist in Streamystats and flag it as promoted, those watchlists become available as sections in Streamyfin. You're not just tracking what you've watched; you're creating rows that show up on your phone or tablet. You curate them directly in Streamystats. Mark items as watched or unwatched. Reorder them. Share them across your household. The promoted ones automatically surface alongside your automated collections. Both approaches serve the same purpose: surfacing specific curated or request-driven content on your home screen without relying on collection build automation. Your Year in Review data stays in Streamystats, but your best-of-the-year watchlist (if you're on Streamyfin) can live as a row on your home screen all year long. For Moonfin users, Seerr's Recent Requests row does something similar, showing what your household has been requesting to watch. Tautulli was observability. It told you what happened. Streamystats is observability with personality. It tells you what happened, suggests what matters, answers the question you were about to ask, and occasionally reminds you of the year you just had. For me, that's the difference. Tautulli was a tool I opened occasionally to satisfy curiosity. Streamystats is something I check regularly because it keeps showing me things I didn't know I wanted to know. The Year in Review sits somewhere in between. It's not essential infrastructure. It's just fun for end of the year reflection. Your household watching patterns are more interesting than they seem at first glance. Streamystats makes those patterns visible and occasionally celebratory in a way Tautulli never quite managed. That's not revolutionary. It's just better. What do you think? Make a free account here and leave a comment below!

0 views
daniel.haxx.se 3 months ago

A curl mountain movie

One of my favorite visuals for known vulnerabilities in curl is the mountain . It shows how many currently known vulnerabilities were present in the code through-out curl’s history. In the end of June 2026 it looks like this: Over time we get more vulnerabilities reported. Since every flaw has a version range during which the problem existed and with more issues that have overlapping version ranges, the mountain grows. It changes shape every time we do a release or we publish a new vulnerability. At this moment in time, curl version 7.34.0 is the release that contains the most number of known vulnerabilities: 101 . The worst one ever if you will. Out of a total of 206. The mountain uses different colors for different severity levels of the published vulnerabilities, as the legend in the top-left of the image explains. To illustrate the ever-changing nature of the shape and size, I wrote a script that renders the mountain the way it looked at specific dates in the past up until today. More specifically, the script renders one image for every month since curl started (March 1998). I then turned these 340 individual images into a little movie that shows how it grew into today’s shape. At four months/second. The data for this come from vuln.pm and the curl git repository . The graph rendering is based on the dashboard scripts . All images put into a movie with ffmpeg of course. Several people have asked what happened in 2016 that caused the notable drop. A slope if you will. If we zoom in on that, we can spot that curl 7.51.0 has eleven fewer vulnerabilities than the version before that. This release was the first one after the 2016 Cure53 code audit , but other than that there is no clear distinct process or obvious code changes that explain this trend shift. Lots of other graphs show just the ordinary pace and growth in various project areas. It was still fairly early days CI-wise but had been running at least a few CI jobs per commit for a few years already by then. curl was adopted into the OSS-Fuzz project in July 2017, which since then makes us find some issues better, but the drop looks like it happened before then. We had already been analyzing the code regularly on Coverity since a few years. Better tooling? New compiler options? We simply don’t know. As we keep announcing more vulnerabilities going forward, things will continue to change. Maybe I will come back and make another movie in five years?

0 views
Unsung 4 months ago

I can’t stop watching Bret Victor’s talks

You might have seen Bret Victor’s 55-minute Inventing On Principle talk soon after he gave it in 2012. If not, you should check it out. If you did, you should check it out again and see how it makes you feel today: = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/i-cant-stop-watching-bret-victors-talks/yt1-play.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/i-cant-stop-watching-bret-victors-talks/yt1-play.1600w.avif" type="image/avif"> It is about interactions but in the service of something grander, which (if I’m doing my job well) you might recognize as Unsung’s core theme. Victor – a designer, researcher, and computing historian – gave a few other talks in the few years since, and I thought a little guide might be helpful: There are some wonderful repeating themes in there: I love this blend of theory and practice, inspiration and pragmatism, high- and low-level. The tools look surprisingly professional for research projects, but underlying their microinteractions is a deep philosophical stance. It all reminds me a bit of Jef Raskin and Doug Engelbart. Victor’s last talk of this era is Seeing Spaces (15 mins) from 2015, serving as a sort of introduction of him moving toward computing in physical spaces. As far as I understand, Victor has been spending time on Dynamicland since, which is definitely more physical computing, but also a lot more academic and scrappy, and as such out of range for this blog. (His website is worth checking out , especially if you’re not in the mood for talks and would like to get to know his work in a different way.) #conference talk #flow #interface design #youtube Media For Thinking The Unthinkable (40 mins) is the continuation, with slightly more academic examples. Drawing Dynamic Visualizations (35 mins) is specifically about information visualization, chiefly a demo of the “Illustrator, but programmatic” tool showed briefly in the above talk. There’s also a bit more theory. Similarly, Stop Drawing Dead Fish (53 mins) is a demonstration of a different programmatic tool to make animations. The Humane Representation of Thought (56 mins) presents more theoretical underpinnings to the Inventing On Principle talk. The Future of Programming (32 mins) is a – mostly static/​traditional – history lesson about what programming could be. We can do and expect better from computing and interactions. You have to know your history to march confidently toward the future. Ideas need an environment that nurtures them. Playful environments leads to more discoveries. Feedback doesn’t just have to happen. It has to happen immediately and comprehensively. There are no left-brained and right-brained people, but our brains have two different modalities: language (algebra) vs. spatial (geometry). A big emphasis on two-handed operation (kind of like Fontificator just yesterday ).

0 views
Maurycy 4 months ago

Arp 185:

North is left. 0.35"/pixel (7.7' by 20' field). FWHM = 5.5" Arp filled this under "galaxies with narrow filaments", but I don't really see why. It's a barred sprial with irregular arms and a faint extended disk. That being said, the galaxy's bar is slightly misaligned with the core: which would sudgest that it might have experienced some graviational interaction in the past. The main galaxy is at z=0.0045, and the faint background galaxy in the lower left is UGC 10509 at z=0.055. It's actually a spiral with a really intresting structure, but it's too small (and dim) to see in my image. Taken during a night with exceptionally bad seeing, resulting in a FWHM of 5" instead of the usual 3". Not that I ever get good seeing, presumably due to nearby heat sources: My telescope was set up between a asphalt road and asphalt roof. Both of these get super hot during the day and retain that heat all night. Callibration (dark + flat) Stacking (average w/ rejection) White balance and background subtraction Asinh stretch /astro/arp185/stacked.fits.fz : Raw stacks. http://ned.ipac.caltech.edu/level5/Arp/Figures/big_arp185.jpeg : Arp's image.

0 views
blog.philz.dev 4 months ago

Page on the client, really, even for 1M rows

Too often do we find ourselves clicking, and clicking, and clicking "Next" through trivial amounts of data--a few thousand rows--or fighting a search box that doesn't really work. Though the DOM genuinely doesn't seem to like tens of thousands of rows, a little virtualization (e.g., via the excellent DataTables library) goes a long way. If you're up for some DuckDB-Wasm fun, you can page through a million row just fine. Obligatory demo below. (Demo vibe-coded, unlike this text.) Thanks, OpenStreetMap data. → Open the demo full-screen

0 views
Maurycy 4 months ago

Arp 114:

This one has two entries in the Altas of Peculiar Galaxies : Once as Arp 25 under "one heavy arm" and again as Arp 114 under "ellipticals close to and perturbing spirals". The spiral structure of NGC 2276 is somewhat unusual looking, although it's not an obvious case of tidal interaction. Hubble data suggests it's actually moving quite fast (950 km/sec) through space, which causes the lob-sided arms as it interacts with sparse gas clouds. If this is correct, the galaxy is not actually head-on as seen from earth: It tilts toward us on the eastern (top) side and away on the western (bottom) edge. This would require the galaxy to be stretched by the gravity of NGC 2300, so it's still an interacting galaxy, but in a different way. The origin of spiral structures in galaxies is somewhat debated because the arms are unstable : The inner regions of the galaxy orbit the core faster, which should mix them together into a uniform disk. It's generally believed that — instead of being physical structures — the arms are density waves traveling through interstellar gas and triggering rapid star formation, making those regions much brighter. The galaxy at the top (NGC 2300) doesn't have very much gas so it's glow is mostly composed of long-lived red stars. Elliptical galaxies like this are the majority in the universe, but are quite rare in our part of it: this is the first time I've intentionally imaged one instead of having it sneak into the background. ... although spiral galaxies have an elliptical-style halo around them. It's usually much dimmer than the spiral arms, but it is there. As for imaging, both are less than 5 degrees from the north pole , so it's one of the few galaxies that could work well with untracked photography: using a normal tripod and telephoto lens. Some of my data came from a night with really poor transparency and the rest was taken during the full moon. This was not ideal for the low surface brightness of the elliptical galaxy. The spiral galaxy is around 100 arcseconds wide (~2 jupiters) so it's not really a good match for my perpetually bad seeing: the image FWHM is around 4 arcseconds. ... but I'm not convinced clear nights with no moon or/and good seeing even exist at my place. Callibration (dark + flats) Stacking (average w/ rejection) White balance and background subtraction Asinh stretch Light denoising stack.fits.fz: 32-bit compressed FITs image https://esahubble.org/images/heic2106a/ : High-resolution hubble image. https://arxiv.org/pdf/2512.17486 : Ram pressure write up.

0 views
Unsung 5 months ago

“193 hours of attempts (and practice)”

More unexpected Mario content: a 7-minute video speedrunning composite by FlibidyDibidy : = 2x) and (width >= 700px)" srcset="https://unsung.aresluna.org/_media/193-hours-of-attempts-and-practice/yt1.2096w.avif" type="image/avif"> = 3x) or (width >= 700px)" srcset="https://unsung.aresluna.org/_media/193-hours-of-attempts-and-practice/yt1.1600w.avif" type="image/avif"> This video combines my first 5,162 attempts to speedrun Super Mario Bros. I recorded 193 hours of attempts (and practice) on an original 1985 Nintendo Entertainment System, then I wrote a custom computer program to process those videos and combine them via machine learning and conventional image processing techniques. This is not just fun to look at, and – presumably – study as you’re speedrunning yourself. A sign of a good visualization is that it makes you see stuff that you haven’t before and here, at some point (after 1:42), you start noticing strange comb-like patterns in Mario running. Turns out this is actually a thing called a “frame rule,” a quirk of game’s timing code where it only checks for a completion of the level every 21 frames, or one third of a second. That means that for every level after the first one, your start will be rounded up to the nearest 21st frame : The analogy often given is to think of a bus that leaves every 21 frames, and levels can only end by getting on that bus, and so other than in the last level (which has no new level to load at the end of it), improvements in Super Mario Bros. can only happen in 21 frame increments. If you save a frame or two in a level, but it’s not enough to make the previous frame rule, it’s not enough to take the previous bus, you’ll just end up waiting for it to happen anyway. Stay tuned to the end of the video for some fun stats, and click through in the description below to see the same tech applied live during an in-person speedrunning event. #speedrunning #super mario bros #youtube

0 views
Simon Willison 6 months ago

Qwen3.6-35B-A3B on my laptop drew me a better pelican than Claude Opus 4.7

For anyone who has been (inadvisably) taking my pelican riding a bicycle benchmark seriously as a robust way to test models, here are pelicans from this morning's two big model releases - Qwen3.6-35B-A3B from Alibaba and Claude Opus 4.7 from Anthropic . Here's the Qwen 3.6 pelican, generated using this 20.9GB Qwen3.6-35B-A3B-UD-Q4_K_S.gguf quantized model by Unsloth, running on my MacBook Pro M5 via LM Studio (and the llm-lmstudio plugin) - transcript here : And here's one I got from Anthropic's brand new Claude Opus 4.7 ( transcript ): I'm giving this one to Qwen 3.6. Opus managed to mess up the bicycle frame! I tried Opus a second time passing . It didn't do much better ( transcript ): A lot of people are convinced that the labs train for my stupid benchmark . I don't think they do, but honestly this result did give me a little glint of suspicion. So I'm burning one of my secret backup tests - here's what I got from Qwen3.6-35B-A3B and Opus 4.7 for "Generate an SVG of a flamingo riding a unicycle": I'm giving this one to Qwen too, partly for the excellent SVG comment. The pelican benchmark has always been meant as a joke - it's mainly a statement on how obtuse and absurd the task of comparing these models is. The weird thing about that joke is that, for the most part, there has been a direct correlation between the quality of the pelicans produced and the general usefulness of the models. Those first pelicans from October 2024 were junk. The more recent entries have generally been much, much better - to the point that Gemini 3.1 Pro produces illustrations you could actually use somewhere , provided you had a pressing need to illustrate a pelican riding a bicycle. Today, even that loose connection to utility has been broken. I have enormous respect for Qwen, but I very much doubt that a 21GB quantized version of their latest model is more powerful or useful than Anthropic's latest proprietary release. If the thing you need is an SVG illustration of a pelican riding a bicycle though, right now Qwen3.6-35B-A3B running on a laptop is a better bet than Opus 4.7! You are only seeing the long-form articles from my blog. Subscribe to /atom/everything/ to get all of my posts, or take a look at my other subscription options .

1 views
Martin Alderson 6 months ago

A little tool to visualise MoE expert routing

I've been curious for a while about what's actually happening inside Mixture of Experts models when they generate tokens. Nearly every frontier model these days (Qwen 3.5, DeepSeek, Kimi, and almost certainly Opus and GPT-5.x) is a MoE - but it's hard to get an intuition for what "expert routing" actually looks like in practice. So I built a small tool to visualise it: moe-viz.martinalderson.com You can pick between a few different prompts, watch the generation animate out, and see exactly which experts fire at each layer for each token. The top panel shows routing as the token is generated, the bottom panel builds up a cumulative heatmap across the whole generation. I built this by modifying the llama.cpp codebase to output more profiling data, with Claude Code's help. So it may have serious mistakes, but it was a really fun weekend project. The thing that really surprised me: for any given (albeit short) prompt, ~25% of experts never activate at all. But it's always a different 25% - run a different prompt and a different set of experts goes dormant. That's a much more interesting result than I expected. Interestingly Gemma 26BA4 runs really well with the "CPU MoE" feature - 4b params is not a lot to run on a fairly fast CPU and having KV cache on GPU really helps. I think there's a lot of performance improvements that could be done with MoE inference locally as well - eg caching certain experts on GPU vs CPU. If you're interested in learning more about LLM inference internals I'd certainly recommend pointing your favourite coding agent at the llama.cpp codebase and getting it to explain the various parts - it really helped me learn a lot.

0 views
Unsung 6 months ago

“So I wrote a script that takes monthly screenshots of Google and Apple Maps.”

From 2010 to 2021, Justin O’Beirne had been writing about online cartography , specifically in Google Maps and Apple Maps. While both of these services changed a lot since the essays, they are still worth reading. They might be the closest to modern reviews of software as I can think of, and the way the essays are done also teaches us storytelling lessons – from nice visualizations and comparisons, to rich footnotes. There is also a great balance of high-level overview, and then jumping into specifics that reinforce it. Here’s one example of cool tooling O’Beirne used to make his points more sticky: I wrote a script that takes monthly screenshots of Google and Apple Maps. And thirteen months later, we now have a year’s worth of images: The result is informative and mesmerizing: Among the essays, I’d particularly recommend these: There are also book recommendations and a memorable user story . #above and beyond #colors #complexity #details #toolmaking #typography #web The back-and-forth of Google Maps’s Moat and New Apple Maps : Reverse engineering areas of interest, thinking of how the slow changes in visuals lead up to strategy, good visual comparison of competition, and small fascinating anecdotes of places like Parkfield, California. (And a great example of the old adage: don’t get into the business of predicting the future as this will age your writing the most.) A Year Of Google Maps & Apple Maps : Evolution and redesign as ways to “increase capacity.” Google Maps & Label Readability : A fascinating discovery of “city donuts.” What Happens to Google Maps? How cross-device compatibility can mess up maps.

0 views