Jimmy Miller 2 weeks ago

AI Has Made it Easy to Own Your Tools

I am a digital hoarder. Probably not an extreme one. I don't have some fancy complicated raid system for endless terabytes of media. But when I find a link or a pdf to some interesting article, I want to save it. But I almost never review it. That isn't to say I don't read articles. I read plenty. But I also collect them. But I've always had a problem: how do I organize them, classify them, keep them? For links, my answer was pocket. Well, you can imagine how that went. My exported list is living in Raindrop for the time being. My PDFs had been living in Muse . A nice piece of software, but it has diverged from the minimalism I enjoyed about it. So I've been on the search for alternatives. But there has always been this nagging feeling that what I want is not just some different piece of software, but a custom, fully owned setup for myself. But where would I ever find the time to do that? I admit that I'm far from having the tools I want. The tools I've built so far are rudimentary. But they were built in minutes, and they were exactly what I asked for. And they did things that a few years ago would not have been possible. Local LLMs are getting better and better. Seeing that trend, I bought a framework desktop . Using it, I was able to have Claude write a quite simple script that found every pdf on my machine, grabbed some of the initial text from them, and passed them to gpt-120b and asked, is this pdf about programming? Now I can find all those programming PDFs. But I needed to sort them. Now that I have the initial collection of potential PDFs. How was I going to sort them? I didn't want a tagging system. There's a chance that later I will. But for now, I wanted discrete categories. But what categories? I'd only find out once I started organizing them. So I asked Claude and got an app specifically designed to let me categorize them . A simple tool for syncing PDFs by hash to S3. Some of the PDFs have nice metadata built in. So we can just go ahead and extract that . That's why this tool. For the rest, we pass to Qwen3-VL-30B to grab the title and author . A swift application compiled for my Mac and iPad that lets me annotate PDFs . The app is far from fully featured yet. But the fact that it syncs between my Mac and iPad seamlessly is wonderful. I use this mainly for the podcast, so I haven't gotten to do a ton with it yet. But having it be something I can customize to my needs already has me excited. A page on my website that statically generates the archive for all to browse. We've yet to reach the point where local models can quite replace Claude for coding. But having a local model where I never had to worry about the cost to send it experiments, one where it ran not on my laptop, but in the background on a "server", was such an enabling feature. I would have been very hesitant to send all these PDFs to a model. But with a local model, I had so much flexibility to use it wherever judgment could be a substitute for deterministic processing. Nothing here was groundbreaking. Nothing here is something I couldn't have made myself. But they are all things I put off. All things I would have never prioritized, but wanted made. They are all imperfect tools. Many of them are one-offs. There were actually countless other small tools made along the way. Cleaning up titles, a tool that chose between the metadata title and the OCR ones (ocr were usually better). Any one of these little bottlenecks might have been enough for me to stop working on this project. I see lots of discussions about AI all having to do with "production code". I'm sure I'll write my thoughts about that at some point. But I also think it is important that we remember that this isn't the only code that exists. In this case, it's personal code . Code, I enjoy having the ability to modify. Code that I am happy isn't robust; it doesn't try to handle every edgecase. It doesn't need a complicated sync process. Doesn't need granular permissions. This is just the start of what I expect to be a continual evolution of my pdf (and eventually link) management software. But for the first time in my programming life (not career, not everything is a business transaction), I don't feel the weight of maintenance I've created for myself. I feel a sense of freedom to build more without incurring such a heavy cost. This, to me, is one of the most exciting features of our new AI capabilities.

Swift AI Programming

▲ 0

1 views

Jimmy Miller 1 months ago

The Easiest Way to Build a Type Checker

Type checkers are a piece of software that feel incredibly simple, yet incredibly complex. Seeing Hindley-Milner written in a logic programming language is almost magical, but it never helped me understand how it was implemented. Nor does actually trying to read anything about Algorithm W or any academic paper explaining a type system. But thanks to David Christiansen , I have discovered a setup for type checking that is so conceptually simple it demystified the whole thing for me. It goes by the name Bidirectional Type Checking. The two directions in this type checker are types and types. Unlike Hindley-Milner, we do need some type annotations, but these are typically at function definitions. So code like the sillyExample below is completely valid and fully type checks despite lacking annotations. How far can we take this? I'm not a type theory person. Reading papers in type theory takes me a while, and my comprehension is always lacking, but this paper seems like a good starting point for answering that question. So, how do we actually create a bidirectional type checker? I think the easiest way to understand it is to see a full working implementation. So that's what I have below for a very simple language. To understand it, start by looking at the types to figure out what the language supports, then look at each of the cases. But don't worry, if it doesn't make sense, I will explain in more detail below. Here we have, in ~100 lines, a fully functional type checker for a small language. Is it without flaw? Is it feature complete? Not at all. In a real type checker, you might not want to know only if something typechecks, but you might want to decorate the various parts with their type; we don't do that here. We don't do a lot of things. But I've found that this tiny bit of code is enough to start extending to much larger, more complicated code examples. If you aren't super familiar with the implementation of programming languages, some of this code might strike you as a bit odd, so let me very quickly walk through the implementation. First, we have our data structures for representing our code: Using this data structure, we can write code in a way that is much easier to work with than the actual string that we use to represent code. This kind of structure is called an "abstract syntax tree". For example This structure makes it easy to walk through our program and check things bit by bit. This simple line of code is the key to how all variables, all functions, etc, work. When we enter a function or a block, we make a new Map that will let us hold the local variables and their types. We pass this map around, and now we know the types of things that came before it. If we wanted to let you define functions out of order, we'd simply need to do two passes over the tree. The first to gather up the top-level functions, and the next to type-check the whole program. (This code gets more complicated with nested function definitions, but we'll ignore that here.) Each little bit of may seem a bit trivial. So, to explain it, let's add a new feature, addition. Now we have something just a bit more complicated, so how would we write our inference for this? Well, we are going to do the simple case; we are only allowed to add numbers together. Given that our code would look something like this: This may seem a bit magical. How does make this just work? Imagine that we have the following expression: There is no special handling in for so we end up at If you trace out the recursion (once you get used to recursion, you don't actually need to do this, but I've found it helps people who aren't used to it), we get something like So now for our first left, we will recurse back to , then to , and finally bottom out in some simple thing we know how to . This is the beauty of our bidirectional checker. We can interleave these and calls at will! How would we change our add to work with strings? Or coerce between number and string? I leave that as an exercise to the reader. It only takes just a little bit more code. I know for a lot of people this might all seem a bit abstract. So here is a very quick, simple proof of concept that uses this same strategy above for a subset of TypeScript syntax (it does not try to recreate the TypeScript semantics for types). If you play with this, I'm sure you will find bugs. You will find features that aren't supported. But you will also see the beginnings of a reasonable type checker. (It does a bit more than the one above, because otherwise the demos would be lame. Mainly multiple arguments and adding binary operators.) But the real takeaway here, I hope, is just how straightforward type checking can be. If you see some literal, you can its type. If you have a variable, you can look up its type. If you have a type annotation, you can the type of the value and it against that annotation. I have found that following this formula makes it quite easy to add more and more features.

TypeScript Programming Tutorial

▲ 0

0 views

Jimmy Miller 2 months ago

The Overly Humble Programmer

The competent programmer is fully aware of the strictly limited size of his own skull; therefore he approaches the programming task in full humility - Edsger W. Dijkstra The Humble Programmer Humble is a rather unlikely term to be found in a description of the "modern programmer". Can I suggest: proud, grandiose, delusional, self-absorbed, fad-chaser, bro-y. Even if we ignore the egregious cases of the cryptobro and the AI doomer, programmer discourse is full of complete dismissals of others and overly grand claims about one's own (or one's tribe's) work. Programmers always have the answers. They always know best practices, they are always sure of their technology choices; they could always build it better. Even when we step outside of this online bubble, when we peel back this toxic, frustrating environment, we might still feel the pull that we are not humble enough. Aren't we always so sure of ourselves? Aren't we always over-confident in our abilities, always believing our code is good enough, good enough without more testing, without proof, without formal methods? Don't we all need to learn the lesson Dijkstra tried to teach us a half-century ago? Shouldn't we be more humble? It is a diagnosis many of us believe, and the cure we have sought is precisely what Dijkstra prescribed . I now suggest that we confine ourselves to the design and implementation of intellectually manageable programs. Linters, type checkers, libraries, well-designed languages, these are all tools that we've made since the times of Dijkstra to limit the intellectual burden of writing our programs. The beautiful thing about software is that the size and complexity of "intellectually manageable programs" can change over time as we develop better technologies. This approach of building abstractions, as the key to managing software, has taken us far. But I can't help but feel we may have taken the advice too seriously . Or at least I have, at one point in my career. Not directly, of course, I did not sit and read The Humble Programmer and devote myself to its teachings. But Dijkstra's advice, filtered through the culture, made its way to me. I believed it deeply. And I took from it a lesson I'm sure Dijkstra never intended: don't try hard things. Perhaps the bros in Silicon Valley could use some humility, but for the rest of us, we are already quite a bit too humble. We've created a culture in which "library authors" are the ones equipped and ready to handle complexity. We've taken the fantastic idea of libraries to the extreme. Leftpad has become a common punchline in this regard, but we have to ask ourselves, why do programmers feel the need to grab these kinds of micro libraries? It could be that they are lazy. Or perhaps because it has been ingrained in them over and over again, "smarter people have already solved this problem". This myth of "smarter people" is the retort made to anyone who dares attempt to write things from scratch. When I talked about building an editor in rust , I was asked why I would do that. Editors are hard and smart people have already made them. When I wanted to learn machine code, I just kept being told to write assembly instead, but it turns out machine code isn't scary . But I'm not upset at these people. They, too, have internalized the message we've all been told. All of us. Even famously accomplished programmers. "Nobody can build a C++ compiler. That's not a thing. Why are you even messing around with this?" - Episode 205: Chris Lattner Interview Transcript As someone who started coding for the "web", I always believed that I wasn't smart enough to understand the layers on which my software was built; the layers written in those arcane "low-level" languages. But not only that, I believed that we ought to transcend low-level programming. That the correct answer for all problems was a sufficiently high-level programming language and a sufficiently smart compiler. I believed that I had to stand on the shoulders of giants. That I could not forge my own path, create my own tools, start from scratch. Giving up on this belief has brought me back to the fun difficulties that brought me into programming to begin with. I had to start being less humble. Being less humble does not have to mean giving up on all dependencies. It's not about going back to C99 and swearing off all modern programming languages. It does not mean giving up on garbage collection, giving up on structured programming, and creating a commune where you worship the sacred numbers. It is about questioning those doubts about your own abilities, about thinking through your problem from first principles. It is about being willing to learn the things one level deeper than your current abstraction boundary. To not accept that "how things work" is how they need to work. It is about dealing with the pain the abstraction is attempting to fix. We have increasingly grown accustomed to (and are encouraged to) treat the abstractions we build on as black boxes. We need not do that. Our software hierarchy does not need to create a social one. You are not worse than the kernel hacker, the compiler engineer, or the game engine programmer. You, too, can build whatever you want to build. All software is just software. It's time we treated it that way.

Rust C++ Programming Career

▲ 0

0 views

Jimmy Miller 7 months ago

Stuck? Build Your Language Backwards

I can't count the number of times I started trying to build a programming language just to get stuck in the parser. Some of this was certainly my lack of experience writing parsers, but it was also just how many different choices I had to make. Which way would I parse things? What syntax am I going to use for X? How do I deal with parse errors? Inevitably I abandoned the project before I ever got to the parts I really wanted to learn, namely, all the rest of building a compiler. That is until I stumbled upon, by accident, the technique that has worked flawlessly ever since. Building the language backwards. This means picking the lowest level you intend to target and then writing code that produces it. So, if you are building a bytecode-interpreted language, write a simple bytecode interpreter and something that spits out bytecode. If you are targeting machine code, write a little assembler. If it's wasm, write some code that generates wasm. I have found this an incredible way to get myself unstuck. It may sound counter-intuitive. You probably already have an idea of the language you want to build in mind, so why not start with what you already know? The key for me was that it was so unclear, in my head, how to get from high level -> low level, but once I had the low-level thing already in place, I could see the tediousness, the repetition, the bookkeeping, all the things my higher level language would get rid of. Bridging that gap became much easier. Ever since accidentally stumbling upon this idea, I have had way more success in getting my language projects past that initial hump and I learned quite a lot more along the way. So if you're stuck, try it out and let me know if it works for you.

Programming Tutorial

▲ 0

0 views

Jimmy Miller 7 months ago

Machine Code Isn't Scary

The first programming language I ever learned was ActionScript. Writing code for Macromedia's Flash might be the furthest away from "bare metal" as you can possibly get. As I continued learning new languages, this starting heritage stuck with me. I was mostly interested in high-level, "web languages". Low-level languages felt impenetrable. Over time, I learned a bit more about them here and there, but for some reason, this notion stuck with me. Low-level things are scary, and machine code epitomized that most directly. When I Googled things asking about writing in "straight machine code", I was met with discouraging messages rather than learning. Eventually, I decided I needed to overcome this barrier if I was going to achieve my goals. In doing so, I learned something I didn't expect. Machine code isn't scary. If you can make sure your JSON conforms to a JSON schema, you can write machine code. One problem with machine code is that there isn't simply one standard. There are many different "instruction sets" depending on the processor. Most modern PCs use x86-64 machine code, but newer Macs, Raspberry Pis, and most mobile devices use ARM. There are other architectures out there, especially as you go back in time. The goal of this article won't be to give you a deep understanding of any particular instruction set, but instead, to give you enough information about how machine code typically works so you cannot be afraid of machine code. So we will start by having our examples be in ARM 64-bit (also written as aarch64). Once we have a decent understanding of that, we will talk a bit about x86-64. To understand the basics of machine code, you need three concepts: Instructions are exactly what they sound like; they are the code that will run. Machine code instructions are just numbers. In fact, in AArch64, every instruction is a 32-bit number. Instructions encode what operation the machine should run (add, move, subtract, jump, etc.) and accept some arguments for what data to operate on. These arguments might be constants (meaning like the number 2; these constants are often called "immediates"), but they can also be registers or a memory address. For now, just think of a register as a variable and memory as a list. Here is an example of the instruction add immediate . Now this might look a bit confusing, but once you've seen these tables long enough, they start to be fairly straightforward. Each column in this table represents a single bit in a 32-bit number. If the value is a 0 or 1, that just means it is already filled in. If it has a label, it is a variable that needs to be filled in. tells us whether the registers we are going to use are 64-bit or 32-bit registers. stands for shift. goes in conjunction with imm12, which stands for a 12-bit immediate (constant). So if we want to add to something, we would put in for and set sh to 0 (meaning we aren't shifting the number). But what if we want to represent a number larger than 12 bits? Well, the add instruction doesn't let us represent all such numbers; but setting sh to 1 lets us shift our number by 12 bits. So for example we can represent by leaving our 42 alone and setting sh to 1. This is a clever technique for encoding larger numbers in a small space. Variables that start with R are registers, in this case, Rn is our argument to add, and Rd is our destination. So the above instruction can be thought of like this: Our add instruction is really just a data structure where we put the right parts in the right places. Registers are small places to store values. Every instruction set will have a different number of these registers, different sizes of registers, different kinds of registers, and different naming conventions for registers. For AArch64, there are 31 general-purpose registers numbered X0 through X30 for 64-bit registers. Let's say we want to add 42 to register X0 and store the result in X1; we use this binary number. To encode our registers into our instruction, we just use their number. So register X0 would be 00000 and register X18 would be . Registers are simply places where we can store values. But by convention, registers can be used for different things. These are called calling conventions and they are how "higher" level languages like C encode function calls. Writing out all these binary numbers all the time (or even converting them to hex) can often be tedious. So instead, we usually talk about instructions in a simple text format called assembly. In order to feel cool, people usually write numbers in assembly as hex values. This is just the number 42. You can see that assembly hides some of the details of the encoding we just made. We don't think about sf, sh, what size our number is, that a register is Rn vs Rd. Instead, the destination comes first and the arguments after. Because of this lack of detail, a single assembly instruction might actually map to many different machine code instructions depending on its arguments. The last piece we have to understand for machine code is memory. To understand what is going on with memory, we will look at an instruction that lets us store things in memory. This instruction is called STR or not written in shorthand, store. Using this instruction, we are going to store some value (RT) into the address (RN) + some offset (imm12). So if we think about memory as a big array, this instruction is like writing into that array. . The x here is like our sf before, it controls whether we are using 64-bit values or not. If we want to make this concrete, let's say we have a value in X2, we have an address of memory in X1 and we want to store a value 2 bytes offset from that. We would get this structure: Since writing that all is tedious, we often just write the assembly notation. We are storing the value in x2 based on the address stored in x1 + 2. X86 encoding is a bit different, but it more or less has the same parts. We are still working with instructions, registers, and memory. Some names are a bit different. Instead of the consistent 0-30 naming, we get the historical baggage of the following 64-bit registers: rax, rbx, rcx, rdx, rsi, rdi, rbp, rsp, r8-r15). However, the biggest difference is that x86 is not a fixed width instruction set. We can't simply give a nice little diagram of every instruction using 32 bits. Instead, instructions are assembled from parts. These parts are given different names; when you see an instruction encoding, it tells you how to put the parts together. The first part is called the REX. This is a prefix that we can use to help us with 64-bit operations. Not sure if there is an official justification for the name REX, but my understanding is that it is the "Register Extension Prefix". Unfortunately, because the REX is a prefix, it will only make sense when we see what comes later. REX is there for backward compatibility. The W in REX lets us signal that we are using 64-bit or not for certain operations. The R and B will "extend" our registers in certain operations. In other words, it allows more registers than you used to be able to (These are those r8-r15 registers with a different naming convention than the older registers). We need these because, before 64-bit x86, we had fewer registers and our instructions only had 3 bits per register. With 16 registers, we need an extra bit. (X is for the SIB structure, which we don't cover here). Our next part is ModR/M. ModR/M keeps up with the tradition of naming things incredibly short and confusing names. actually means Mode. tells us if is acting as a register or if it is a pointer to memory. If then rm is being used as a register, otherwise, it is being used as a pointer. just is a register. is simple, it is a number. It can be 1-3 bytes long. There are other parts, but we won't cover them here. With just these parts, we can build up an instruction. Let's say we want to move a 32-bit signed immediate to a 64-bit register. We can consult a table of instruction encodings and we will get this: So now we can assemble our parts and make our instruction. Let's start with REX.W. This notation just means REX with W set to 1. Then there’s B8, which is just a number written in hex. is yet more shorthand for using the ModR/M but setting the reg to 0. Finally, means "immediate doubleword", in other words, a constant number that is 32 bits long. So given all that, we can write our instruction. So let's move the number 42 to the rbx register. Why is RBX 011? Well, because the table says so. Yeah, I did say that x86 is a bit weird. I won't pretend that this is all you need. But I will say that starting here can get you further than you think. There are some other things to learn, like various flags for things like overflow, there’s also calling conventions, which are about which registers you use when for things like function calls. We haven't really talked about the stack here, but that's memory that you write to to keep track of things. Nor have we talked about jumps, or how to encode larger immediates in ARM, but you’ve gotten the basics. It’s easier than you would think to hop on compiler explorer and learn how things are done. Learning machine code and writing things at this low level has unlocked so many things that were mental blocks for me before. Relying on libraries made by others to do these low-level things always left a gap in my knowledge that made me doubt my understanding. Even if I intellectually could explain things, actually doing them has made a huge difference for me. So if you, like me, find low-level things intimidating, I can't recommend enough starting from scratch, at the lowest possible level for your task. What I've found over and over again with low-level details, their not hard, their just poorly documented and poorly explained. Instructions

Programming JSON

▲ 0

0 views

Jimmy Miller 11 months ago

Discovery Coding

I don't take notes; I don't outline, I don't do anything like that. I just flail away at the goddamn thing. - Stephen King In writing (particularly fiction writing), there is a generally accepted distinction between authors who start by writing an outline and those who discover their stories through the process of writing. For some reason, we have no such distinction in programming, so I am here to introduce it. Discovery coding is a practice of understanding a problem by writing code first, rather than attempting to do some design process or thinking beforehand. This means that a discovery programmer does not always start with a plan before starting to write code; rather, they think about the situation they are in. What is the tension we are trying to solve with this new code? What are the situations that have given rise to this demand? How do these various bits of the system interact? It is only in the process of writing code and seeing the pushback the code gives that discovery programmers understand and devise a plan forward. Discovery programmers can often be seen as messy or undisciplined. When working with their outlining counterparts, there may be some tension that isn't properly placed. The outline programmer may feel unease when seeing the haphazard approach of the discovery programmer. The discovery programmer, on the other hand, may be confused by the seemingly incorrectly timed questions posed by the outliner. In our current culture that prefers more structured technologies (static type systems, enforced memory safety, schema enforcement tools, etc.), it is easy to label discovery coding as a bad practice, but it is important to distinguish the process of creation from the end-artifact. There is no reason that a discovery programmer cannot create a highly structured, rigorous end-artifact. Discovery coding and bottom-up design are not the same thing. It wouldn't surprise me if discovery coders often tend to enjoy bottom-up design, and outliners prefer top-down design. But these are orthogonal axes. Top-down design is about the angle by which you tackle a problem, not the process by which you go about understanding it. Imagine this: you write a detailed design document well ahead of coding, laying out core primitives and how they will compose to solve the problem. Here we clearly have an outliner doing a bottom-up design process. Even for the outliner, I think discovery coding can have quite a few benefits. When approaching problems looking for a solution, it is very easy to start with those solutions we are most familiar with. Discovery coding, by its nature, avoids this pitfall. Discovery coding does not have a solution to offer, so the code we begin writing is instead about poking the system and understanding how it works. By doing this, you are filling your mind not with the context of past solutions or descriptions of solutions others have told you. Instead, you are loading up a series of constraints that the system has that your solution must satisfy. For me, this is the only way I can do anything. Anytime I try to outline before I write code, my outline is thrown away within hours of writing code. My design docs that are written beforehand are wholly unsatisfying, even if they get approved. It is only as I begin writing code that I begin to understand my problem. So the next section here won't be on the benefits of outlining. Not because there aren't any, but because I'm not the person to write those. (Nor will I write about the downsides of discovery coding, because I am certain everyone can list those.) I don't think many tools today are designed with discovery coding in mind. Things like live programming in a running system (see how (most) Clojurists or SmallTalkers work) are incredibly valuable for a discovery programmer. Systems that let you visualize on the fly, instrument parts of the system, etc., make it much easier and faster to discover things. This is the sense of dynamic (unrelated to typing) that I feel we often lose sight of. The writing community has learned to accept that some people just are discovery writers. That the very process of outlining before writing can cause some people to be unable to write in the way they want to. My exhortation for the programming community is that we ought to do the same thing. What I'm not advocating for is the banning of all design documents (those can be useful), but instead the recognition and acceptance that some people simply think differently. That being less organized isn't an inferior approach. That being able to think through details before you code isn't superior. Perhaps I have the wrong read of the current cultural milieu (or perhaps you are reading this years later and culture has shifted). Perhaps discovery coding is loved and cherished and outlining is scorned. If so, please invert all of the above. What I hope we can recognize is that there is no one answer. Designing software is a human process and not all humans are the same.

Design Programming

▲ 0

0 views

Jimmy Miller 1 years ago

Dec 24: Against a Universal Definition of ‘type’

This is the final entry in the series and I'll be honest, I'm a bit glad it's over. It's been hard to consistently find the time every day to 1) find a paper I'm truly interested in writing about 2) read that paper in a way I can summarize it 3) write something somewhat interesting about the paper. Honestly, I think finding the papers that fit my mood was the hardest part. I tried to frontload that by making this big list of papers, but I just couldn't be bothered to seriously read some of them. It's one of the reasons I slipped a few philosophy papers into the mix. It's so much easier for me to find a good philosophy paper than a good computer science paper. Another goal was to mostly read new papers. I'd say I did alright, but not great at that task. There were a few papers I hadn't fully read. Some that I had definitely read before. Today's paper is in the category of having read some of, but not completed the paper. It's a paper I'm quite fond of but have complicated feelings about. In this paper, Tomas Petricek argues against giving a universal definition of "type". The paper begins by exploring the ways in which our use of the word "type" has changed over time. Starting with Russell's theory of types to rule out paradoxes. He then goes on to discuss more modern uses of types like unsound types systems, dependent types, and type providers. Along the way, he points out how the old way of using types does not quite fit with this new way of using types. If you find this controversial, I highly suggest reading the paper as I think Petricek is rather convincing on this point. The rest of the paper explores the idea that we really shouldn't have a universal definition of type, because by not having one, we are allowing the concept to expand. This is actually a good thing. People are able to make use of this concept of a type, without immediately formalizing it. Allowing it to take on new shapes. Looking at the history of science supports the main idea of this essay. That is, we should not require a precise definition of the notion of ‘type’. Requiring clarity means that we can only talk about things we already understand – perhaps in greater detail, with more generality, and in more elegant ways, but not about fundamentally new ideas. Petricek continues by showing ways we can live with the fact that types do not have a clear definition. All of his examples are borrowed from philosophers. The details are a bit too subtle here for me to repeat here so I will leave those to people who decide to read the paper. But they are certainly interesting suggestions. If we want to talk about types outside of a narrow research programme, we need to find ways of dealing with types without a precise definition. I proposed three alternatives –those are based on how we use types (inspired by Wittgenstein’s language games), what is the conventional idea of type (based on Putnam’s stereotypes) and what we can do with types (inspired by Hacking’s new experimentalism). I believe that these provide worthwhile methods of inquiry that can provide interesting insights into what types are outside of a narrow boundary delimited by a particular formal definition. I think this is entirely right, if our choice is between a singular, formal definition of types and these suggestions, I choose the latter. But I want to quickly suggest that a formal definition isn't the only kind of definition we can give. Let's consider another word that lacks a universal definition in programming, abstraction. There are so many varied uses of abstraction. No formal definition could ever capture how we use the word abstraction. But I do think having a better, clearer, universal definition of an abstraction would benefit us. How does "an abstraction" relate to things being "more abstract" or "less abstract"? Do abstractions come in levels? Is abstraction always information hiding? How do abstractions relate to notions like interfaces? I do feel these questions have answers and those answers would be illuminating. In the same way, I believe there is some (non-formal) unifier core behind these notions of types. When we create new applications for types, (e.g. type providers) they are new applications for "types" because of some shared underlying way of viewing types. This is something I feel is completely lacking in software. We seem to think that there are two things, undefined things and formal definitions. Of course, in many ways, this paper is such an exploration of types. But it is largely historical, stopping short of trying to synthesize how we do talk about types. Meaning as use doesn't have to mean we just need to find more contexts of use. It can also mean, that if we look at the current way we use the words, we can discover their meanings.

Programming Science

▲ 0

0 views

Jimmy Miller 1 years ago

Dec 23: Do Artifacts Have Politics?

One question that I don't think gets asked enough is how "good code at work" differs from "good code at home"? For many years I didn't quite see the distinction between the two. Isn't it a fact of the matter what things make for good code? Why would code I write for myself have different best practices than code I write for work? Now one obvious answer may be context. At work, there are more people working on the code, so perhaps I need to consider approachability more. Or there are more eyeballs, so maintenance might be more likely to occur. Or tons of people already use this dependency, so I shouldn't be worried about taking a dependency. But as time has increased, I've become skeptical that these contextual differences should be my main focus. Instead, I've started to realize that the major difference is in the values I hold as an individual programmer versus the values implicit in the structure of the companies I work for. Today's article by Langdon Winner looks at these sorts of facts not necessarily in code, but the artificial, technological landscape we build. This paper is remarkably well-written. It is a joy to read, the points made are clear and nuanced. Winner talks about the standard reaction to the idea that artifacts have politics, which is that they are simply misplaced. People have politics and use artifacts to those ends. His response is a fantastic representation of the sort of writing throughout. Hence, the stern advice commonly given to those who flirt with the notion that technical artifacts have political qualities: What matters is not technology itself, but the social or economic system in which it is embedded. This maxim, which in a number of variations is the central premise of a theory that can be called the social determination of technology, has an obvious wisdom. It serves as a needed corrective to those who focus uncritically on such things as "the computer and its social impacts" but who fail to look behind technical things to notice the social circumstances of their development, deployment, and use. This view provides an antidote to naive technological determinism, the idea that technology develops as the sole result of an internal dynamic, and then, unmediated by any other influence, molds society to fit its patterns. Those who have not recognized the ways in which technologies are shaped by social and economic forces have not gotten very far. But the corrective has its own shortcomings; taken literally, it suggests that technical things do not matter at all. Winner goes on to show in great detail how artifacts can shape the Forms of Order of a society. I think his most compelling example is the move to make cities more accessible for people with disabilities. The design of countless technologies limited (and still does limit) people with disabilities from participating fully in society. These are political implications of the technologies themselves. If don't find this particular example convincing, I highly recommend reading the paper as it is chock-full of examples. He next tackles the stronger notion that certain technologies are inherently political, that is by their nature they bring about certain political structures. The discussion here is far too involved and subtle for the time I have here but ultimately, Winner concludes this too is true. It is still true that, in a world in which human beings make and maintain artificial systems, nothing is "required" in an absolute sense. Nevertheless, once a course of action is underway, once artifacts like nuclear power plants have been built and put in operation, the kinds of reasoning that justify the adaptation of social life to technical requirements pop up as spontaneously as flowers in the spring. This paper was a joy to read. It makes me think more deeply about my own code and the values implicit in it. My code at work is made with a structural goal of providing money for the company I work at. Its function supports certain social structures I may not fully agree with. The form the code takes necessitates a managerial structural system. And many more facts besides This value structure is completely lacking in the code I write for myself. Shouldn't that be reflected in the kind of code I write? I'm starting to feel this pull. A pull to rethink the values I hold about code, to find new ones that fit the sort of world I want to exist, that support the sort of structures I believe help the world.

Programming Java

▲ 0

0 views

Jimmy Miller 1 years ago

Dec 22: Once More—A Computer Revolution

Ran out of time today. Will the home computer be as pervasive as today's television se,s? The answer almost certainly is no. The home, pictured in the accounts of home computer advocates, is middle class, or even upper middle class. There are some appliances computers must control: the wall-to-wall carpeting must be cleaned by a robot, roasts are in the oven, the computer helps "the mother" pay the telephone bill, and so on and on. This is a fascinating critique. Completely wrong in its prediction but captures something all too true. We may recall the euphoric dreams that were articulated by then Secretary of Commerce Herbert Hoover, at the dawn of commercial radio broadcasting, and again by others when mass television broadcasting was about to become a reality. It was foreseen that these media would exert an enormously benefcial influence on American culture. Americans would be exposed to correct English, to great literature, great drama, and so on. We all know what actually happened. The technological dream was more than realized, but the cultural dream was not.

Culture Media

▲ 0

0 views

Jimmy Miller 1 years ago

Dec 21: What is Conceptual Engineering and What Should It Be?

I've loved choosing philosophy papers for this series. And so, I decided today to do the same. But before you click away, this one is incredibly applicable to software. In fact, David Chalmers thinks that Software Engineering helps address a worry about Conceptual Engineering. Incidentally, I like the software engineering analogy because it addresses a worry about how conceptual engineering can work if concepts are abstract objects. It’s not obvious that you can engineer a new resident of the third realm. Exactly the same issues arise for software engineering. Programs are usually thought of as abstract objects, but we have no problem saying that programs are engineered. Chalmers is of course quite right that we engineer abstract objects ( see day 2 ), but I don't think he realized just how tied together conceptual engineering and software engineering are. But I guess we are getting a bit ahead of ourselves, so along with Chalmers we must ask. What is Conceptual Engineering? What should it be? Conceptual engineering is about "designing, implementing, and evaluating concepts". One great example given in the paper is Ned Block's distinctions between phenomenal consciousness and access consciousness. Leaving aside what these terms mean, Ned Block developed them as a way to distinguish two things that were both simply being called "consciousness" that were actually distinct phenomena. By giving us these terms, it made it easier to pick different aspects of "consciousness ". But this example might not be the best for programmers. So instead, think about MVC. The concepts of Model View and Controller were developed by Trygve Reenskaug at Xerox PARC. Many people (including myself) have many thoughts (not all good) about MVC. But there is no doubt that the concept of MVC has played an important role in software development since these concepts were "engineered". In the philosophical world though, less attention has been paid to examples like MVC cases where new concepts are created (De Novo) and more attention has been paid to the fixing of deficient concepts. Conceptual engineering of this sort is often applied to societal issues around topics like race and gender. For these topics, we are not trying to create new concepts nor are we trying to ask how people actually use these concepts. Instead, we are asking "what should these concepts be?" What do we mean by what "should" these concepts be? Well, it may be that the concepts we have are inconsistent and should be replaced by consistent ones. It may be that the concepts are part of some unjust system of beliefs and we ought to revise them. There are a number of different reasons we may want to fix our concepts. Chlamers’ paper covers a lot more ground, talking about various questions in the philosophical landscape around conceptual engineering. What does it require to change a concept? When we create a concept, should we use the same word as an existing concept or pick a new word? How does conceptual engineering relate to disputes that are merely verbal? I think all of these are actually very interesting and relevant for us in software, but I will leave those details to those who wish to read the paper. What isn't talked about at all is the way in which conceptual engineering plays out in software. It's my contention that much of what we do day to day is conceptual engineering, but we never directly talk about it. If you are in an Object Oriented language, you have to pick class names. By their very nature, these classes add concepts to your program. The notion of a Customer in your code base is not the same as the general real-world notion of a customer. It is a custom, particular concept of customer, which stands in relation to the customer out there, but it isn't actually that real customer. As you continue to develop your codebase, you find that there is a new sort of (in the real world) customer that needs to be accounted for in your codebase. What do you do? Do you change the concept of Customer? Do you introduce a new concept of SpecialCustomer? Whatever your decision, your job in conceptual engineering doesn't stop there. Now that you have made this conceptual change, how do you get your fellow coworkers to adopt this new concept? These are concerns we have constantly as software engineers. We are surrounded by concepts in need of re-engineering. We are surrounded by problems in need of new concepts to help us understand them. Because we don't think of conceptual engineering as a first-class part of our job, we don't have tips and tricks for doing these conceptual engineering roles. We don't think about the fact that problems can be avoided merely by changing our concepts. There is so much more to say about conceptual engineering. If you are interested and want a full-length treatment (but warning, it is a philosophical text, not a software engineering one), I highly recommend Fixing Language: An Essay on Conceptual Engineering

Programming

▲ 0

0 views

Jimmy Miller 1 years ago

Dec 20: Three Paradigms of Computer Science

In late 2019, I was invited to give a talk at Rebase in Berlin during the summer of 2020. As you can imagine, that didn't happen at least not in person. Instead, I gave a pre-recorded talk. I tried over and over again to sit at a desk and give a talk as I'd seen many people do, but it just felt so lifeless. So I contacted a friend and I was able to access his company's studio. It was just me, my brother, and my friend. Socially distanced, and the two of them masked, as I gave a talk about Thomas Kuhn and programming paradigms . I think the talk was overall good. I fumbled a bit at the beginning, but I stand by the point I was trying to make. Programming Paradigms are not Kuhnian paradigms. (Perhaps I should write this up at some point). However, as part of this talk, I intentionally did not talk about the paradigms of computer science. When asked in the QA about that, I said, I'm an outsider so I don't really have any opinions on that. But anytime I say I don't have opinions, that's a lie. This paper by Amnon H Eden talks about what they take to be three Kuhnian Paradigms of computer science. In his seminal work on scientific revolutions, Thomas Kuhn (1962) defines scientific paradigms as “some accepted examples of actual scientific practice… [that] provide models from which spring particular coherent traditions of scientific research.” The purpose of this paper is to investigate the paradigms of computer science and to expose their philosophical origins. I'm not quite sure they succeed. But before we get there, let's see what the paradigms are supposed to be. (§2) The rationalist paradigm, which was common among theoretical computer scientists, defines the discipline as a branch of mathematics, treats programs on a par with mathematical objects, and seeks certain, a priori knowledge about their ‘correctness’ by means of deductive reasoning. (§3) The technocratic paradigm, promulgated mainly by software engineers, defines computer science as an engineering discipline, treats programs as mere data, and seeks probable, a posteriori knowledge about their reliability empirically using testing suites. (§4) The scientific paradigm, prevalent in artificial intelligence, defines computer science as a natural (empirical) science, takes programs to be on a par with mental processes, and seeks a priori and a posteriori knowledge about them by combining formal deduction and scientific experimentation. Despite the title seeming to be simply descriptive, this paper actually argues for the scientific paradigm. I mean, who is surprised by that one? It is rather unusual for someone to name some "scientific" and then not endorse it. I must say, I think I understand the characterization of the rationalists and technocratics quite well. But I'm a bit lost as to what this "scientific" paradigm is supposed to be. Consider this statement of scientific ontology: Program-scripts are on a par with DNA sequences, in particular with the genetic representation of human organs such as the brain, the product of whose execution—program-processes—are on a par with mental processes: temporal, non-physical, causal, metabolic, contingent upon a physical manifestation, and nonlinear entities. "Program-scripts" here just means the text of the code. Where "program-processes" means the running of that code. What are we supposed to take from this ontologically? DNA is a physical thing that has complex causal relationships. Written down DNA sequences may be thought of using the analogy of code. But how is code itself like molecules in a double helix structure? And why is code execution like a mental process if it is DNA? "Running DNA" doesn't result in a mental process. I'm confused, is the ontology offered simply metaphors? Is there anything I'm supposed to take literally here? If you think I'm being weird and picky, contrast this with the ontology statement given for the rationalists. A program-script sp is a mathematical expression. A program p is that which is fully and precisely represented by sp. Hence, p is a mathematical object. Mathematical objects are platonic universals. Therefore, programs are universals. This is a straightforward ontological statement. The confusion for me continues throughout much of the section arguing for the scientific view. But mathematical objects, such as turing machines, recursive functions, triangles, and numbers cannot be meaningfully said to metabolize nor have a causal effect on the physical reality in any immediate sense. It would be particular difficult to justify also a claim that mathematical objects have a specific lifespan or that they are contingent upon any specific physical manifestation (except possibly as mental artifacts). In this respect, rationalist ontology is inadequate. Since when do programs metabolize? The best we get is the assertion that the "program-process" consumes energy. There is nothing about that fact that is contrary to anything mentioned by the rationalist. One point I was going to make is that part of why these aren't Kuhnian paradigms is that Kuhn's paradigms are not mutually comprehensible. Maybe the statements I made above actually lend evidence this is the case. But in general, I think that is one problem here for calling this Kuhnian. But the real major issue is that for Kuhn, a paradigm always wins. There is always progress and a singular dominant paradigm. If these point to anything, it is a pre-paradigm phase. Or I think more realistically, that computing isn't a science. As described, I'm not sure these views are quite right. I do think the paper has some interesting ideas and is worth a read. Personally, I find the contrast between the rationalist and technocratic views to be a bit of a false dichotomy. Perhaps, this is my suggestion, what do we get if we combine the various elements presented here? Can we mix and match and find our own paradigm that isn't one of these things, but perhaps parts of all of them?

Science Java Programming

▲ 0

0 views

Jimmy Miller 1 years ago

Dec 19: Everybody Clap Your Hands

Have you ever been at a wedding and thought, "You know what would really spice up this dance floor? Arbitrary computation!" this paper is for you. Today, was a busy day for me, so you will get a very short summary of a very silly paper. Everybody Clap Your Hands: The Cha-Cha Slide is Turing Complete by Harrison Goldstein. The Cha Cha slide is in fact turing complete. Show this relies on only a few well known features of the Casper slide part 2 and some perhaps, strange ways of interpreting what is going on. The dance floor is always in one of two states, Funky or Not Funky, which correspond to the states F and N from the machine 𝑀. The DJ can memorize the current state if they wish, but if everyone is doing their job it should be obvious to all involved whether the room is Funky or not. Funkyness is perhaps my favorite aspect of this scheme. You may be thinking (as I was), I know how to make it funky, but how do you make it not funky? The obvious answer, "freeze"! Other than that, we get things like moving the tape by sliding. We get writing to the tape by hopping. The dancer in front of the DJ is in the hot seat and can signal their state by counting in binary on one hand. Since few people have 18 fingers, the count can be shown in binary as illustrated in Figure 2. To avoid being too rude, the original Turing machine might do well to avoid symbol 4 when possible. In any case, the important thing is that the hot-seat dancer communicates their cell’s content to the DJ. In general, the scheme makes sense. There are some extensions that might make things a bit easier to encode things, but I like my turing machine equivilants to be minimal. The paper ends with some discussion about the pointlessness of turing completeness as a metric. But honestly I don't think it needs it. The paper mentions internet trolls saying things about turing completeness. But I find that kind of discourse boring. Instead, I want to ask, is a freestyle cha cha slide a programming language? What if the dj (and the dancers) aren't trying to compute anything at all, they are just experimenting with dancing? When we have debates and say try to include something not traditionally thought of as a programming language into that category, does that same reasoning apply to these Cha Cha slide dancers? Do we want it to?

Programming

▲ 0

0 views

Jimmy Miller 1 years ago

Dec 18: The Structure and Legal Interpretation of Computer Programs

I do not have much time to write today. I've been dealing with the seemingly never-ending problem of getting a bed frame delivered properly. It's been weeks of dealing with customer support to get the hardware, for it to arrive today, and halfway through getting the bed together realizing I'm still missing a crucial part. Then hours returning the bed. So you are getting a very short summary of a paper that deserves much more to be said about it. James Grimmelmann has written this wonderfully titled and fantastic paper about the meaning of programs. He walks through three different wants of understanding the meaning of a program, naive functional meaning, literal functional meaning, and ordinary functional meaning. All of these ways of taking meaning find their use in law. I won't detail what they mean here, but I will say as practitioners, we do have to deal with these different kinds of meanings all the time. Ordinary functional meaning and literal functional meaning are involved when debugging software. But despite not telling you what these meanings exactly are, I will give you the punch line. The meaning of our programs depends on social facts, it isn't merely objective. This is a fascinating conclusion that I believe is well argued for. If we are to find a grounding for the meaning of our programs, they must be grounded in social facts. Facts like what the community agrees is normative for language definition. Much of what Grimmelmann argues for about software is in concert with discussion in day of our papers about software ontology. But literal functional meaning also connects with day 17 about the ontology of programming languages. This paper is a wonderful exploration of program meaning, particularly in a legal context. I will admit, I haven't found a ton of great papers about programming from the social sciences (please if you know of them share them with me!). But I've been finding quite a few more from a legal context. This is a refreshing read that takes seriously programming in its social context. It's very readable and I can't recommend it highly enough.

Programming Science Legal

▲ 0

0 views

Jimmy Miller 1 years ago

Dec 17: The Cultural Part of Cognition

As part of this series, I set out to read papers in my backlog of papers, that I literally hadn't read at all. This paper is one of them. It's a paper published in 1981 that from the abstract seems to be drawing a comparison between "cultural programs" and computer programs. But there are times when I read a paper and I feel I have understood every word, but seem lost as to what the paper is really about. This is one of those times. Roy Goodwin D'Andrade wants us to focus on more than just how cognition works or in other words, how the brain operates. The proposal of this paper is that we should focus on the "cognitive organization of culture". Most of what we know comes from cultural transmission. As a species, we are remarkable for our ability to pass information down through generations in a way that grows over time. But what shape does this information take? It must be a shape that is a natural fit for the way our cognition works. An important assumption of cognitive anthropology is that in the process of repeated social transmission, cultural programs come to take forms which have a good fit to the natural capacities and constraints of the human brain. Thus, when similar cultural forms are found in most societies around the world, there is reason to search for psychological factors which could account for these similarities. Much is made about this idea of cultural programs. But at each turn, we are just told how they differ from computer programs. [O]ne major difference between cultural and computer programs is in the degree of precision with which computer programs must be stated" How does this great difference between the highly specified computer program and the learned-by-guided-discovery cultural program relate to differences in human versus computer processing characteristics? Among other things, this difference may have to do with the fact that humans must learn their programs, while computers "load" theirs. Another example of a major difference between human and computer programs is the strong tendency for human problem solving procedures to be highly local and content specific, rather than global and formal. We are never really told how these two are supposed to coincide. I take it this is just taken for granted. But I'm honestly not sure I see it. In which way are cultural programs like computer programs? One example given a cultural program is playing chess at the Master's level. We already know quite well that computer programs play chess quite differently than people do. Even more mundane things, like how to use a pencil, seem to be examples of cultural programs. What do these have in common at all? One current formulation in anthropology treats the pool of cultural information as if it were a pool of algorithms or programs, a giant stock of procedures and representational declarations embodied primarily in natural language I thought this "one current" was going to lead us to an alternative formulation. But as far as I can find, it never does. Is this really the way this paper thinks about all the things we do in life? Am I currently following a pool of algorithms to write this paper? I never intended this to be yet another paper on computation and instantiation but it seems to be. The best I can see is that this paper is trying to point to how important culture is for the way we think and what we think about. It is trying to say that a study of culture can be a study of the brain. Maybe that first part needed to be said in 1981 to this group of cognitive scientists. As for the second part, it may well be true, but looking at culture in terms of programs doesn't seem all that fruitful to me.

Culture Science

▲ 0

0 views

Jimmy Miller 1 years ago

Dec 16: Will Computers Ever Become Easy to Use?

This paper brings back some nostalgia for me. Not because I was around when it was written (I was 5), but because my computing journey saw me growing up with a different Raskin, Jef's son Aza. Some of the first programs I wrote that had real utility were for the Enso launcher by Humanized. Their Weblog had a ton of articles that were very influential in my thinking about interface design and how programs relate to people. It was only many years later that I learned the connection between Aza and Jef. The Human Interface is a fantastic book and if I ever find myself with enough time to write about books rather than papers, I'll definitely re-read it and write that up. Today's paper though I have to admit is a bit lackluster. The question is an interesting question. But the framing is a bit disappointing. But I've gotten into this habit of reading (skimming) too many papers before making a decision on what paper to post about and I just need to commit if I'm going to make this advent of papers work. Raskin points out things that were true in 1997 and are certainly true today. Users are scared of computers. They do not want to touch things and change them. They need experts to help them do anything complicated. Applications are full of mode errors and all sorts of other design problems that cause endless frustration for non-programmer users. I think this fact can often be under-appreciated by programmers. While we do encounter these flaws in software, we are typically able to work around them or avoid that kind of software. Making settings changes isn't scary. But when I see less technical folk using computers I remember being a kid playing Sim City 3000 Unlimited. I didn't know where to save the game (what folder) and I was scared I would mess something up. I asked my mom and she didn't know either. So the game just never got saved. Raskin sees these problems as fundamental to our application-centric GUI interfaces. Each application offers very different affordances and patterns that must be relearned constantly. Not only are these problems baked into our methods, but we do not have a field of study that could help us design better alternatives. Raskin proposes a focus on what he calls cognetics: "an engineering technology of applied physics, ergonomics, and cognitive psychology". Raskin wants to define "formal measures of productivity". Given these measurements and design work, we will be able to make much better interfaces that are understandable for everyone and much more efficient. He ends the article with quite a bit of optimism about this research project. At present Silicon Valley and the other technological triangles, loops, and glens are somewhat complacent and consider the problem of interface design essentially “solved,” needing only a bit of touch-up here and there. For many users and developers, today’s GUI is the only conceivable interface. Their very success impedes the needed radical improvements. In spite of these formidable difficulties, we will do better. Those companies that cling to the status quo will not be able to also hold on to their customers. I love some of Raskin's radical ideas around dissolving application boundaries. But this paper just struck me the wrong way. Is a productivity measure really our goal in making better interfaces? I guess you could say it is the only goal that is going to "win in the market" or something. But I think that generally is a naive view of the market as well. Enterprise software has absolutely terrible interfaces and is wildly successful. In many ways, they are successful because of how bad the interface is. (People need support and services, etc which makes for stickiness and a thriving ecosystem.) As I look back though, I do see a spirit for redesigning the desktop metaphor that I feel is not very bright today. This is one of the things that makes me sad that so many future of code projects focus on the web. They in many cases assume the very things I think we might want to explore dissolving by putting this platform as a requirement for their work. I get it, it is hard to start over. But I hope we will see that sort of fundamental reworking of the way we interact with our computers again someday.

Programming Design

▲ 0

0 views

Jimmy Miller 1 years ago

Dec 15: Programming Languages as Technical Artifacts

At first blush, the topic of this article may seem no different than our day 2 article Software is an Abstract Artifact . However, Raymond Turner's notion is quite different. First I think it's important that we make clear in this article we are talking about programming languages. In the previous article, we were talking about a piece of software (e.g. Microsoft Word). Is there really a difference between these two? Aren't programming languages pieces of software after all? Go back to the year 1960. McCarthy has invented Lisp, but no implementation of it exists. Is it still a programming language? Or I have written up an operational semantics for a new DSL I'm planning on making. Is my DSL not a programming language? Contrast this with software. Software does not exist unless it is implemented. There is quite a bit more that is different about these articles. Abstract Artifacts are not the same as Technical Artifacts. We will get into that later. But I want to comment on the fact that distinctions like software vs programming languages and their existence criteria are something we pre-theoretical have intuitions about. Perhaps your intuitions disagree with mine. Maybe you think a UML diagram is all that is needed for a piece of software to come into being. But regardless, you have these views. Yet these kinds of questions aren't considered part of computer science proper. Only a handful of philosophers of computer science talk about these issues and yet it isn't as if computer scientists have no views on the matter. For example, Laurence Tratt's article Compiled and Interpreted Languages: Two Ways of Saying Tomato shows how complex the notions of interpretation vs compilation can be. Elements of interpretation and compilation are interweaved in many language implementations. But what it also contains is clear (if a bit implicit) commitments to constraints on the ontological status of programming languages. For example, Tratt does not believe that languages are identical to their implementations. So languages are not the bits on the computer that make up the implementation. But perhaps surprisingly, nor does he appear to believe that languages are identical to their specifications! He says this in a footnote: "Some languages don’t even have a single specification, such as BASIC, allowing many different languages and implementations with “BASIC” in their name to diverge substantially from one another". Perhaps that is a bit vague, in the first part of the sentence he seems to imply the (singular) language Basic has multiple specifications, but then talks about different languages (plural) that are "Basic". Tratt's view is a fairly common one from what I've seen. More explicitly stated views might say something like "Interpreted language" is a meaningless phrase. What they have in mind here I've gathered is that it is a category error. Languages aren't their implementations and only implementations can be interpreted. We will ignore the larger issues of language, how clearly that can't be a meaningless phrase given that people use it and communicate things all the time. (Happy to debate anyone on this) What I wanted to draw out from it is merely that computer scientists already have made these sorts of ontological commitments. This isn't an imposition from the outside. So if a programming language is a Technical Artifact, what exactly is that? Technical artifacts include all the common objects of everyday life such as televisions, computer keyboards, paper clips, telephones, smartphones, dog collars, and cars. They are intentionally produced things. This is an essential part of being a technical artifact. For example, a physical object that accidentally carries out arithmetic is not by itself a calculator. This teleological aspect distinguishes them from other physical objects. This is first in contrast to natural objects. A rock is not made with any particular purpose in mind. It is not an intentionally produced object. But technical artifacts also differ from other socially constructed objects like money, in that there is a necessary connection between function and physical structure. Money does not need to have a particular structure to fulfill its function, but things like a paper clip do. A technical artifact is individuated by the combination of two properties, its functional properties (what it is intended to do) and its structural properties (its physical makeup). A technical artifact is not simply the physical object considered in and of itself. It must have these functional properties as well. (How exactly those are understood is a bit complicated, but we will touch on them a bit more here. See the full article and citations for more information). For our purposes, we are just going to talk about the which takes the function of a technical artifact to be related to the intention of its construction. What exactly specifies the function of a programming language? Well, the paper does I think a pretty great job of diving into topics like operational semantics, denotational semantics, virtual machines, and abstract machines. We don't have the space here to dive into these details, but there is one major point that is important for us to capture here, the physical implementation of a language cannot be the ground of its function. I think for some more academically minded, this is obvious, so I won't defend specifications here. But from an everyday perspective, isn't the "meaning" of a Python program just defined by what CPython does when it runs the program? The fact that there can be bugs in CPython shows this to not be the case. The notion of a bug in an implementation of a programming language requires that there is something outside the implementation itself that specifies the correctness criteria for the implementation. Or to put it in philosophy speak, semantics must provide normative criteria. This is actually crucial for any notion of meaning. The fact that the expression means something implies that there is a whole set of normative truths about my behavior with that expression; namely, that my use of it is correct in application to certain objects and not in application to others. (Boghossian 1989) But "When is an implementation of a language correct relative to its semantic account?" This question actually brings us back to day 4's question Is the Brain a Digital Computer . In John Searle's paper he shows how given a sufficiently complex physical object, we can map that object onto any program specification. This includes a programming language specification! So any object can implement any programming language! (If this confuses you, see section 6 in the paper). Turner actually gives us a way around this problem for technical artifacts. If I observe a man arranging stones in a way that is consistent with the extensional arrangement determined by some abstract machine, I may not legitimately infer that he is building a device according to the abstract machine viewed as a functional specification. He might just be arranging them for esthetic reasons. In general, a physical device may be in accord with an abstract machine without being governed by it. How might we tell the difference? How can we tell if she is under the authority of the abstract machine or building a work of art? Presumably, we can ask: “what are your intentions?”, “why did you do that?” For the former, the answer must refer to the abstract specification. So rather than using the mapping procedure in a post-hoc manner after the computation has been run to see if it meets the specification, we must look at the intention behind the physical process. As the paper points out, this might seem that we are saying that the meaning of a language is actually in the head of its creator, but quoting Kripke, Turner sees this as problematic. (In the following passage quus is like mathematical plus, but just thinking about it as wrapping at some boundary like 32bit plus would.) Given . . . that everything in my mental history is compatible both with the conclusion that I meant plus and with the conclusion that I meant quus, it is clear that the sceptical challenge is not really an epistemological one. It purports to show that nothing in my mental history of past behavior—not even what an omniscient God would know—could establish whether I meant plus or quus. But then it appears to follow that there was no fact about me that constituted my having meant plus rather than quus. (Kripke 1982). So Turner concludes that the meanings of programming languages are not in the head, but rather are mathematical objects. By themselves, these semantics can be studied as just any other mathematical object, but when used by an agent they provide normative force over the implementation of a programming language, defining its correctness. I am always refreshed reading these kinds of papers. I understand many people find them exhausting. What difference can these esoteric distinctions possibly make for the actual practice of people? Well, luckily for you Turner has a suggestion for exactly that. Programming languages in order to be used for practical purposes, must have physical instantiations. Given this, computer science isn't merely an abstract discipline, it is something that produces technical artifacts. Computer science is part math, part technology. So shouldn't the philosophy of computer science be the combination of the two? You may take this as still decidedly impractical, who cares about the philosophy of computer science? For that, I have no retort. Are there no answers to the questions we seek in the philosophy of computer science? Or do the truths of certain questions simply not matter? Whichever it is, I can't understand why people wouldn't care about these questions.

Programming Python

▲ 0

0 views

Jimmy Miller 1 years ago

Dec 14: Bidrectional Type Checking

Type checking is one of those topics in programming that I feel is often taught in a way that makes it rather confusing. Hindley Milner is a rather elegant, but confusing algorithm. Seeing how easy it is to make in mini-Karen, certainly didn't give me any insight into how it works. When you dive into papers about type checking it's full of formalism and very little code. For academics, I'm sure that's fine. However, for an industry programmer like myself, I often find it rather difficult. This paper (Bidirectional Typing Rules: A Tutorial by David Raymond Christiansen) is a great contrast to this usual setup. It isn't lacking in formalism, but provides a nice discussion of the formalism and some simple translation to code! There's even a talk which is how I first discovered this paper (though there are some audio issues). The talk goes into a few more details. But this paper is still a great introduction. Bidirectional type checking is a method somewhere between full type inference and no type inference. An imprecise characterization of this is that you only have to write type annotations on type-level entities, but not on every variable. So, for function definitions you need to write your types, but not much else. The bidirectionality bit gets its name because this algorithm works by checking types and inferring (or synthesizing) types in an interleaving fashion. If we know a type and we have a value, we can check that value against a type. If we don't know a type but have a value, we can infer the type and then check it. This may seem a bit simplistic. When I first saw it, I thought there was no way this kind of setup could work but it does! Consider the very simple example: We can very easily infer the type of the literal and then we can check that the type of matches the annotation. This very simple idea scales up incredibly way to larger structures. Just from the features laid out in this paper, I was able to implement a simple bidirectional type checker! I actually did it twice. Once in a terse way in Clojure using a library called Meander a friend of mine wrote. (Maybe I need to find a good paper on term rewriting, not sure I ever found one of those). Then I decided to translate that into more verbose, but maybe a bit more familiar Javascript . This post was rather short as I didn't have a ton of time today. But I promise, this paper is super readable. It is full of code translations and hints on how to read things. For example: Translating the formalism from above into the code below makes this whole thing way more understandable! I know there are reasons academics prefer the notion they use for these things, but every time I see them translated into code, I just imagine an alternative universe where that didn't happen and how happy I'd be.

Tutorial Clojure JavaScript Programming

▲ 0

0 views

Jimmy Miller 1 years ago

Dec 13: What Knowledge Isn't

I read four different papers today. None of them stuck with me. There were a few papers on social aspects of programming, all of the empirical bent. I was hoping to find a good paper that captured the sense of frustration I feel with Agile; a paper that could summarize the perverse labor relationship that Agile has forged between software engineers, product managers, and the software itself. But I've tried in vain. I looked through my absurd back catalog. None of them stood out to me as papers I wanted to discuss today. So I'm instead giving you yet another philosophy paper. But this paper leaves you with no excuse but to read it. Coming in under 3 pages. This philosophy paper turned the world of epistemology on its head. Introducing Edmund L. Gettier's Is Justified True Belief Knowledge . These are all examples of things I think I know. But how can I be sure? What exactly does it mean for something to be knowledge? Well, when we look at the kinds of things we intuitively count as knowledge, there seems to be fairly wide agreement on the core of the account. First, the things you know, you must believe. Now of course we might say something like "I know I'm going to fail, but I don't want to believe it." But here we aren't being literal. When we say that we know something we are at least saying we believe it. Or to make it silly programmer speech, knowledge is a subclass of belief. But not all belief rises to the level of knowledge. What is missing? "I know that the earth is flat." "No, you don't, because it isn't." Replace the word "know" about with the word "believe". Note the asymmetry. With "know", this logic feels sound, but replacing it with believe makes the tort non-sensical. What does this show? That we can't know things that aren't true. Philosophers call this the "factive" portion of knowledge. In order for something to count as knowledge, it must actually be true. But it's not enough for something to be true for you to correctly assert that you know it. "It's going to rain tomorrow" "How do you know that?" "I rolled a d20 and got a natural 20" Unless the above sentence was uttered during some TTRPG, we would in fact reject this person's claim to knowledge. But what if it turns out it did in fact rain tomorrow? Did they know it all along? No, they just got lucky. What they lacked was any good reason for that belief. Philosophers call this "justification". Justification means that a person is internally in possession of a good reason for their belief. When asked why they believe something, they could offer a good reason. But does this intuitive view work? Gettier shows us that it doesn't. You should read the paper. So I'm not even going to give a Gettier example. Instead, I'll use a different example. The pyromaniac (Skyrms 1967). A pyromaniac reaches eagerly for his box of Sure-Fire matches. He has excellent evidence of the past reliability of such matches, as well as of the present conditions — the clear air and dry matches — being as they should be, if his aim of lighting one of the matches is to be satisfied. He thus has good justification for believing, of the particular match he proceeds to pluck from the box, that it will light. This is what occurs, too: the match does light. However, what the pyromaniac did not realize is that there were impurities in this specific match, and that it would not have lit if not for the sudden (and rare) jolt of Q-radiation it receives exactly when he is striking it. His belief is therefore true and well justified. But is it knowledge? Gettier shows us in the paper multiple cases where true justified belief fails to be knowledge. So what exactly is knowledge? Well, this is a whole area of debate in the philosophical community. But out of this has spawned fascinating, interesting work. There are debates about whether the criteria for knowledge requires something internal (i.e. mental) or external (the world, our brain chemistry, etc). There are debates about whether there is one unified conception of knowledge, or if knowledge is contextual. Finally, there are debates on whether knowledge has an analysis at all or if we ought to take knowledge as fundamental and define things like belief in terms of knowledge. The example given above and the examples given in the paper may seem far-fetched, but it is my contention that programming is a constant Gettier problem generator. How many times have you believed you had found the cause of some bug, and had good reason to believe it, it turns out you were correct, but then when looking closer at the bug realize all your reasons were completely wrong! You had true justified belief, but not knowledge. Philosophy (well, analytic philosophy, which is the kind I enjoy) is a lot like programming. It goes deep on topics most people don't care about. It introduces concepts that seem overly technical and complicated. There are endless debates and disagreements with no hope of reconciliation. But what I see from philosophy that I find so hard to find in these computer science papers, is an awareness of the complexity of our topics. A willingness to engage with those who disagree. No pretension that your approach is the best approach. No need to have a related work section, because instead, you directly engage with others' works. My name is Jimmy Agile is a terrible way to make software The earth is round My dog sheds a ton of hair No prime numbers are prime ministers

Philosophy Agile Science

▲ 0

0 views

Jimmy Miller 1 years ago

Dec 12: Lazy Evaluation of Transactions in Database Systems

This is a paper I have found utterly fascinating since I first read it many years ago. Is it a good idea? I have no clue. Does the data in the paper make me think it is a good idea? Probably not. But why should that be the bar? If there is anything this series has hopefully shown you, interesting papers don't need to be practical. The most valuable papers for me have been those that are completely impractical. Papers that make me think. That changes the way I view systems. That said, what I don't enjoy are papers that pretend to be practical, but ignore the realities of practice. I don't think this paper falls into that category. Here, they are investigating a rather interesting idea, what if instead of executing all the steps of a database transaction immediately, we just record a thunk of what should happen? How would the database respond? How exactly would lazy evaluation of a database transaction work? Well, for full details I'd suggest reading the paper, but for the general idea, we need a couple of things. First, our transactions must be deterministic. Second, we need the ability to decide given the current state of the database if a transaction will abort or commit. Finally, we need to be able to track the write set of a given transaction. With these things in place, we can have full ACID, yet lazy transactions. We can implement this lazy evaluation transaction by use of what they call "stickies". In essence, instead of actually executing our transaction, which may read from multiple fields and then write to multiple fields, we put a stickie note on the values we are going to write to. Once those sticky notes are in place, if someone reads those values, the sticky note tells them what transaction(s) to execute to get the current value. Now why would you want to do this? Because you can get all sorts of interesting load-handling benefits! In real-world applications, it is (in my experience) quite common to have writes that are never read. Or if they are read, they are read rarely at a much later date. But what is also common is to have rather spiky, inconsistent traffic. When traffic load increases, it can often be hard to keep up with the pace of transaction writes. Having lazy transactions allows us to defer the cost of those writes to later, perhaps off-peak times! This does come at the cost of potential read latency increases, because now reads may have to process transactions that were only lazily written. In these graphs from the paper, you can see a few things. First, the top graph shows you the traffic in this experimental setup. Transactions per second start low, peak, and drop back. The second graph shows you throughput. Things are a bit confusing here. The black line labeled "stickification" is the throughput from the user's perspective. These are the transactions that completed and were accepted by the system. The light blue "lazy" line gives you insight into how the lazy transactions are running behind the scenes. Finally, the third graph shows the success rate. As you can see, the eager process was not able to keep up with demand and dropped nearly 60% of transactions! While the lazy process did not drop a single one! I think this is actually a really interesting mechanism. Even if it never found its way into a general-purpose database system, (I can imagine people would find this behavior confusing), it is a fascinating idea that I think could be applied to many fit-for-purpose systems. Something talked about, but not explored in this paper is the idea of realizing these transactions in non-peak times. This will of course happen naturally as read slowly eek in. But in my experience, many real-world systems have times of significantly lower load. I can imagine adapting this process so that when these lower load moments occur, the system executes these lazy transactions. There are so many interesting ideas and opportunities laid out in this paper. I love the way in which it combines programming language ideas (lazy evaluation) and databases in an interesting way. I'd love to see more cross-over between these communities.

Database Performance

▲ 0

0 views

Jimmy Miller 1 years ago

Dec 11: On Understanding Data Abstraction Revisited

One of my meta-criteria for choosing a paper in this series is the likelihood we will cover the paper on the future of coding. I won't completely rule out covering papers I think we will do at some point. But for the most part, I've decided to steer clear of them. This paper is one I don't think we'd cover on a podcast because I'm not sure it makes for great conversation. This is one of the difficult things about the podcasting medium. Or at least the medium as I conceive it. Not all papers lend themselves to a free-wheeling discussion about the paper. A lecture might work for a paper, and a blog post almost certainly works, but some things just don't come out well in a discussion. One class of papers I don't think translate well are those that just tell you facts of the matter. I don't mean of course that there isn't bias in these papers. But simply that the goal of the paper isn't to convince you of anything but to simply tell you about things. In some ways, this paper doesn't quite qualify. It is in fact an argument, but not about something I'd consider a real live debate for me. Are objects and abstract data types different things? I'm not sure I really care about that distinction and yet this paper has stuck with me ever since I read it. So, I want to cast this paper in a different light for you readers who may, like me, not really care about objects vs adts. Rather than make a division between objects and adts, I want you to view this paper as bridging a divide between the values held by object-oriented programmers and those held by functional programmers. Object-oriented programmers value the ability to have multiple implementations of objects interoperate together. They want systems to be extended not modified. Functional programmers value immutability. They want to be certain that the values they have cannot change out from under them. Until I read this paper, I had believed the view that objects were about encapsulating state. In this paper, I see a very object-oriented way of programming, that constructs completely immutable objects. In keeping with my now adopted goal of not replacing the paper, but instead enticing you to read it. I will leave off what William R. Cook has to say about the difference between ADTs (Abstract not Algebraic) and Objects. It is actually quite interesting. He shows how the standard Java way of programming and has us build ADTs rather than Objects. But I won't explain further. Instead, I want to show from his example of what an interface for an object-oriented way of defining sets might look like. Here, in our fictional language, is an object-oriented interface for defining a set. We have not defined in any way a concrete implementation of a set. We have just talked about the operations a set must have. This is one of the keys, Cook tells us, to the object-oriented style. Most notably here, there isn't a means of constructing a set. These will come from objects that conform to this interface. Consider this implementation Here we have the empty set (In pseudo rust). It contains nothing. It mutates nothing. To insert things into the empty set, we just return a new set called Insert. To union, just pass whatever set we want to union with. What do Insert and Union look like? Here we see no inheritance, no encapsulating of state. Just completely immutable objects. But we also see object-oriented principles upheld. Most notably, these objects only know about themselves, they do not inspect the representation of the arguments they are passed except through their public interfaces. He calls this term Autognosis. Autognosis means ‘self-knowledge’. An autognostic object can only have detailed knowledge of itself. All other objects are abstract. The converse is quite useful: any programming model that allows inspection of the representation of more than one abstraction at a time is not object-oriented. But I must admit, these objects are not the ones that got me so excited about this idea. They are fairly straightforward, but when you embrace this style of behavioral abstraction you can make some much more interesting implementations. Here are sets that not only are completely immutable, but they also store a large quantity of things (infinitely many), without storing almost anything at all. I actually made my own little implementation of a language like this in Clojure and wrote quite a number of things in it. The paper explains all sorts of limitations with this particular interface for sets and the tradeoffs that come with object-oriented interfaces. How do you add the intersect operator to this interface? You can't. Why? Because there is no way to tell if two set intersections are empty without iterating over the sets. But how do you iterate over these infinitely large sets? I find all of these tradeoffs fascinating and wish this was the kind of stuff talked about with Object Oriented software (in before someone tells me that of course this stuff is talked about everywhere and I'm just not reading it). I'd highly recommend trying to write some software in this style. It's a fascinating set of constraints that make for some really interesting programs.

Rust Data Programming Clojure Java

▲ 0

0 views