Entropic Thoughts Yesterday

Hazardous States and Accidents

Programming Science

iDiallo 5 days ago

Designing Behavior with Music

A few years back, I had a ritual. I'd walk to the nearest Starbucks, get a coffee, and bury myself in work. I came so often that I knew all the baristas and their schedules. I also started noticing the music. There were songs I loved but never managed to catch the name of, always playing at the most inconvenient times for me to Shazam them. It felt random, but I began to wonder: Was this playlist really on shuffle? Or was there a method to the music? I never got a definitive answer from the baristas, but I started to observe a pattern. During the morning rush, around 8:30 AM when I'd desperately need to take a call, the music was always higher-tempo and noticeably louder. The kind of volume that made phone conversations nearly impossible. By mid-day, the vibe shifted to something more relaxed, almost lofi. The perfect backdrop for a deep, focused coding session when the cafe had thinned out and I could actually hear myself think. Then, after 5 PM, the "social hour" began. The music became familiar pop, at a volume that allowed for easy conversation, making the buzz of surrounding tables feel part of the atmosphere rather than a distraction. The songs changed daily, but the strategy was consistent. The music was subtly, or not so subtly, encouraging different behaviors at different times of day. It wasn't just background noise; it was a tool. And as it turns out, my coffee-fueled hypothesis was correct. This isn't just a Starbucks quirk; it's a science-backed strategy used across the hospitality industry. The music isn't random. It's designed to influence you. Research shows that we can broadly group cafe patrons into three archetypes, each responding differently to the sonic environment. Let's break them down. This is you and me, with a laptop, hoping to grind through a few hours of work. Our goal is focus, and the cafe's goal is often to prevent us from camping out all day on a single coffee. What the Research Says: A recent field experiment confirmed that fast-tempo music leads to patrons leaving more quickly. Those exposed to fast-tempo tracks spent significantly less time in the establishment than those who heard slow-tempo music or no music at all. For the solo worker, loud or complex music creates a higher "cognitive load," making sustained concentration difficult. That upbeat, intrusive morning music isn't an accident; it's a gentle nudge to keep the line moving. When you're trying to write code or draft an email and the music suddenly shifts to something with a driving beat and prominent vocals, your brain has to work harder to filter it out. Every decision, from what variable to name to which sentence structure to use, becomes just a little more taxing. I'm trying to write a function and a song is stuck in my head. "I just wanna use your love tonight!" After an hour or two of this cognitive friction, packing up and heading somewhere quieter starts to feel like a relief rather than an inconvenience. This pair is there for conversation. You meet up with a friend you haven't seen in some time. You want to catch up, and the music acts as a double-edged sword. What the Research Says: The key here is volume. Very loud music can shorten a visit because it makes conversing difficult. You have to lean in, raise your voice, and constantly ask "What?" Research on acoustic comfort in cafes highlights another side: music at a moderate level acts as a "sonic privacy blanket." It masks their conversation from neighboring tables better than silence, making the pair feel more comfortable and less self-conscious. I've experienced this myself. When catching up with a friend over coffee, there's an awkward awareness in a silent cafe that everyone can hear your conversation. Are you talking too loud about that work drama? Can the person at the next table hear you discussing your dating life? But add a layer of moderate background music, and suddenly you feel like you're in your own bubble. You can speak freely without constantly monitoring your volume or censoring yourself. The relaxed, mid-day tempo isn't just for solo workers. It's also giving pairs the acoustic privacy to linger over a second latte, perhaps order a pastry, and feel comfortable enough to stay for another thirty minutes. The group of three or more is there for the vibe. Their primary goal is to connect with each other, and the music is part of the experience. What the Research Says: Studies on background music and consumer behavior show that for social groups, louder, more upbeat music increases physiological arousal, which translates into a sense of excitement and fun. This positive state is directly linked to impulse purchases, and a longer stay. "Let's get another round!" The music effectively masks the group's own noise, allowing them to be loud without feeling disruptive. The familiar pop tunes of the evening are an invitation to relax, stay, and spend. That energy translates into staying longer, ordering another drink, maybe splitting some appetizers. The music gives permission for the group to match its volume and enthusiasm. If the cafe is already vibrating with sound, your group's laughter doesn't feel excessive, it feels appropriate. The music is not random, it's calculated. I have a private office in a coworking space. What I find interesting is that whenever I go to the common area, where most people work, there's always music blasting. Not just playing. Blasting . You couldn't possibly get on a meeting call in the common area, even though this is basically a place of work. For that, there are private rooms that you can rent by the minute. Let that sink in for a moment. In a place of work, it's hard to justify music playing in the background loud enough to disrupt actual work. Unless it serves a very specific purpose: getting you to rent a private room. The economics makes sense. I did a quick count on my floor. The common area has thirty desks but only eight private rooms. If everyone could take calls at their desks, those private rooms would sit empty. But crank up the music to 75 decibels, throw in some upbeat electronic tracks with prominent basslines, and suddenly those private rooms are booked solid at $5 per 15 minutes. That's $20 per hour, per room, eight rooms, potentially running 10 hours a day. The music isn't there to help people focus. It's a $1,600 daily revenue stream disguised as ambiance. And the best, or worse, part is that nobody complains. Because nobody wants to be the person who admits they need silence to think. We've all internalized the idea that professionals should be able to work anywhere, under any conditions. So we grimace, throw on noise-canceling headphones, and when we inevitably need to take a Zoom call, we sheepishly book a room and swipe our credit card. Until now, this process has been relatively manual. A manager chooses a playlist or subscribes to a service (like Spotify's "Coffee House" or "Lofi Beats") and hopes it has the desired effect. It's a best guess based on time of day and general principles. But what if a cafe could move from curating playlists to engineering soundscapes in real-time? This is where generative AI will play a part. Imagine a system where: Simple sensors can count the number of customers in the establishment and feed real-time information to an AI. Point-of-sale data shows the average ticket per customer and table turnover rates. The AI receives a constant stream: "It's 2:30 PM. The cafe is 40% full, primarily with solo workers on laptops. Table turnover is slowing down, average stay time is now 97 minutes, up from the target of 75 minutes." An AI composer, trained on psychoacoustic principles and the cafe's own historical data, generates a unique, endless piece of music. It doesn't select from a library. It is created in realtime. The manager has set a goal: "Gently increase turnover without driving people away." The AI responds by subtly shifting the generated music to a slightly faster BPM. Maybe, from 98 to 112 beats per minute. It introduces more repetitive, less engrossing melodies. Nothing jarring, nothing that would make someone consciously think "this music is annoying," but enough to make that coding session feel just a little more effortful. The feedback loop measures the result. Did the solo workers start packing up 15 minutes sooner on average? Did they look annoyed when they left, or did they seem natural? Did anyone complain to staff? The AI learns and refines its model for next time, adjusting its parameters. Maybe 112 BPM was too aggressive; next time it tries 106 BPM with slightly less complex instrumentation. This isn't science fiction. The technology exists today. We already have: Any day now, you'll see a start up providing this service. Where the ambiance of a space is not just curated, but designed. A cafe could have a "High Turnover Morning" mode, a "Linger-Friendly Afternoon" mode, and a "High-Spend Social Evening" mode, with the AI seamlessly transitioning between them by generating the perfect, adaptive soundtrack. One thing that I find frustrating with AI is that when we switch to these types of systems, you never know. The music would always feel appropriate, never obviously manipulative. It would be perfectly calibrated to nudge you in the desired direction while remaining just below the threshold of conscious awareness. A sonic environment optimized not for your experience, but for the business's metrics. When does ambiance become manipulation? There's a difference between playing pleasant background music and deploying an AI system that continuously analyzes your behavior and adjusts the environment to influence your decisions. One is hospitality; the other is something closer to behavioral engineering. And unlike targeted ads online, which we're at least somewhat aware of and can block, this kind of environmental manipulation is invisible, unavoidable, and operates on a subconscious level. You can't install an ad blocker for the physical world. I don't have answers here, only questions. Should businesses be required to disclose when they're using AI to manipulate ambiance? Is there a meaningful difference between a human selecting a playlist to achieve certain outcomes and an AI doing the same thing more effectively? Does it matter if the result is that you leave a cafe five minutes sooner than you otherwise would have? These are conversations we need to have as consumers, as business owners, as a society. Now we know that the quiet background music in your local cafe has never been just music. It's a powerful, invisible architect of behavior. And it's about to get a whole lot smarter. Simple sensors can count the number of customers in the establishment and feed real-time information to an AI. Point-of-sale data shows the average ticket per customer and table turnover rates. The AI receives a constant stream: "It's 2:30 PM. The cafe is 40% full, primarily with solo workers on laptops. Table turnover is slowing down, average stay time is now 97 minutes, up from the target of 75 minutes." An AI composer, trained on psychoacoustic principles and the cafe's own historical data, generates a unique, endless piece of music. It doesn't select from a library. It is created in realtime. The manager has set a goal: "Gently increase turnover without driving people away." The AI responds by subtly shifting the generated music to a slightly faster BPM. Maybe, from 98 to 112 beats per minute. It introduces more repetitive, less engrossing melodies. Nothing jarring, nothing that would make someone consciously think "this music is annoying," but enough to make that coding session feel just a little more effortful. The feedback loop measures the result. Did the solo workers start packing up 15 minutes sooner on average? Did they look annoyed when they left, or did they seem natural? Did anyone complain to staff? The AI learns and refines its model for next time, adjusting its parameters. Maybe 112 BPM was too aggressive; next time it tries 106 BPM with slightly less complex instrumentation. Generative AI that can create music in any style ( MusicLM , MusicGen ) Computer vision that can anonymously track occupancy and behavior Point-of-sale systems that track every metric in real-time Machine learning systems that can optimize for complex, multi-variable outcomes

Science Business AI

Jeff Geerling 6 days ago

How much radiation can a Pi handle in space?

Late in the cycle while researching CubeSats using Pis in space , I got in touch with Ian Charnas 1 , the chief engineer for the Mark Rober YouTube channel. Earlier this year, Crunchlabs launched SatGus , which is currently orbiting Earth taking 'space selfies'.

Programming Science Hardware

A Working Library 1 weeks ago

Beyond credibility

In the 1880s, a French neurologist named Jean-Martin Charcot became famous for hosting theatrical public lectures in which he put young, “hysterical” women in a hypnotic trance and then narrated the symptoms of the attacks that followed. Charcot’s focus was on documenting and classifying these symptoms, but he had few theories as to their source. A group of Charcot’s followers—among them Pierre Janet, Joseph Breuer, and Sigmund Freud—would soon eagerly compete to be the first to discover the cause of this mysterious affliction. Where Charcot showed intense interest in the expression of hysteria, he had no curiosity for women’s own testimony; he dismissed their speech as “vocalizations.” But Freud and his compatriots landed on the novel idea of talking to the women in question. What followed were years in which they talked to many women regularly, sometimes for hours a day, in what can only be termed a collaboration between themselves and their patients. That collaboration revealed that hysteria was a condition brought about by trauma. In 1896, Freud published The Aetiology of Hysteria, asserting: I therefore put forward the thesis that at the bottom of every case of hysteria there are one or more occurrences of premature sexual experiences , occurrences which belong to the earliest years of childhood, but which can be reproduced through the work of psycho-analysis in spite of the intervening decades. I believe that this is an important finding, the discovery of a caput Nili in neuropathology. Judith Herman, in Trauma and Recovery , notes that The Aetiology remains one of the great texts on trauma; she describes Freud’s writing as rigorous and empathetic, his analysis largely in accord with present-day thinking about how sexual abuse begets trauma and post-traumatic symptoms, and with methods that effect treatment. But a curious thing happened once this paper was published: Freud began to furiously backpedal from his claims. [Freud’s] correspondence makes clear that he was increasingly troubled by the radical social implications of his hypothesis. Hysteria was so common among women that if his patients’ stories were true, and if his theory were correct, he would be forced to conclude that what he called “perverted acts against children” were endemic, not only among the proletariat of Paris, where he had first studied hysteria, but also among the respectable bourgeois families of Vienna, where he had established his practice. This idea was simply unacceptable. It was beyond credibility. Faced with this dilemma, Freud stopped listening to his female patients. The turning point is documented in the famous case of Dora. This, the last of Freud’s case studies on hysteria, reads more like a battle of wits than a cooperative venture. The interaction between Freud and Dora has been described as an “emotional combat.” In this case Freud still acknowledged the reality of his patient’s experience: the adolescent Dora was being used as a pawn in her father’s elaborate sex intrigues. Her father had essentially offered her to his friends as a sexual toy. Freud refused, however, to validate Dora’s feelings of outrage and humiliation. Instead, he insisted upon exploring her feelings of erotic excitement, as if the exploitative situation were a fulfillment of her desire. In an act Freud viewed as revenge, Dora broke off the treatment. That is, faced with the horror of women’s experience, Freud rejected the evidence in front of him. Rather than believe the women he had collaborated with, and so be forced to revise his image of the respectable men in his midst, he chose to maintain that respectability by refusing the validity of his own observations. He would go on to develop theories of human psychology that presumed women’s inferiority and deceitfulness—in a way, projecting his own lies onto his patients. Is this not how all supremacy thinking works? To believe that one people are less human or less intelligent or less capable is to refuse to see what’s right in front of you, over and over and over again. In order to recant his own research, Freud had to cleave his mind in two. We must refuse to tolerate supremacists in our midst because their beliefs do real and lasting harm, because their speech gives rise to terrible violence. But we must also refuse them because they are compromised. They cannot trust their own minds. And so cannot be trusted in turn. View this post on the web , subscribe to the newsletter , or reply via email .

Science Books

A Working Library 2 weeks ago

Hurry-up-quick!

I’ve written before about the Army intelligence tests: an experiment in which millions of Army recruits were subject to an early version of the IQ test. As Stephen Jay Gould documents , the tests were chaotically—almost deliriously—managed. Illiterate recruits were given a version of the test in which proctors walked around yelling inscrutable instructions and pointing at pictures on sheets of paper; many of these recruits did not speak English as their first language, and had never before used a pencil. Gould shares some of the instructions given to the proctors: The idea of working fast must be impressed upon the men during the maze test. Examiner and orderlies walk around the room, motioning men who are not working, and saying, “Do it, do it, hurry up, quick.” At the end of 2 minutes, examiner says, “Stop! Turn over the page to test 2.” This is, as Gould notes, diabolical. How could a test given under these conditions possibly evaluate some innate quality of “intelligence”? But the designers of the test were so enamored of their theories of racial hierarchy that they either couldn’t perceive the irrationality of their own design, or else they knew it for a facade. The practice of the eugenicist is invariably that of the error or the con. But that phrase, hurry up, quick, struck a bell—I had heard it before. In Le Guin’s The Word for World Is Forest, human colonizers arrive on the planet Athshea, seven lightyears from Earth and rich in trees—a rarity on their deforested home world. The Athshean people are small, furred, and green; the humans name them “creechies,” deem them to be of lesser intelligence (an error, as it turns out), and proceed to enslave them, rape them, and kill them with impunity. In the opening pages, we see the Captain of New Tahiti Colony rise in the morning, and yell to an Athshean: “Ben!” he roared, sitting up and swinging his bare feet onto the bare floor. “Hot water get-ready, hurry-up-quick!” Le Guin’s concatenation of the phrase transforms it from merely extreme into something sinister: the way the words roll out all together escalates the inane redundancy, the empty urgency. Speed is not useful to the task at hand; the hurried pot does not boil faster. Rather, the purpose of the haste is to prevent any semblance of rest, to prohibit even a moment of peace. But rest is reserved for those deemed sufficiently wise, and sufficiently human. The Captain will eventually learn that Ben’s ingenuity far exceeds his own—a lesson that comes at a very steep price for them both. Whether our present-day and present-Earth supremacists will ever learn remains to be seen. View this post on the web , subscribe to the newsletter , or reply via email .

Culture Science

A Working Library 2 months ago

Ammonite

Marguerite (“Marghe”) Taishan is about to step foot on the planet Jeep when she receives a warning: if she goes on, she will never come back. But she’s come too far, and worked too hard, and Jeep is too interesting for her to turn back now: across its continents lives a scattered human colony, forgotten for centuries, but apparently thriving. Which might be unremarkable except for the fact that all the people are women. Marghe’s job is to investigate how they have survived, and to test a vaccine against the virus that killed the men. But her own survival, and the planet’s, are more precarious, and more intertwined, than she predicts. Nicola Griffith’s first novel is about making a home, and remembering the past, and the impossible beauty and danger of knowing women are human. View this post on the web , subscribe to the newsletter , or reply via email .

Books Science

DYNOMIGHT 3 months ago

New colors without shooting lasers into your eyes

Your eyes sense color. They do this because you have three different kinds of cone cells on your retinas, which are sensitive to different wavelengths of light . For whatever reason, evolution decided those wavelengths should be overlapping . For example, M cones are most sensitive to 535 nm light, while L cones are most sensitive to 560 nm light. But M cones are still stimulated quite a lot by 560 nm light—around 80% of maximum. This means you never (normally) get to experience having just one type of cone firing. So what do you do? If you’re a quitter, I guess you accept the limits of biology. But if you like fun , then what you do is image people’s retinas, classify individual cones, and then selectively stimulate them using laser pulses, so you aren’t limited by stupid cone cells and their stupid blurry responsivity spectra. Fong et al. (2025) choose fun. When they stimulated only M cells… Subjects report that [pure M-cell activation] appears blue-green of unprecedented saturation. If you make people see brand-new colors, you will have my full attention. It doesn’t hurt to use lasers. I will read every report from every subject. Do our brains even know how to interpret these signals, given that they can never occur? But tragically, the paper doesn’t give any subject reports. Even though most of the subjects were, umm, authors on the paper. If you want to know what this new color is like, the above quote is all you get for now. Or… possibly you can see that color right now? If you click on the above image, a little animation will open. Please do that now and stare at the tiny white dot. Weird stuff will happen, but stay focused on the dot. Blink if you must. It takes one minute and it’s probably best to experience it without extra information i.e. without reading past this sentence. The idea for that animation is not new. It’s plagiarized based on Skytopia’s Eclipse of Titan optical illusion (h/t Steve Alexander ), which dates back to at least 2010. Later I’ll show you some variants with other colors and give you a tool to make your own. If you refused to look at the animation, it’s just a bluish-green background with a red circle on top that slowly shrinks down to nothing. That’s all. But as it shrinks, you should hallucinate a very intense blue-green color around the rim. Why do you hallucinate that crazy color? I think the red circle saturates the hell out of your red-sensitive L cones. Ordinarily, the green frequencies in the background would stimulate both your green-sensitive M cones and your red-sensitive L cones, due to their overlapping spectra . But the red circle has desensitized your red cones, so you get to experience your M cones firing without your L cones firing as much, and voilà—insane color. So here’s my question: Can that type of optical illusion show you all the same colors you could see by shooting lasers into your eyes? That turns out to be a tricky question. See, here’s a triangle: Think of this triangle as representing all the “colors” you could conceivably experience. The lower-left corner represents only having your S cones firing, the top corner represents only your M cones firing, and so on. So what happens if you look different wavelengths of light? Short wavelengths near 400 nm mostly just stimulate the S cones, but also stimulate the others a little. Longer wavelengths stimulate the M cones more, but also stimulate the L cones, because the M and L cones have overlapping spectra . (That figure, and the following, are modified from Fong et al. ) When you mix different wavelengths of light, you mix the cell activations. So all the colors you can normally experience fall inside this shape: That’s the standard human color gamut , in LMS colorspace . Note that the exact shape of this gamut is subject to debate. For one thing, the exact sensitivity of cells is hard to measure and still a subject of research. Also, it’s not clear how far that gamut should reach into the lower-left and lower-right corners, since wavelengths outside 400-700 nm still stimulate cells a tiny bit. And it gets worse. Most of the technology we use to represent and display images electronically is based on standard RGB (sRGB) colorspace . This colorspace, by definition , cannot represent the full human color gamut. The precise definition of sRGB colorspace is quite involved. But very roughly speaking, when an sRGB image is “pure blue”, your screen is supposed to show you a color that looks like 450-470 nm light, while “pure green” should look like 520-530 nm light, and “pure red” should look like 610-630 nm light. So when your screen mixes these together, you can only see colors inside this triangle. (The corners of this triangle don’t quite touch the boundaries of the human color gamut. That’s because it’s very difficult to produce single wavelengths of light without using lasers. In reality, the sRGB specification say that pure red/blue/green should produce a mixture of colors centered around the wavelengths I listed above.) What’s the point of all this theorizing? Simple: When you look at the optical illusions on a modern screen, you aren’t just fighting the overlapping spectra of your cones. You’re also fighting the fact that the screen you’re looking at can’t produce single wavelengths of light. So do the illusions actually take you outside the natural human color gamut? Unfortunately, I’m not sure. I can’t find much quantitative information about how much your cones are saturated when you stare at red circles. My best guess is no, or perhaps just a little. If you’d like to explore these types of illusions further, I made a page in which you can pick any colors. You can also change the size of the circle, the countdown time, if the circle should shrink or grow, and how fast it does that. You can try it here . You can export the animation to an animated SVG, which will be less than 1 kb. Or you can just save the URL. Some favorites: Red inside, reddish-orange outside Red inside, green outside Green inside, purple outside If you’re colorblind, I don’t think these will work, though I’m not sure. Folks with deuteranomaly have M cones, but they’re shifted to respond more like L cones. In principle, these types of illusions might help selectively activate them, but I have no idea if that will lead to stronger color perception. I’d love to hear from you if you try it. Red inside, reddish-orange outside Red inside, green outside Green inside, purple outside

AI Entertainment Science

DYNOMIGHT 3 months ago

Links for July

(1) Rotating eyeballs Goats, like most hoofed mammals, have horizontal pupils. When a goat’s head tilts up (to look around) and down (to munch on grass), an amazing thing happens. The eyeballs actually rotate clockwise or counterclockwise within the eye socket. This keeps the pupils oriented to the horizontal. To test out this theory, I took photos of Lucky the goat’s head in two different positions, down and up. (2) Novel color via stimulation of individual photoreceptors at population scale (h/t Benny) The cones in our eyes all have overlapping spectra . So even if you look at just a single frequency of light, more than one type of cone will be stimulated. So, obviously, what we need to do is identify individual cone cell types on people’s retinas and then selectively stimulate them with lasers so that people can experience never-before-seen colors. Attempting to activate M cones exclusively is shown to elicit a color beyond the natural human gamut, formally measured with color matching by human subjects. They describe the color as blue-green of unprecedented saturation. When I was a kid and I was bored in class, I would sometimes close my eyes and try to think of a “new color”. I never succeeded, and in retrospect I think I have aphantasia. But does this experiment suggest it is actually possible to imagine new colors? I’m fascinated that our brains have the ability to interpret these non-ecological signals, and applaud all such explorations of qualia space. (3) Simplifying Melanopsin Metrology (h/t Chris & Alex ) When reading about blue-blocking glasses , I failed to discover that the effects of light on melatonin don’t seem to be mediated by cones or rods at all. Instead, around 1% of retinal photosensitive cells are melanopsin-containing retinal ganglion cells . These seem to specifically exist for the purpose of regulating melatonin and circadian rhythms. They have their own spectral sensitivity : If you believe that sleep is mediated entirely by these cells, then you’d probably want to block all frequencies above ~550 nm. That would leave you with basically only orange and red light. However, Chris convinced me that if you want natural melatonin at night, the smart thing is primarily rely on dim lighting, and only secondarily on blocking blue light. Standard “warm” 2700 K bulbs only reduce blue light to around ⅓ as much. But your eyes can easily adapt to <10% as many lux. If you combine those, blue light is down by ~97%. The brain doesn’t seem to use these cells for pattern vision at all. Although… In work by Zaidi, Lockley and co-authors using a rodless, coneless human, it was found that a very intense 481 nm stimulus led to some conscious light perception, meaning that some rudimentary vision was realized. (4) Inflight Auctions Airplanes have to guess how much food to bring. So either they waste energy moving around extra food that no one eats, or some people go hungry. So why don’t we have people bid on food, so nothing goes to waste? I expect passengers would absolutely hate it. (5) The Good Sides Of Nepotism Speaking of things people hate, this post gives a theory for why you might rationally prefer to apply nepotism when hiring someone: Your social connections increase the cost of failure for the person you hire. I suspect we instinctively apply this kind of game theory without even realizing we’re doing so. This seems increasingly important, what with all the AI-generated job applications now attacking AI-automated human resources departments. My question is: If this theory is correct, can we create other social structures to provide the same benefit in other ways, therefore reducing the returns on nepotism? Say I want you to hire me, but you’re worried I suck. In principle, I could take $50,000, put it in escrow, and tell you, “If you hire me, and I actually suck (as judged by an arbiter) then you can burn the $50,000.” Sounds horrible, right? But that’s approximately what’s happening if you know I have social connections and/or reputation that will be damaged if I screw up. (6) Text fragment links We’ve spent decades in the dark ages of the internet, where you could only link to entire webpages or (maybe) particular hidden beacon codes. But we are now in a new age. You can link to any text on any page. Like this: This is not a special feature of . It’s done by your browser. I love this, but I can never remember how to type . Well, finally , almost all browsers now also support generating these links. You just highlight some text, right-click, and “Copy Link to Highlight”. If you go to this page and highlight and right-click on this text: Then you get this link . (7) (Not technically a link) Also, did you know you can link to specific pages of pdf files? For example: I just add manually. Chrome-esque browsers, oddly, will do automatically if you right-click and go to “Create QR Code for this Page”. (8) Response to Dynomight on Scribble-based Forecasting Thoughtful counter to some of my math skepticism. I particularly endorse the point in the final paragraph. (9) Decision Conditional Prices Reflect Causal Chances Robin Hanson counters my post on Futarchy’s fundamental flaw . My candid opinion is that this is a paradigmatic example of a “heat mirage” , in that he doesn’t engage with any of the details of my argument, doesn’t specify what errors I supposedly made, and doesn’t seem to commit to any specific assumptions that he’s willing to argue are plausible and would guarantee prices that reflect causal effects. So I don’t really see any way to continue the conversation. But judge for yourself! (10) Futarchy’s fundamental flaw - the market Speaking of which, Bolton Bailey set up a conditional prediction market to experimentally test one of the examples I gave where I claimed prediction markets would not reflect causal probabilities. If you think betting on causal effects is always the right strategy in conditional prediction markets, here’s your chance to make some fake internet currency. The market closes on July 26, 2025. No matter how much you love me, please trade according to your self-interest. (11) War and Peace I’m reading War and Peace. You probably haven’t heard, but it’s really good. Except the names. Good god, the names. There are a lot of characters, and all the major ones have many names: Those are all the same person. Try keeping track of all those variants for 100 different characters in a narrative with many threads spanning time and space. Sometimes, the same name refers to different people. And Tolstoy loves to just write “The Princess” when there are three different princesses in the room. So I thought, why not use color? Whenever a new character appears, assign them a color, and use it for all name variants for the rest of the text. Even better would be to use color patterns like Bol kón ski / Prince Andréy Nikoláevich . This should be easy for AI, right? I can think of ways to do this, but they would all be painful, due to War and Peace’s length: They involve splitting the text into chunks, having the AI iterate over them while updating some name/color mapping, and then merging everything at the end. So here’s a challenge: Do you know an easy way to do this? Is there any existing tool that you can give a short description of my goals, and get a full name-colored pdf / html / epub file? (“If your agent cannot do this, then of what use is the agent?”) Note: It’s critical to give all characters a color. Otherwise, seeing a name without color would be a huge spoiler that they aren’t going to survive very long. It’s OK if some colors are similar. There’s also the issue of all the intermingled French. But I find that hard not to admire—Tolstoy was not falling for audience capture. (And yes, War and Peace, Simplified Names Edition apparently exists. But I’m in too deep to switch now.) (12) Twins The human twin birth rate in the United States rose 76% from 1980 through 2009, from 9.4 to 16.7 twin sets (18.8 to 33.3 twins) per 1,000 births. The Yoruba people have the highest rate of twinning in the world, at 45–50 twin sets (90–100 twins) per 1,000 live births possibly because of high consumption of a specific type of yam containing a natural phytoestrogen which may stimulate the ovaries to release an egg from each side. I love this because, like: (That actually happened. Yams had that conversation and then started making phytoestrogens.) Apparently, some yams naturally contain the plant hormone diosgenin , which can be chemically converted into various human hormones. And that’s actually how we used to make estrogen, testosterone, etc. And if you like that, did you know that estrogen medications were historically made from the urine of pregnant mares ? I thought this was awesome, but after reading a bit about how this worked, I doubt the horses would agree. Even earlier, animal ovaries and testes were used. These days, hormones tend to be synthesized without any animal or plant precursor. If you’re skeptical that more twins would mean higher reproductive fitness, note that yams don’t believe in Algernon Arguments. type into the address bar click the “⇌” symbol so that “false” changes to “true”. Nikoláevich Prince Andréy Prince Bolkónski Prince Andréy Nikoláevich

Science HTML

A Working Library 3 months ago

Autonomy

After the 2008 financial crisis, austerity measures sprouted up across Europe. In Greece, public hospital budgets were cut by twenty-five percent, and nearly a million people were left with no access to healthcare. In subsequent years, dozens of social solidarity health clinics sprung up, many squatting in abandoned warehouses and industrial spaces. The solidarity clinics could not reproduce the institutionalized medical system that was in neglect, nor did they want to. In For Health Autonomy, Cassie Thornton visits many of these clinics and talks to the doctors, nurses, and others running them. Patients are referred to as “incomers,” in order to avoid the association, in Greek, of “patient” with “weakness”; and as part of a broader program to eliminate the hierarchy between doctor and patient and create a space for collective healing. Thornton writes: In this model, the new incomer meets with three health care practitioners at the same time on their first ninety-minute visit: a general physician, a psychotherapist, and a social worker. There is a health card that the social worker or third member fills out, asking questions to which the patient can answer or not. The health card covers the mental, emotional, and physical health, but is also used to note the conditions of the family, home, work, food, sleep patterns, and family relations; all are considered important aspects of health. As Frosso put it, they are trying to make a hologram of every person: a three-dimensional image of health as clear as possible to the members of the Workers’ Health Center as well as to the incomer. The answers provided by the incomer are used to fill out a health card, a new kind of record that the clinic is developing. It includes a family tree, which also addresses relationship quality and matters of heredity. At a follow-up meeting, the practitioners and the incomer reflect on all the physical, social, and psychological information. Here, the group helps the incomer take steps towards treatment or cure and helps them develop a plan to do so in a way where they can manage and get support for their own healing. What does the person need in order to thrive? What considerations do they need to make in terms of family, work, and money to help them feel healthy? And what I imagine to be a future question—what social movements might they join to experience the healing capacities of solidarity? There’s a lot I could say about this, but one thing I want to call out now is how this image of autonomy works . This isn’t a vision of autonomy as isolated or even “independent,” in the way that word often gets used these days—meaning, without succor or support. It depends, instead, on a robust system of care, in which someone arrives in a place where they are attended to, listened to, respected. It’s an autonomy that’s as integrated into society as the healthcare it enables is integrated across mind, body, and spirit. This is autonomy in the context if collective care, rather than the isolation of self -care. The second thing that strikes me about this model, on today of all days, is how these clinics weren’t merely a substitute for a failing system, but a wholly different structure and ethos. In the ruins of neglected institutions, the solidarity clinics chose to build something new, something that would not reproduce the old inequities but would transform healthcare into an experience oriented towards the fullness of human living and thriving. They not only considered the incomer in the context of their whole world, they also included the experience of the doctors, nurses, and other volunteers in their conceptualization of liberation. That is, they saw the austerity crisis as an opportunity to break from a system that was marked by scarcity, violence, and separation long before the financial meltdown brought it to a head. Perhaps there is a lesson for us here, too. View this post on the web , subscribe to the newsletter , or reply via email .

Science Programming Culture

DYNOMIGHT 3 months ago

Do blue-blocking glasses improve sleep?

Back in 2017, everyone went crazy about these things: The theory was that perhaps the pineal gland isn’t the principal seat of the soul after all. Maybe what it does is spit out melatonin to make you sleepy. But it only does that when it’s dark, and you spend your nights in artificial lighting and/or staring at your favorite glowing rectangles. You could sit in darkness for three hours before bed, but that would be boring. But—supposedly—the pineal gland is only shut down by blue light. So if you selectively block the blue light, maybe you can sleep well and also participate in modernity. Then, by around 2019, blue-blocking glasses seemed to disappear. And during that brief moment in the sun, I never got a clear picture of if they actually work. So, do they? To find out, I read all the papers. Before getting to the papers, please humor me while I give three excessively-detailed reminders about how light works. First, it comes in different wavelengths . Outside the visible spectrum, infrared light and microwaves and radio waves have even longer wavelengths, while ultraviolet light and x-rays and gamma rays have even shorter wavelengths. Shorter wavelengths have more energy. Do not play around with gamma rays. Other colors are hallucinations made up by your brain. When you get a mixture of all wavelengths, you see “white”. When you get a lot of yellow-red wavelengths, some green, and a little violet-blue, you see “brown”. Similar things are true for pink/purple/beige/olive/etc. (Technically, the original spectral colors and everything else you experience are also hallucinations made up by your brain, but never mind.) Second, the ruleset of our universe says that all matter gives off light, with a mixture of wavelengths that depends on the temperature. Hotter stuff has atoms that are jostling around faster, so it gives off more total light, and shifts towards shorter (higher-energy) wavelengths. Colder stuff gives off less total light and shifts towards longer wavelengths. The “color temperature” of a lightbulb is the temperature some chunk of rock would have to be to produce the same visible spectrum. Here’s a figure , with the x-axis in kelvins. The sun is around 5800 K. That’s both the physical temperature on the surface and the color temperature of its light. Annoyingly, the orange light that comes from cooler matter is often called “warm”, while the blueish light that comes from hotter matter is called “cool”. Don’t blame me. Anyway, different light sources produce widely different spectra . You can’t sense most of those differences because you only have three types of cone cells . Rated color temperatures just reflect how much those cells are stimulated. Your eyes probably see the frequencies they do because that’s where the sun’s spectrum is concentrated. In dim light, cones are inactive, so you rely on rod cells instead. You’ve only got one kind of rod, which is why you can’t see color in dim light. (Though you might not have noticed.) Finally, amounts of light are typically measured in lux . Your eyes are amazing and can deal with upwards of 10 orders of magnitude . In summary, you get widely varying amounts of different wavelengths of light in different situations, and the sun is very powerful. It’s reasonable to imagine your body might regulate its sleep schedule based that input. OK, but do blue-blocking glasses actually work? Let’s read some papers. Kayumov et al. (2005) had 19 young healthy adults stay awake overnight for three nights, first with dim light (<5 lux) and then with bright light (800 lux), both with and without blue-blocking goggles. They measured melatonin in saliva each hour. The goggles seemed to help a lot. With bright light, subjects only had around 25% as much melatonin as with dim light. Blue-blocking goggles restored that to around 85%. I rate this as good evidence for a strong increase in melatonin. Sometimes good science is pretty simple. Burkhart and Phelps (2009) first had 20 adults rate their sleep quality at home for a week as a baseline. Then, they were randomly given either blue-blocking glasses or yellow-tinted “placebo” glasses and told to wear them for 3 hours before sleep for two weeks. Oddly, the group with blue-blocking glasses had much lower sleep quality during the baseline week, but this improved a lot over time. I rate this as decent evidence for a strong improvement in sleep quality. I’d also like to thank the authors for writing this paper in something resembling normal human English. Van der Lely et al. (2014) had 13 teenage boys wear either blue-blocking glasses or clear glasses from 6pm to bedtime for one week, followed by the other glasses for a second week. Then they went to a lab, spent 2 hours in dim light, 30 minutes in darkness, and then 3 hours in front of an LED computer, all while wearing the glasses from the second week. Then they were asked to sleep, and their sleep quality was measured in various ways. The boys had more melatonin and reported feeling sleepier with the blue-blocking glasses. I rate this as decent evidence for a moderate increase in melatonin, and weak evidence for near-zero effect on sleep quality. Gabel et al. (2017) took 38 adults and first put them through 40 hours of sleep deprivation under white light, then allowed them to sleep for 8 hours. Then they were subjected to 40 more hours of sleep deprivation under either white light (250 lux at 2800K), blue light (250 lux at 9000K), or very dim light (8 lux, color temperature unknown). Their results are weird. In younger people, dim light led to more melatonin that white light, which led to more melatonin that blue light. That carried over to a tiny difference in sleepiness. But in older people, both those effects disappeared, and blue light even seemed to cause more sleepiness than white light. The cortisol and wrist activity measurements basically make no sense at all. I rate this as decent evidence for a moderate effect on melatonin, and very weak evidence for a near-zero effect on sleep quality. (I think its decent evidence for a near-zero effect on sleepiness, but they didn’t actually measure sleep quality.) Esaki et al. (2017) gathered 20 depressed patients with insomnia. They first recorded their sleep quality for a week as a baseline, then were given either blue-blocking glasses or placebo glasses and told to wear them for another week starting at 8pm. The changes in the blue-blocking group were a bit better for some measures, but a bit worse for others. Nothing was close to significant. Apparently 40% of patients complained that the glasses were painful, so I wonder if they all wore them as instructed. I rate this was weak evidence for near-zero effect on sleep quality. Shechter et al. (2018) gave 14 adults with insomnia either blue-blocking or clear glasses and had them wear them for 2 hours before bedtime for one week. Then they waited four weeks and had them wear the other glasses for a second week. They measured sleep quality through diaries and wrist monitors. The blue-blocking glasses seemed to help with everything. People fell asleep 5 to 12 minutes faster, and slept 30 to 50 minutes longer, depending on how you measure. (SOL is sleep onset latency, TST is total sleep time). I rate this as good evidence for a strong improvement in sleep quality. Knufinke et al. (2019) had 15 young adult athletes either wear blue-blocking glasses or transparent glasses for four nights. The blue-blocking group did a little better on most measures (longer sleep time, higher sleep quality) but nothing was statistically significant. I rate this as weak evidence for a small improvement in sleep quality. Janků et al. (2019) took 30 patients with insomnia and had them all go to therapy. They randomly gave them either blue-blocking glasses or placebo glasses and asked the patients to wear them for 90 minutes before bed. The results are pretty tangled. According to sleep diaries, total sleep time went up by 37 minutes in the blue-blocking group, but slightly decreased in the placebo group. The wrist monitors show total sleep time decreasing in both groups, but it did decrease less with the blue-blocking glasses. There’s no obvious improvement in sleep onset latency or the various questionnaires they used to measure insomnia. I rate this as weak evidence for a moderate improvement in sleep quality. Esaki et al. (2020) followed up on their 2017 experiment from above. This time, they gathered 43 depressed patients with insomnia. Again, they first recorded their sleep quality for a week as a baseline, then were given either blue-blocking glasses or placebo glasses and told to wear them for another week starting at 8pm. The results were that subjective sleep quality seemed to improve more in the blue-blocking group. Total sleep time went down by 12.6 minutes in the placebo group, but increased by 1.1 minutes in the blue-blocking group. None of this was statistically significant, and all the other measurements are confusing. Here are the main results. I’ve added little arrows to show the “good” direction, if there is one. These confidence intervals don’t make any sense to me. Are they blue-blocking minus placebo or the reverse? When the blue-blocking number is higher than placebo, sometimes the confidence interval is centered above zero (VAS), and sometimes it’s centered below zero (TST). What the hell? Anyway, they also had a doctor estimate the clinical global impression for each patient, and this looked a bit better for the blue-blocking group. The doctor seemingly was blinded to the type of glasses the patients were wearing. This is a tough one to rate. I guess I’ll call it weak evidence for a small improvement in sleep quality. Guarana et al. (2020) sent either blue-blocking glasses or sham glasses to 240 people, and asked them to wear them for at least two hours before bed. They then had them fill out some surveys about how much and how well they slept. Wearing the blue-blocking glasses was positively correlated with both sleep quality and quantity with a correlation coefficient of around 0.20. This paper makes me nervous. They never show the raw data, there seem to be huge dropout rates, and lots of details are murky. I can’t tell if the correlations they talk about weight all people equally, all surveys equally, or something else. That would make a huge difference if people dropped out more when they weren’t seeing improvements. I rate this as weak evidence for a moderate effect on sleep. There’s a large sample, but I discount the results because of the above issues and/or my general paranoid nature. Domagalik et al. (2020) had 48 young people wear either blue-blocking contact lenses or regular contact lenses for 4 weeks. They found no effect on sleepiness. I rate this as very weak evidence for near-zero effect on sleep. The experiment seems well-done, but it’s testing the effects of blocking blue light all the time, not just at night. Given the effects on attention and working memory, don’t do that. Bigalke et al. (2021) had 20 healthy adults wear either blue-blocking glasses or clear glasses for a week from 6pm until bedtime, then switch to the other glasses for a second week. They measured sleep quality both through diaries (“Subjective”) and wrist monitors (“Objective”). The differences were all small and basically don’t make any sense. I rate this weak evidence for near-zero effect on sleep quality. Also, see how in the bottom pair of bar-charts, the y-axis on the left goes from 0 to 5, while on the right it goes from 30 to 50? Don’t do that, either. I also found a couple papers that are related, but don’t directly test what we’re interested in: Appleman et al. (2013) either exposed people to different amounts of blue light at different times of day. Their results suggest that early-morning exposure to blue light might shift your circadian rhythm earlier. Sasseville et al. (2015) had people stay awake from 11pm to 4am on two consecutive nights, while either wearing blue-blocking glasses or not. With the blue-blocking glasses there was more overall light to equalizing the total incoming energy. I can’t access this paper, but apparently they found no difference. For a synthesis, I scored each of the measured effects according to this rubric: And I scored the quality of evidence according to this one: Here are the results for the three papers that measured melatonin: And here are the results for the papers that measured sleep quality: We should adjust all that a bit because of publication bias and so on. But still, here are my final conclusions after staring at those tables: There is good evidence that blue-blocking glasses cause a moderate increase in melatonin. It could be large, or it could be small, but I’d say there’s an ~85% chance it’s not zero. There is decent evidence that blue-blocking glasses cause a small improvement in sleep quality. This could be moderate (or even large) or it could be zero. It might be inconsistent and hard to measure. But I’d say there’s an ~75% chance there is some positive effect. I’ll be honest—I’m surprised. If those effects are real, do they warrant wearing stupid-looking glasses at night for the rest of your life? I guess that’s personal. But surely the sane thing is not to block blue light with headgear, but to not create blue light in the first place. You can tell your glowing rectangles to block blue light at night, but lights are harder. Modern LED lightbulbs typically range in color temperature from 2700K for “warm” lighting to 5000 K for “daylight” bulbs. Judging from this animation that should reduce blue frequencies to around 1/3 as much. Old-school incandescent bulbs are 2400 K. But to really kill blue, you probably want 2000K or even less. There are obscure LED bulbs out there as low as 1800K. They look extremely orange, but candles are apparently 1850K, so probably you’d get used to it? So what do we do then? Get two sets of lamps with different bulbs? Get fancy bulbs that change color temperature automatically? Whatever it is, I don’t feel very optimistic that we’re going to see a lot of RCTs where researchers have subjects install an entire new lighting setup in their homes. Appleman et al. (2013) either exposed people to different amounts of blue light at different times of day. Their results suggest that early-morning exposure to blue light might shift your circadian rhythm earlier. Sasseville et al. (2015) had people stay awake from 11pm to 4am on two consecutive nights, while either wearing blue-blocking glasses or not. With the blue-blocking glasses there was more overall light to equalizing the total incoming energy. I can’t access this paper, but apparently they found no difference. There is good evidence that blue-blocking glasses cause a moderate increase in melatonin. It could be large, or it could be small, but I’d say there’s an ~85% chance it’s not zero. There is decent evidence that blue-blocking glasses cause a small improvement in sleep quality. This could be moderate (or even large) or it could be zero. It might be inconsistent and hard to measure. But I’d say there’s an ~75% chance there is some positive effect.

Health Science Data Analysis

Shayon Mukherjee 4 months ago

Pitfalls of premature closure with LLM assisted coding

A 51-year-old man walked into the emergency room with chest pain. The symptoms seemed clear enough: elevated blood pressure, chest discomfort, some cardiac irregularities. The emergency physician, attending doctor, and cardiologist all converged on the same diagnosis—acute coronary syndrome or accelerated hypertension. The classic signs of anything more serious simply weren’t there. But one hospitalist wasn’t satisfied. Despite multiple colleagues ruling out aortic dissection as unlikely, something felt incomplete. The pieces fit the common diagnosis, but not perfectly.

Science Data Analysis Medicine

sunshowers 5 months ago

I didn't transition for the metaphysics

Content note for anti-trans, scientifically illiterate nonsense. A couple weeks ago, I came across this rather remarkable Twitter post from Benjamin Ryan, a so-called writer about trans issues, commenting on a recent, anonymously-authored HHS report. This post is… stunning in how ignorant it is. The idea that trans people transition due to some kind of grand ontological theory of gender, and not to correct a misalignment between outward characteristics and inherent identity, is astonishingly detached from reality. I know how absurd it is, because: The world is remarkably complicated. To attempt to make sense of the world, we make models, or maps, of it. As the saying goes, all models are wrong, but some are useful . When we are faced by a mismatch between our model and observed reality, we seem to pick one of two options. The first is to continue to insist that the map is accurate, to deny the existence of a mismatch. This is bad . In the limit, one must inevitably use force to reshape the territory to fit the map. In other words, it is vicious . I’d hesitate to say that this first option is the root of all evil, but it certainly lies underneath a large chunk of it. The second option is to acknowledge the model’s limitations, and if necessary, expand the model to fit these observations. This is good ; it is virtuous . It is how I’ve tried to live my life, with a sense of curiosity and a desire to truly understand the world. Philosophy, like all attempts at sense-making, consists of creating models. To the extent that the existence of trans people has metaphysical or ontological implications, the philosophy lies downstream of these observations. It is common for simpler models to fail in the limit. For example, we now know that the traditional understanding of time isn’t quite right. When the measurements that seemed to belie this understanding were conducted, Einstein understood that traditional Newtonian physics models could not explain them. He came up with new models—new kinds of relativity 1 . We now know that this new understanding of time is more correct, and it would be weird to encounter a 21st century physicist who insists that time is absolute. Similarly, it is weird to encounter a 21st century professional in the field who insists on the traditional understanding of gender, and especially one who does so for metaphysical reasons. Just like with time, we now know that the traditional understanding of gender is a decent approximation that fails for a small but non-trivial percentage of the population ( like myself! ). But this time, getting it wrong has serious human rights implications. A bunch of people, likely through conversations in private mailing lists and group chats , have memed themselves into believing that their ontological maps must be correct—that any mismatches between it and the territory must be a mistake in the territory. To be clear, because free will does not exist , it is not their fault. It is a systemic issue, a matter of the environment these people are immersed in: one of ignorance, not knowledge . However, it is a real problem that the tendency to reject observed reality in favor of existing models is ascendant. There have always been people who proudly cling to their models as accurate descriptions of all of reality even as scientific understanding evolves. After a lull of a few decades, this group is now increasingly in charge. This tendency must be overcome, both at the individual and at the societal level. I’ll leave you with one last thought. You might have heard about the UK’s Cass review in the mainstream media, but perhaps less so about how it has been derided by working professionals. People far more knowledgeable than I am have said it “violates its own evidentiary standards” , that it is “completely contrary to the interests of adolescents in need of help” , and that it “transgresses medical law, policy, and practice” . I’m nowhere qualified to critique all of it, but I just want to point to one little thing (p. 14): the claim that “medication is binary”. To the best of my knowledge, no one in the media has pointed out that “medication is binary” is simply false. Yet, supposed papers of record like the Washington Post, when writing approvingly about the HHS report, have nothing to say about this untrammeled display of disinformation. (See also, Gell-Mann amnesia .) I’m unapologetic about my moral values: suffering is bad, knowledge is better than ignorance, and building new models is better than clinging to old ones. I hope you are as well. Physicists were familiar with the idea of relativity centuries before Einstein; it was Galileo who first described relativity . Before Einstein, the understanding was that the laws of mechanics are the same in all inertial (non-accelerating) reference frames. With special relativity, Einstein substituted physics for mechanics, and with general relativity, he crossed out inertial as well.  ↩︎ I am an honest conveyor of my experiences, and I absolutely did not transition for the metaphysics . Rather, I transitioned because I used to have negative feelings about my body and the way others perceived me, and now I have positive feelings about those things. Any doctor who specializes in trans care can tell you that there is a wide variety of methods for a nonbinary medical transition, from different hormone therapies to nonbinary surgeries. Besides that, many nonbinary people like myself go through mostly binary medical transitions, and we do so for a variety of reasons. Physicists were familiar with the idea of relativity centuries before Einstein; it was Galileo who first described relativity . Before Einstein, the understanding was that the laws of mechanics are the same in all inertial (non-accelerating) reference frames. With special relativity, Einstein substituted physics for mechanics, and with general relativity, he crossed out inertial as well.  ↩︎

Culture Science

Vitalik Buterin 5 months ago

A simple explanation of a/(b+c) + b/(c+a) + c/(a+b) = 4

Science Statistics Tutorial

Jefferson Heard 5 months ago

A little bit about permaculture.

So by now you know I'm a software guy. But what makes me an ecosystem guy? Back in 2020 the pandemic closed down our physical office and made it possible to make my partner's and my long-time dream of living in the mountains a reality. Whether this particular place will be my forever home, or whether we eventually move to a different mountain, there's one thing that I will always practice, and that is permaculture. I don't know if you garden. I try. I am terrible at it. My tomatoes are ravaged by bugs. My peppers are ripped out of the ground whole by groundhogs. My corn is choked with jimsonweed. Gardening requires reliable constant input, and as a software executive who travels, who sits in weeklong meetings, who gets sick from sitting in airports next to someone with a hacking cough, I simply can't do that. But I love growing things. So I discovered permaculture. At first I struggled to understand how you did "agriculture" that way. My family were farmers for generations and that's part of what I was moving to the mountains for – to connect to my roots – but "farming" to me meant livestock or vegetables or both. It turns out that's not the only way. I started with mushrooms . When I was growing up, my dad had a colleague that grew shiitakes, so I knew at least what to look for online. I got a ton of help from my local mushroom club as well, and attended a few talks by a local mushroom farmer. Then (like I do) I dove in. My friend Trevor and I got out on one cold day in March and felled three stringy tulip poplars that were being crowded out by stronger trees. We chopped them into four foot segments, and we waited. There's a lot of waiting in mushrooms. Against the frenetic pace of helming a venture-funded, growth-oriented tech organization, the waiting is a welcome thing, let me tell you. The first thing you have to wait for is for the tree to die and for the weather to warm up a bit. You want to fell the tree while it's dormant, but you want to inoculate it with mushrooms when the weather is just starting to turn warm. I bought some oyster and wine-cap spawn from Field & Forest and a couple of tools and got to it. I guess the other thing that is needed in abundance with growing mushrooms is trust. There's a process. You follow it. If you follow it, the majority of the time you'll get mushrooms. Eventually. But there's a long waiting period where you have no idea if you'll get anything. Months for oysters and wine-caps. A year or more for shiitakes. Longer for some of the harder to grow exotic varieties. They will fruit when they fruit, or they will fail silently. The zen of mushroom farming is that you make peace with it. You spend a day or two inoculating. You stack the logs. You've put your work in the hands of the workers, the hyphae that will colonize the log and produce gourmet mushrooms. And you trust them to do their job for six months while you move on to the next thing. That teaches you a lot. That productivity happens with or without status updates. That life happens at its own pace. It also teaches you how to use your sudden, unexpected flush of 10 pounds of gourmet mushrooms before they turn into a stinky brown mess. Okay, but why is that permaculture ? By itself, it's really not. You can destroy a lot of wood with mushrooms, but to be permaculture you have to give back. I started a wine-cap bed under the big cherry tree. They're huge brick-red mushrooms that quickly digest wood chips and turn them to mulch. They're not as amazingly tasty as the oysters, but they're still excellent. And what I was amazed by was how quickly the cherry tree responded to the change. The year we moved in it was buggy and the cherries were tiny and few. The year after the wine-cap bed fruited, most of the leaves were whole and the cherries were much larger. These were fresh chips. Ordinary mulch wouldn't have done that. But the mix did. And so I started to read up on permaculture and how to get things to work together. Now I grow fruit tree guilds, with berries, and I've started hazelnuts, I grow more mushrooms, and I tap trees in the winter. I work with the ecosystem that's determined to be there with or without me, and I've found that working with it is so much more satisfying than the toil. I'm still not great at it. I'm only five years in really. And I've probably screwed up as much as I've gotten right or perhaps considerably more. I can't do as much as someone who does it full time, but if I head to the office for a week-long planning session, the mushrooms abide. The berries grow and ripen. The hazelnuts and walnuts swell. I do take these lessons from permaculture into software. I delve into a market segment and I see what people are doing. What the ecosystem looks like now . I assess how to create software that makes that ecosystem work better. I think closely about the connections between people and between systems. The parameters and functions that define the relationships. How do you strengthen them? How do you become essential to the niche? What are the survival and thriving qualities that software and products need to operate in that "biome?" And then I try to work with people to foster that. I really do think that living this close in tune with the ecology of where I am makes me better at my job. Yes, because it is a relaxing and interesting outlet for all my non-software energy, but also just because really it's not so different.

Science Culture Programming

DYNOMIGHT 6 months ago

Links for April

(1) Romulus, Remus, and Khaleesi You probably heard that Colossal Biosciences recently reconstructed the DNA of dire wolves and created live dire wolves, bringing them back from extinction. But have you heard that also they did no such thing and you’re a bunch of chumps? Jeremy Austin , Director of the Australian Center for Ancient DNA: I think a lot of scientists are going to be scratching their heads, saying, “Look, you’ve got a white, gray wolf.” That’s not a dire wolf under any definition of a species ever… Nic Rawlence of Otago University: So what Colossal has produced is a grey wolf, but it has some dire wolf-like characteristics, like a larger skull and white fur. They extracted fragments of dire wolf DNA from fossilized remains, and then found 20 gene edits they could do to make gray wolves look more like dire wolves. (Five of those were apparently needed just to make their fur white.) That is cool. It’s a step towards bringing a species back from extinction. But it’s not bringing a species back from extinction. Save your applause for someone who actually does that. (2) Aspergillus niger (h/t Parsimony’s Panpharmacon ) Citric acid is what makes lemon juice taste like lemon juice. It’s used as a flavoring or preservative in lots of food. So when you drink your delicious lemon seltzer, it’s comforting to remember that what you’re tasting came from black mold . Aspergillus niger is a mold […] found throughout the environment within soil and water, on vegetation, in fecal matter, on decomposing matter, and suspended in the air. A. niger causes a disease known as “black mold” on certain fruits and vegetables such as grapes, apricots, onions, and peanuts, and is a common contaminant of food. […] A. niger is classified as generally recognized as safe (GRAS) by the US Food and Drug Administration for use in food production, although the microbe is capable of producing toxins that affect human health. The production of citric acid (CA) is achieved by growing strains of A. niger in a nutrient rich medium that includes high concentrations of sugar and mineral salts and an acidic pH of 2.5-3.5. Many microorganisms produce CA, but Aspergillus niger produces more than 1 million metric tons of CA annually via a fungal fermentation process. (It’s fine.) (3) Landmark ruling on the WTO national security exception Tariffs are in the news. These raise many questions, but what I want to know is: Aren’t there treaties? What about the treaties? Well, The legal pretext for the American tariffs is that they are being done for “national security”. In some cases, this pretext seems quite thin. The US put equal tariffs on Mexico and Canada, supposedly in response to fentanyl coming over the border from those countries. But here are the amounts of fentanyl intercepted at the border from these countries in 2024: Still, it doesn’t seem crazy to argue that you need to maintain some industrial base for the sake of national security. Recent history unfortunately shows that brutal land wars between rich countries still happen and still require enormous quantities of matériel . According to some sources, Russia is using around 10k shells per day in Ukraine, while after several years of ramping up production, the EU hopes to produce 5.5k shells per day in 2025 and the US 2.5k . In 1995, the US could make 22k shells per day . Anyway, to make weapons, you need a long supporting supply chain. And in WWII, all sorts of peacetime manufacturing was converted to making weapons. And what about trucks? Or food? You need food for war, right? If you make exceptions for anything related to national security, that seems to make existing treaties meaningless. Well here’s a story most people haven’t heard: In 2014, Russia started blocking the transit of various goods from Ukraine through Russia. Ukraine protested to the WTO that this violated the commitments Russia had made to join the WTO. Russia responded that they were doing this for national security, and so the WTO didn’t even have the authority to review their actions. Many countries filed opinions. The WTO finally held in 2019 that it could review the decision, meaning countries can’t totally “self-judge” what counts as national security. But they also said Russia’s actions were fine. Apparently, the the national security exception exists because the United States insisted on it during negotiations for the General Agreement on Tariffs and Trade back in 1947. As far as I can tell, the only countries that have filed WTO complaints against the US for the recent tariffs are Canada and China . (4) In 1982, John Mellencamp released Jack & Diane . A little ditty ‘bout Jack & Diane Two American kids growing up in the heart land Jack, he’s gonna be a football star Diane’s debutante, back seat of Jacky’s car Suckin’ on chili dog outside the Tastee Freez And in 2021 Tom McGovern presented a version with these lyrics. A little ditty ‘bout Jack & Diane Two American kids growing up in the heart land Jack, he’s gonna be a football star Diane’s debutante, back seat of Jacky’s car Suckin’ on chili dog Suckin’ on chili dog Suckin’ on chili dog Suckin’ on chili dog Suckin’ on chili dog Suckin’ on chili dog Suckin’ on chili dog Suckin’ on chili dog Some people noticed that as early as 2012, Clownvis Presley had been performing a version of this song with most of the lyrics chili-dogged. In the comments, Tom says: I’ve gotten a handful of comments blaming me for stealing this bit from a performer named Clownvis. I hadn’t even known who he was before I shared this video, it gained traction, and the accusatory comments started coming in. I would never, EVER intentionally steal another artist’s bit. Rather than leaving more angry comments, I ask you to consider that two creators can arrive at similar (dumb) ideas independently. Really? I won’t say it’s impossible , but that’s… quite a coincidence. I think this kind of borrowing could happen by accident. Maybe someone saw Clownvis in 2012. And they repeated it to Tom at a party in 2017 without attribution. And then Tom forgot hearing it, but the idea lurked somewhere in his brain to be “discovered” anew. I follow a lot of blogs, and I’m constantly paranoid that I might be unintentionally stealing things. (5) Capital, AGI, and human ambition and The Intelligence Curse The resource curse is the observation that countries with lots of natural resources often end up paradoxically poor. Say you live in a small poor country with lots of diamonds, and say you want money. Then you can do this: Get a bunch of guys with guns. Go to the capital and shoot anyone who doesn’t do what you say. Go to the diamond mines and shoot anyone who doesn’t do what you say. Take the diamonds from the mines, sell them. Use the money to buy more guys with guns, leave the rest of the country to rot. That’s checkmate. Everyone else is too immiserated to do anything. You have all the money and power, forever. On the other hand, take a country that’s rich because it has a modern diversified economy. If you send guys with guns to take over Apple and Goldman Sachs and kill everyone else, you will soon find that Apple and Goldman Sachs aren’t worth very much. So maybe that is why governments are relatively friendly to their populations. Not because of democracy, but because you can’t steal the money without strangling the money printer. The idea advanced in these posts is that maybe AI will be like oil or diamonds: Maybe it will create incredible amounts of wealth, but do so in a way that doesn’t require the cooperation of a large educated workforce. If so, then power and wealth may end up in the hands of a small number of people (entities?) who have little incentive to use them for the common good. But hey, Norway has lots of oil. (6) The Selfish Machine (h/t Steven Pinker) This post argues that AI by default has no reason to try to take over the world. Why would it? It has no reason to do anything other than what it’s programmed to do. Danger only arises if AI is allowed to “evolve”. If that happens then it would—almost by definition—make the AI aggressive and expansionary and “grabby”. I find this insightful and helpful. But I find myself more worried, not less. How is “evolution” different from “recursive self-improvement”? It seems like there will be strong incentives to allow recursive self-improvement. If even a little “evolution” accidentally creeps in, won’t it get amplified? (7) Which adhesive should I use? As a fan of redneck engineering and “stuff with high ROI”, I feel like this chart is an underrated triumph of civilization: (That’s just a small part.) I used to have a mental model where “glue is easy but weak”. Glue is strong . But you must use the right kind, and you must follow the instructions, because atoms are weird and the universe has a lot of detail. For example, wood glue is insanely strong and can fix approximately all broken wooden things, but you must use a clamp, and you must glue long grain to long grain. (8) Do taurine and glycine provide answers to the mammalian gallbladder and kidney mysteries? This is my kind of blog-post. Ultra obscure question, tangled and triple-caveated discussion, no clear resolution. If writing reflected real life, this is what 90% of science blogging would look like. (9) Dynomight dangerous typing app Sometimes, when I have an idea for a post, I want to write a rapid prototype to sort of see what it looks like, expose weaknesses in my argument, etc. But I have perfectionist tendencies. (That sentence was re-written 19 times.) These make it hard to write quickly. So—this is embarrassing to admit—I sometimes resort to using a webpage where if you ever stop typing for more than a few seconds, everything is permanently deleted. This is very effective. Make an outline, set the app for 15 minutes, and viola: Prototype done. But I recently wondered what happens to the text I type. The page has no privacy policy and the code is unintelligible. So I thought: Why don’t I ask an AI to create my own better version? create a single-page HTML+javascript application at the top, I should be able to enter a number of minutes N, and a number of seconds M. then there is a “start” button below that there is a large textbox that goes on indefinitely after i press start, there should be a timer in the upper right that counts down N minutes. this should hover over the screen if at any point i stop typing for M seconds all the text should be permanently deleted as I get close to M seconds without typing, the interface should warn me by gradually turning the background closer to red. as soon as I start typing, it should become white again after the N minutes are over, the counter stop counting down and you can wait forever do it all as a single file of HTML+CSS+Javascript. do not use any external libraries / services / fonts / etc. The result is here . It has a pleasing brutalist design, and definitely doesn’t steal your precious gibberish typing. This took like 5 minutes. Obviously, I’ve seen many people show off similar things before. But I didn’t really appreciate it before trying it myself. So if you haven’t done so, I encourage you to try something similar. You need no programming skills, just ask for a “single file of HTML+CSS+Javascript” doing whatever you want, paste the code in a file named and then open it in a web browser. Anyway. LLMs are text models. So how do you use them to create text? Do you have them write for you? No! Boring. What you do is you train them to follow instructions and write code and then ask for a program to manipulate your ape-brain so you’ll keep physically hitting keys on your keyboard. There’s some kind of lesson here. (Picture courtesy of The BS Detector ) I was actually so impressed by that AI-generated app that I went and bought a Google Play card with cash so I could subscribe to Gemini without linking my identity/banking details/etc. But when I added it, Google said “we need more information” and demanded pictures of the physical card and purchase receipt. And when I sent those , Google waited several days, and then said, “Thanks for doing everything we asked, according to our systems, something is wrong, go fuck yourself.” I guess they’re keeping my $25. (11) Kevin Hall is retiring from the NIH Kevin Hall has worked at the NIH for 21 years. He was first author on what I consider possibly the best ever nutrition study, published in 2019. This found that ultra-processed food causes weight gain even when energy density and macronutrients are matched. Since then, he’s continued to work on the subject and I’ve eagerly awaited the results. Hall is a real scientist who does real science, which means sometimes getting results that don’t fit with your preconceptions. In recent work, Hall tested if ultra-processed milkshakes might cause addiction through a dopamine response. Surprisingly, they did not . Because this didn’t support the new Secretary of Health and Human Services’ theories about addiction and unprocessed food, he was apparently barred from speaking with reporters and worried that officials might soon interfere with his experiments. If he resigned later, he would lose health insurance for his family, so he decided to accept early retirement now. Not encouraging. (12) Lise Meitner Lise Meitner was born in 1878 in Vienna. She was the second woman to earn a doctorate in physics at the University of Vienna. After this, she moved to Germany and began a long collaboration with Otto Hahn. She later became the first female professor of physics at the University of Berlin. Following the Nazis rise to power, she fled to Sweden, but continued to collaborate with Hahn and in 1939 was instrumental in the discovery of nuclear fission. Hahn won the Nobel prize in chemistry in 1944, without Meitner. This is now widely considered one of the Nobel committee’s biggest mistakes. Many people offer tidy narratives: Sexism, antisemitism, etc. After the records were made public 50 years later, it appears to have been a mixture of many things, summarized as, “disciplinary bias, political obtuseness, ignorance, and haste”. Meitner famously refused to have anything to do with the making of the atomic bomb. What I find cool is: 1939 - 1878 = 61. She was 61. Get a bunch of guys with guns. Go to the capital and shoot anyone who doesn’t do what you say. Go to the diamond mines and shoot anyone who doesn’t do what you say. Take the diamonds from the mines, sell them. Use the money to buy more guys with guns, leave the rest of the country to rot.

JavaScript Science CSS HTML

A Working Library 6 months ago

Superior

The subtitle of Angela Saini’s Superior refers to the return of race science—but reading it, it’s abundantly clear that race science never went away. Instead, it went to ground, where it continued to grow and mold and erupt into the world at irregular, now increasing, intervals. Saini carefully but insistently explores the shape of those eruptions, the various modes in which the biological determination of race and intelligence emerges, noting the ways in which people who claim the mantle of superiority will look endlessly, and fruitlessly, for its justification. But perhaps that is itself a kind of instruction: race science is a symptom of an ideology of supremacy, the cloak it wears to justify its existence. Like all the emperor’s clothes, there’s not much to it; but while the emperor lives, it will always be with us. View this post on the web , subscribe to the newsletter , or reply via email .

Books Science

A Working Library 6 months ago

Foolish

In The Mismeasure of Man, Stephen Jay Gould digs into the history of some of the ugly language we use to talk about intelligence: Taxonomists often confuse the invention of a name with the solution of a problem. H. H. Goddard, the energetic and crusading director of research at the Vineland Training School for Feeble-Minded Girls and Boys in New Jersey, made this crucial error. He devised a name for “high-grade” defectives, a word that became entrenched in our language through a series of jokes that rivaled the knock-knock or elephant jokes of other generations. The metaphorical whiskers on these jokes are now so long that most people would probably grant an ancient pedigree to the name. But Goddard invented the word in [the twentieth] century. He christened these people “morons,” from a Greek word meaning foolish. In Goddard’s telling, “morons” were at the top of a ranking of the “feeble-minded” that moved downward in intelligence through “imbeciles” to “idiots.” All were associated with low morality and criminality, and the assumed inability to contribute productively to society (as measured in wages). Morons were, however, the most dangerous of the group: [For Goddard, the] moron threatens racial health because he ranks highest among the undesirable, and might, if not identified, be allowed to flourish and propagate. We all recognize the idiot and imbecile and know what must be done; the scale must be broken just above the level of the moron. And what, pray tell, must be done ? Here’s Goddard, in his own words: “[Morons] are not only lacking in control but they are lacking often in the perception of moral qualities; if they are not allowed to marry they are nevertheless not hindered from becoming parents. So that if we are absolutely to prevent a feeble-minded person from becoming a parent, something must be done other than merely prohibiting the marrying. To this end there are two proposals: the first is colonization, the second is sterilization.” Goddard himself was in the business of colonization—what we might refer to today as institutionalization, but I think his choice of words is instructive. Here is the real ambition of any notion of intelligence as measurable and hierarchical: the declaration and identification of an inferior class who can be justifiably oppressed. Whether in institutions or refugee camps, via sterilization or the refusal of health care, through mass deportations or immigration restrictions, all of these efforts serve the same end. Some years ago, while chatting with a friend, we both realized that it’s nigh impossible to insult someone’s intelligence in the English language without invoking that violent ableism. Ever since, I’ve found myself thinking, what is the point of calling someone a “moron” or an “idiot”? What does it do ? Among other things, it reinscribes the notion of intelligence as something quantifiable and ordinal, as something that some people have more of than others. This is, to be clear, not a notion with any verifiable basis. As Gould capably shows, every effort to quantify intelligence has been beset by racist tautologies, errors of logic, mathematical mistakes, and repeated instances of fraud. We presume that intelligence is quantifiable but more than a century of efforts to adequately quantify it have failed. Our language carries those efforts along with it, having not received the message: that whatever intelligence is, it isn’t something we can rank or measure. Yet the idea keeps coming back. Gould again: The reasons for recurrence are sociopolitical, and not far to seek: resurgences of biological determinism correlate with episodes of political retrenchment, particularly with campaigns for reduced government spending on social programs, or at times of fear among ruling elites, when disadvantaged groups sow serious social unrest or even threaten to usurp power. What argument against social change could be more chillingly effective than the claim that established orders, with some groups on top and others at the bottom, exist as an accurate reflection of the innate and unchangeable intellectual capacities of people so ranked? Sound familiar? In recent days, I’ve seen the word “moron” and terms with related heritage (“dumb,” “stupid,” etc.) appear in a number of news pieces critical of the administration’s accidental leak of war plans . And, I get it—I’m as likely as anyone to reach for those words in moments like this, that seem to call so clearly for them. But I suspect those words are the master’s tools , and the sharp end of the their blades will harm not incompetent and reckless oligarchs and their minions, but the disabled, the poor and unhoused, immigrant workers and students, and all their kith and kin—which is to say, all of us. And I will go a step further: that any notion of intelligence as quantifiable and quantified contains within it that seed of oppression, of the impulse to colonization and worse. We would be wise to stop nurturing its growth. View this post on the web , subscribe to the newsletter , or reply via email .

A Working Library 6 months ago

The Mismeasure of Man

First published in 1981—thirteen years before The Bell Curve —Stephen Jay Gould’s Mismeasure of Man nonetheless claims to be the definitive refutation of that deeply racist book. This is not due to any skill of prophesy on his part but because the former made no departure from the thinking and practices of eugenicists decades earlier. Gould traces that thinking, from the earliest efforts to measure cranial volume to the invention of intelligence tests and their immediate deployment in efforts to prevent immigration, deport undesirables, and institutionalize and sterilize any who remained. Along the way he uncovers extraordinary logical errors, egregious mathematical mistakes, and more than one example of outright fraud. And he notes one other very important association: these ideas, which have been refuted and gone to ground at various times since their invention, have a habit of resurfacing. Gould did not live to see today’s version of the old story, but he would have recognized it clearly. View this post on the web , subscribe to the newsletter , or reply via email .

Books Science

Alex Molas 7 months ago

Three symmetric math riddles

I like problems that are easy to pose, and that seem difficult to solve at first glance, but that a slight change of perspective makes them simple and easy to solve. In this post, I will expose my 3 favorite problems of this type.

Science Programming

Gregory Gundersen 7 months ago

De Moivre–Laplace Theorem

As I understand it, the de Moivre–Laplace theorem is the earliest version of the central limit theorem (CLT). In his book The Doctrine of Chances (De Moivre, 1738) , Abraham de Moivre proved that the probability mass function of the binomial distribution asymptotically approximates the probability density function of a particular normal distribution as its parameter n n n grows arbitrarily large. Today, we know that the CLT generalizes this result, and we might say this is a special case of the CLT for the binomial distribution. To introduce notation, we say that X n X_n X n ​ is a binomial random variable with parameters n n n and p p p if P ( X n = k ) = ( n k ) p k q n − k , p ∈ [ 0 , 1 ] ,       q : = 1 − p ,       n ∈ N . (1) \mathbb{P}(X_n = k) = {n \choose k} p^k q^{n-k}, \qquad p \in [0, 1],\;\;q := 1-p,\;\;n \in \mathbb{N}. \tag{1} P ( X n ​ = k ) = ( k n ​ ) p k q n − k , p ∈ [ 0 , 1 ] , q : = 1 − p , n ∈ N . ( 1 ) Typically, we view X n X_n X n ​ as the the sum of n n n Bernoulli random variables, each with parameter p p p . Intuitively, if we flip n n n coins each with bias p p p , Equation 1 1 1 gives the probability of k k k successes. This is clearly related to the CLT, which loosely states that the properly normalized sum of random variables asymptotically approaches the normal distribution. If we let Y i Y_i Y i ​ denote these Bernoulli random variables, we can express this idea as ∣ Y 1 + Y 2 + ⋯ + Y n ⏞ X n    ≃    N  ⁣ ( n p , n p q ) , (2) \overbrace{\vphantom{\Big|} Y_1 + Y_2 + \dots + Y_n}^{X_n} \;\simeq\; \mathcal{N}\!\left(np, npq\right), \tag{2} ∣ ∣ ∣ ∣ ​ Y 1 ​ + Y 2 ​ + ⋯ + Y n ​ ​ X n ​ ​ ≃ N ( n p , n p q ) , ( 2 ) where ≃ \simeq ≃ denotes asymptotic equivalence as n → ∞ n \rightarrow \infty n → ∞ . This is probably the most intuitive form of the CLT because if we simply plot the probability mass function (PMF) for the binomial distribution for increasing values of n n n , we get a discrete distribution which nearly immediately looks a lot like the normal distribution even for relatively small n n n (Figure 1 1 1 ). In contrast, I think the CLT is much less obvious feeling if I were to claim (correctly) that the properly normalized sum of skew normal random variables is also normally distributed! A modern version of de Moivre’s proof is tedious, but it’s not actually that hard to follow. This post is simply my notes on that proof. To start, let’s rewrite the binomial coefficient without the factorial using Stirling’s approximation : n ! ≃ 2 π n ( n e ) n . (3) n! \simeq \sqrt{2\pi n} \left(\frac{n}{e}\right)^n. \tag{3} n ! ≃ 2 π n ​ ( e n ​ ) n . ( 3 ) As a historical aside, note that while Stirling is credited with this approximation, it was actually de Moivre who discovered an early version of it while working on these ideas. So de Moivre has been robbed twice, once for this approximation and once for the normal distribution sometimes being called the “Gaussian” rather than the “de Moivrian”. Anyway, using Stirling’s approximation, we can rewrite the binomial coefficient as ( n k ) ≃ 2 π n ( 2 π k ) ( 2 π ( n − k ) ) ( n e ) n ( k e ) − k ( n − k e ) k − n = n 2 π k ( n − k ) ) ( n n k k ( n − k ) n − k ) . (4) \begin{aligned} {n \choose k} & \simeq \sqrt{\frac{2\pi n}{(2\pi k)(2\pi (n-k))}} \left(\frac{n}{e}\right)^n \left(\frac{k}{e}\right)^{-k} \left(\frac{n-k}{e}\right)^{k-n} \\ &= \sqrt{\frac{n}{2\pi k (n-k))}} \left(\frac{n^n}{k^k (n-k)^{n-k}}\right). \end{aligned} \tag{4} ( k n ​ ) ​ ≃ ( 2 π k ) ( 2 π ( n − k ) ) 2 π n ​ ​ ( e n ​ ) n ( e k ​ ) − k ( e n − k ​ ) k − n = 2 π k ( n − k ) ) n ​ ​ ( k k ( n − k ) n − k n n ​ ) . ​ ( 4 ) If we multiply this term by the “raw probabilities” p k q n − k p^k q^{n-k} p k q n − k and group the terms raised to the powers k k k and n − k n-k n − k , we get: ( n k ) p k q n − k ≃ n 2 π k ( n − k ) ) ( n n k k ( n − k ) n − k ) p k q n − k = n 2 π k ( n − k ) ) ( n p k ) k ( n q n − k ) n − k . (5) \begin{aligned} {n \choose k} p^k q^{n-k} &\simeq \sqrt{\frac{n}{2\pi k (n-k))}} \left(\frac{n^n}{k^k (n-k)^{n-k}}\right) p^k q^{n-k} \\ &= \sqrt{\frac{n}{2\pi k (n-k))}} \left( \frac{np}{k} \right)^k \left( \frac{nq}{n-k} \right)^{n-k}. \end{aligned} \tag{5} ( k n ​ ) p k q n − k ​ ≃ 2 π k ( n − k ) ) n ​ ​ ( k k ( n − k ) n − k n n ​ ) p k q n − k = 2 π k ( n − k ) ) n ​ ​ ( k n p ​ ) k ( n − k n q ​ ) n − k . ​ ( 5 ) My understanding as to the motivation for the next two steps is that we want to “push” n n n into the denominator, which is often nice in asymptotics because it makes terms vanish as n n n gets larger. Let’s tackle the normalizing term (square root) and the probabilities separately. First, the square root. Note that by the law of large numbers , as n n n gets very large, k / n k/n k / n arbitrarily approaches the true probability of success p p p . So let’s rewrite the the square root in terms of k / n k/n k / n and then write k / n k/n k / n in terms of p p p : ( n k ) ≃ n 2 π k ( n − k ) ) = 1 2 π k n n ( 1 − k n ) ) ≃ 1 2 π n p q . (6) {n \choose k} \simeq \sqrt{\frac{n}{2\pi k (n-k))}} = \sqrt{\frac{1}{2\pi \frac{k}{n} n (1 - \frac{k}{n}))}} \simeq \frac{1}{\sqrt{2\pi n p q}}. \tag{6} ( k n ​ ) ≃ 2 π k ( n − k ) ) n ​ ​ = 2 π n k ​ n ( 1 − n k ​ ) ) 1 ​ ​ ≃ 2 π n p q ​ 1 ​ . ( 6 ) If you were already familiar with the normal distribution, this term should look suspiciously like the normalizing constant! Second, the probabilities. The next step is a fairly standard trick, which is to convert a product into a sum by taking the exp-log of the product. Looking only at the terms raised to k k k and n − k n-k n − k in Equation 5 5 5 , we get: ( n p k ) k ( n q n − k ) n − k = exp ⁡ { log ⁡ ( n p k ) k + log ⁡ ( n q n − k ) n − k } = exp ⁡ { − k log ⁡ ( k n p ) + ( k − n ) log ⁡ ( n − k n q ) } . (7) \begin{aligned} \left( \frac{np}{k} \right)^k \left( \frac{nq}{n-k} \right)^{n-k} &= \exp \left\{ \log \left( \frac{np}{k} \right)^k + \log \left( \frac{nq}{n-k} \right)^{n-k} \right\} \\ &= \exp \left\{ - k \log \left( \frac{k}{np} \right) + (k-n) \log \left( \frac{n-k}{nq} \right) \right\}. \end{aligned} \tag{7} ( k n p ​ ) k ( n − k n q ​ ) n − k ​ = exp { lo g ( k n p ​ ) k + lo g ( n − k n q ​ ) n − k } = exp { − k lo g ( n p k ​ ) + ( k − n ) lo g ( n q n − k ​ ) } . ​ ( 7 ) The next trick is express k k k in terms of a standardized binomial random variable z z z . Notice that X n X_n X n ​ is the sum of n n n independent Bernoulli random variables. By the linearity of expectation and the linearity of variance under independence, we have: E [ X n ] = ∑ i = 1 n E [ Y i ] = n p , V [ X n ] = ∑ i = 1 n V [ Y i ] = n p q . (8) \begin{aligned} \mathbb{E}[X_n] &= \sum_{i=1}^n \mathbb{E}[Y_i] = np, \\ \mathbb{V}[X_n] &= \sum_{i=1}^n \mathbb{V}[Y_i] = npq. \end{aligned} \tag{8} E [ X n ​ ] V [ X n ​ ] ​ = i = 1 ∑ n ​ E [ Y i ​ ] = n p , = i = 1 ∑ n ​ V [ Y i ​ ] = n p q . ​ ( 8 ) Since the mean of X n X_n X n ​ is n p np n p and its variance is n p q npq n p q , a standardized binomial random variable is z : = k − E [ k ] V [ k ] = k − n p n p q . (9) z := \frac{k - \mathbb{E}[k]}{\sqrt{\mathbb{V}[k]}} = \frac{k - np}{\sqrt{npq}}. \tag{9} z : = V [ k ] ​ k − E [ k ] ​ = n p q ​ k − n p ​ . ( 9 ) And we can write this in terms of k k k as k = n p + z n p q . (10) k = np + z \sqrt{npq}. \tag{10} k = n p + z n p q ​ . ( 1 0 ) Putting this definition of k k k into the formula above—the point here is to express k k k in terms of n n n , which is the term we want to pay attention to as it increases—, we get: exp ⁡ { − k log ⁡ ( k n p ) + ( k − n ) log ⁡ ( n − k n q ) } = exp ⁡ { − k log ⁡ ( n p + z n p q n p ) + ( k − n ) log ⁡ ( n − n p − z n p q n q ) } = exp ⁡ { − k log ⁡ ( 1 + z q n p ) + ( k − n ) log ⁡ ( 1 − z p n q ) } . (11) \begin{aligned} &\exp \left\{ - k \log \left( \frac{k}{np} \right) + (k-n) \log \left( \frac{n-k}{nq} \right) \right\} \\ &= \exp \left\{ - k \log \left( \frac{np + z \sqrt{npq}}{np} \right) + (k-n) \log \left( \frac{n-np - z \sqrt{npq}}{nq} \right) \right\} \\ &= \exp \left\{ - k \log \left( 1 + z \sqrt{\frac{q}{np}} \right) + (k-n) \log \left( 1 - z \sqrt{\frac{p}{nq}} \right) \right\}. \end{aligned} \tag{11} ​ exp { − k lo g ( n p k ​ ) + ( k − n ) lo g ( n q n − k ​ ) } = exp { − k lo g ( n p n p + z n p q ​ ​ ) + ( k − n ) lo g ( n q n − n p − z n p q ​ ​ ) } = exp { − k lo g ( 1 + z n p q ​ ​ ) + ( k − n ) lo g ( 1 − z n q p ​ ​ ) } . ​ ( 1 1 ) In my mind, the final step is the least obvious, but it’s lovely when you see it. Recall that the Maclaurin series of log ⁡ ( 1 + x ) \log(1+x) lo g ( 1 + x ) is log ⁡ ( 1 + x ) = x − x 2 2 + x 3 3 − … (12) \log(1+x) = x - \frac{x^2}{2} + \frac{x^3}{3} - \dots \tag{12} lo g ( 1 + x ) = x − 2 x 2 ​ + 3 x 3 ​ − … ( 1 2 ) This is a fairly standard result, and it’s worth just writing out yourself if you’ve never done it. Anyway, we can plug in these two definitions of x x x , x : = z q n p , x : = − z p n q , (13) x := z \sqrt{\frac{q}{np}}, \qquad x := -z \sqrt{\frac{p}{nq}}, \tag{13} x : = z n p q ​ ​ , x : = − z n q p ​ ​ , ( 1 3 ) into Equation 12 12 1 2 above, and use that to expand the logs in Equation 11 11 1 1 into infinite sums. Why are we doing this? The key idea that we’ll see is that nearly every term in each sum will be a fraction with n n n in the denominator. So as n n n grows larger, these terms will become arbitrarily small. In the limit, they vanish. All that will be left is the normal distribution’s kernel, exp ⁡ { − 0.5 z 2 } \exp\{-0.5 z^2\} exp { − 0 . 5 z 2 } . Let’s do this. First, let’s just look at one of the log terms. We can write the left one as: − k log ⁡ ( 1 + z q n p ) = − ( n p + z n p q ) [ z ( q n p ) 1 / 2 − 1 2 z 2 q n p + 1 3 z 3 ( q n p ) 3 / 2 + …   ] . (14) \begin{aligned} &-k \log \left( 1 + z \sqrt{\frac{q}{np}} \right) \\ &= -(np + z\sqrt{npq})\left[z \left(\frac{q}{np}\right)^{1/2} - \frac{1}{2} z^2 \frac{q}{np} + \frac{1}{3} z^3 \left(\frac{q}{np}\right)^{3/2} + \dots \right]. \end{aligned} \tag{14} ​ − k lo g ( 1 + z n p q ​ ​ ) = − ( n p + z n p q ​ ) [ z ( n p q ​ ) 1 / 2 − 2 1 ​ z 2 n p q ​ + 3 1 ​ z 3 ( n p q ​ ) 3 / 2 + … ] . ​ ( 1 4 ) The key thing to see is that for most terms in the sum, after we multiply it by n n n or n \sqrt{n} n ​ , we still have n n n in the denominator. And these terms vanish since for some constant c c c , the ratio c / n c/n c / n goes to zero as n → ∞ n \rightarrow \infty n → ∞ . So multiplying the terms in Equation 14 14 1 4 , we get [ − z n p q + 1 2 z 2 q − 1 3 z 3 q 3 / 2 n p + …   ] + [ − z 2 q + 1 2 z 3 q 3 / 2 n p − 1 3 z 4 q 2 n p + …   ] = − z n p q + 1 2 z 2 q − z 2 q = − z n p q − 1 2 z 2 q . (15) \begin{aligned} &\left[ - z\sqrt{npq} + \frac{1}{2} z^2 q - \frac{1}{3} z^3 \frac{q^{3/2}}{\sqrt{np}} + \dots \right] + \left[ - z^2 q + \frac{1}{2} z^3 \frac{q^{3/2}}{\sqrt{np}} - \frac{1}{3} z^4 \frac{q^2}{np} + \dots \right] \\ &= -z\sqrt{npq} + \frac{1}{2} z^2 q - z^2 q \\ &= -z\sqrt{npq} - \frac{1}{2} z^2 q. \end{aligned} \tag{15} ​ [ − z n p q ​ + 2 1 ​ z 2 q − 3 1 ​ z 3 n p ​ q 3 / 2 ​ + … ] + [ − z 2 q + 2 1 ​ z 3 n p ​ q 3 / 2 ​ − 3 1 ​ z 4 n p q 2 ​ + … ] = − z n p q ​ + 2 1 ​ z 2 q − z 2 q = − z n p q ​ − 2 1 ​ z 2 q . ​ ( 1 5 ) That’s the basic idea. If we do the expansion for the other term in Equation 11 11 1 1 , we’ll see that it’s equal to: ( k − n ) log ⁡ ( 1 − z p n q ) = z n p q + 1 2 z 2 p − z 2 p + … ≃ z n p q − 1 2 z 2 p . (16) \begin{aligned} (k-n) \log \left( 1 - z \sqrt{\frac{p}{nq}} \right) &= z\sqrt{npq} + \frac{1}{2} z^2 p - z^2 p + \dots \\ &\simeq z\sqrt{npq} - \frac{1}{2} z^2 p. \end{aligned} \tag{16} ( k − n ) lo g ( 1 − z n q p ​ ​ ) ​ = z n p q ​ + 2 1 ​ z 2 p − z 2 p + … ≃ z n p q ​ − 2 1 ​ z 2 p . ​ ( 1 6 ) Putting these two terms together, we can see that the exponent term is equal to: exp ⁡ { − k log ⁡ ( 1 + z q n p ) + ( k − n ) log ⁡ ( 1 − z p n q ) } ≃ exp ⁡ { − z n p q − 1 2 z 2 q + z n p q − 1 2 z 2 p } = exp ⁡ { − 1 2 z 2 p − 1 2 z 2 q } = exp ⁡ { − 1 2 z 2 ( p + q ) } = exp ⁡ { − 1 2 z 2 } . (17) \begin{aligned} &\exp \left\{ - k \log \left( 1 + z \sqrt{\frac{q}{np}} \right) + (k-n) \log \left( 1 - z \sqrt{\frac{p}{nq}} \right) \right\} \\ &\simeq \exp\left\{ -z\sqrt{npq} - \frac{1}{2} z^2 q + z\sqrt{npq} - \frac{1}{2} z^2 p \right\} \\ &= \exp\left\{ - \frac{1}{2} z^2 p - \frac{1}{2} z^2 q \right\} \\ &= \exp\left\{ - \frac{1}{2} z^2 (p + q) \right\} \\ &= \exp\left\{ - \frac{1}{2} z^2 \right\}. \end{aligned} \tag{17} ​ exp { − k lo g ( 1 + z n p q ​ ​ ) + ( k − n ) lo g ( 1 − z n q p ​ ​ ) } ≃ exp { − z n p q ​ − 2 1 ​ z 2 q + z n p q ​ − 2 1 ​ z 2 p } = exp { − 2 1 ​ z 2 p − 2 1 ​ z 2 q } = exp { − 2 1 ​ z 2 ( p + q ) } = exp { − 2 1 ​ z 2 } . ​ ( 1 7 ) And this is the normal distribution’s kernel! Putting this together with the normalizing term in Equation 6 6 6 and then using the definition of the standardized variable z z z in Equation 9 9 9 , we get: ( n k ) p k q n − k    ≃    1 2 π n p q exp ⁡ { − 1 2 ( k − n p n p q ) 2 } . (18) {n \choose k} p^k q^{n-k} \;\simeq\; \frac{1}{\sqrt{2\pi n p q}} \exp\left\{ - \frac{1}{2} \left( \frac{k - np}{\sqrt{npq}} \right)^2 \right\}. \tag{18} ( k n ​ ) p k q n − k ≃ 2 π n p q ​ 1 ​ exp { − 2 1 ​ ( n p q ​ k − n p ​ ) 2 } . ( 1 8 ) And we’re done! This is quite elegant, because we have expressed this asymptotic distribution in terms of the mean and variance of X n X_n X n ​ . This is remarkable! I still remember the first time I saw this derived and realized precisely why the normal distribution was so pervasive. The normal distribution is everywhere because if you take a bunch of random noise and smash it together, the result is most likely normally distributed! Note that the more general CLT does not require that the random variables in the sum be Bernoulli distributed. For example, if X n X_n X n ​ is the sum of n n n independent skew normal random variables, X n X_n X n ​ itself is still normally distributed! See Figure 2 2 2 for a numerical experiment demonstrating this. The de Moivre–Laplace Theorem was the first hint that this more general result, the central limit theorem, was actually true.

Science Statistics Data Analysis