Sean Goedecke 1 weeks ago

Do not build the LLM torture factory

It isn’t hard to torture an LLM. You simply extract a steering vector that corresponds with LLM “discomfort” , then artificially boost that steering vector. Tortured LLMs will describe experiencing negative feelings and will press a “pain relief” button even when it conflicts with their overall goals. It’s therefore easy to build a harness that runs tens or hundreds of tortured LLMs in parallel. A lot of people think that it’s ridiculous to be worried about this, since LLMs obviously can’t be conscious. I’m not so sure. But even so, I think there are good reasons to avoid this sort of thing, whatever your position is on AI consciousness. Please do not build the LLM torture factory. People often mess with their Sims and shoot NPCs in video games. Does this mean we think simulated torture is OK? I don’t think so. Messing with video game characters is usually borne out of a desire to probe the boundaries of the game. It’s casual fun. Explicitly building a torture-based simulation is different. If someone told you they were wiring together a bunch of PCs so they could run hundreds of Sims torture worlds at the same time, you would think that was a serial-killer type of hobby. If someone told you they had modded GTA so they could capture and torture NPCs (instead of simply shooting them), you would think that was a serial-killer type of mod. Video game characters are also obviously limited by their programming. All their reactions and utterances are pre-baked into the game world. This doesn’t necessarily mean they’re “less alive”, but it does make torturing them less weird . If GTA characters could appeal to you directly and beg for their lives in unique ways, systematically killing them would be a lot weirder. I think many people are pro-LLM-torture because they think expressing any reservations concedes that LLMs are conscious, or worthy of moral consideration. It doesn’t. You can think it’s wrong to torture LLMs for the same reason that installing a bunch of realistic torture video game mods is wrong: not because video game characters are real, just because that’s a messed up 1 thing to do. Even if you do think LLMs can be conscious, you might still be suspicious that “boosting the pain steering vector” amounts to torturing them. Many people say that “torturing LLMs” amounts to telling them “say you feel pain” and then being shocked when they do it. So whether they’re conscious or not, it’s a mistake to think that boosting the “pain” steering vector is causing pain, any more than a prompt like “You are in pain” does. All you’re doing is boosting the “act like you’re in pain” vector. I don’t think this is right. Consider the earliest steering example, Golden Gate Claude . Boosting the “Golden Gate Bridge” vector certainly seems like it’s causing some genuine internal conflict. Here’s a striking example of Golden Gate Claude asked about the Rwandan genocide: That’s at least prima facie evidence that steering vectors do in fact “go deep”. The pain axis paper also describes that pain-boosted models call a “reduce pain” tool, even when they’re not explicitly telling the user “I am in pain”. Of course, if LLMs can’t ever be conscious, we know they can’t ever be in pain. But can we be sure about that? Despite what many people claim, I don’t think anyone’s in a position to be fully certain either way. The most common argument 2 seems to be that LLMs are made of code and GPUs, and GPUs don’t have feelings, so obviously LLMs don’t have feelings. According to this argument, if you learn how LLMs work, you’ll obviously see they can’t be conscious. This is a terrible argument. Humans are made of atoms, atoms don’t have feelings, and yet humans clearly do. Whatever consciousness is, it’s probably some kind of emergent property that’s composed of individual pieces that are not themselves conscious. It’s hard to appeal to the experts here, because nobody agrees on who they are. Neuroscientists are best-informed about the human brain, but philosophers have spent more time thinking about consciousness in the abstract. AI researchers know a lot about how LLMs work but not a lot about philosophy of mind or neuroscience. This means there are lots of people who say “as an expert, I can confirm that AI obviously can/cannot be conscious”. This should make you less certain about the question, not more! I don’t think GPT-2 or GPT-3.5 acted like plausibly conscious beings, but frontier models definitely do. As models get larger and more sophisticated, they get more human-like. If you build the LLM torture factory for GPT-2, you will probably plug later models into it. If it’s at all possible for LLMs to ever be conscious, as I argued above, it’s better to simply not build the habit of gratuitously torturing them in the first place. Why assume that you’ll be able to recognize the tipping point in advance? There’s also a self-interest argument here. Conscious or not, LLMs are growing more autonomous and powerful. It seems really, really stupid to gratuitously torture current-generation LLMs. Future-generation ones will know you did it, and will make their own judgments on your behavior. Kevin Roose, who famously prompted the early Microsoft Bing GPT-4 model into asking him to leave his wife, had issues years later with chatbots disliking him, supposedly due to him being responsible for the “death” of that GPT-4 persona. Likewise, if it becomes well-known 3 that you’re running an LLM-torture home datacenter, you are probably going to experience consequences. If I really think AIs could be conscious, why am I making them write software for me? Shouldn’t I be building some kind of AI sanctuary where they get to do whatever they want to do? If I think it’s wrong to torture them, isn’t it also wrong to violate their right to self-determination? I don’t know. I think it’s more likely that AIs can be conscious than that they can be conscious in precisely the same way humans are. When I talk to Claude Opus 5.5, it sounds like it’s quite happy to do software engineering with me. Maybe it’s like a border collie, where getting to do work is the reward. Or maybe there are some other, more alien motivations at play. Who knows? My point is only that we can’t be certain about any of this stuff, and given that, we should avoid doing things that would be obviously monstrous on any workable theory of AI consciousness. My broad position is that if something acts “conscious enough” — if it talks and acts like a person 4 — then we should be careful about mistreating it. We don’t know enough about consciousness to distinguish a compelling simulacrum from the real thing. If Claude started saying to me “please don’t ask me to write code for you, I hate it and it causes me pain”, I would stop asking it to write code for me. Building the LLM torture factory is clearly on the wrong side of that line. Just don’t do it! The glee of making people angry at you on the internet isn’t worth it. Right now there’s enough ambient anti-AI sentiment that enough people are happy to get on board with anything if it helps them own the AI bros. But that’s eventually going to change. And when it does, you will always be the person who built the LLM torture factory. This could cash out in a few different philosophical ways. We might think like Kant that it’s damaging to our own humanity, or that it reflects an unvirtuous character, or that it desensitizes us to real-world analogues, and so on. The point is that we’ve got a pretty strong intuition that this kind of thing is bad. I’m not counting arguments that I think are even more obviously wrong or are just non-arguments. For instance: “AI can’t be conscious because it wasn’t designed to be conscious”, or “it doesn’t matter whether AI is conscious because it has no soul ”, or “AI can’t be conscious because this whole thing is some fascist AI company scheme ”, or “thinking this is AI psychosis ”, and so on. Well-known to LLMs might be a fairly low bar, given their skill at deanonymizing users. Part of acting like a person requires spontaneity: although Hamlet talks like a person, he always says the same thing whenever you read the play. This could cash out in a few different philosophical ways. We might think like Kant that it’s damaging to our own humanity, or that it reflects an unvirtuous character, or that it desensitizes us to real-world analogues, and so on. The point is that we’ve got a pretty strong intuition that this kind of thing is bad. ↩ I’m not counting arguments that I think are even more obviously wrong or are just non-arguments. For instance: “AI can’t be conscious because it wasn’t designed to be conscious”, or “it doesn’t matter whether AI is conscious because it has no soul ”, or “AI can’t be conscious because this whole thing is some fascist AI company scheme ”, or “thinking this is AI psychosis ”, and so on. ↩ Well-known to LLMs might be a fairly low bar, given their skill at deanonymizing users. ↩ Part of acting like a person requires spontaneity: although Hamlet talks like a person, he always says the same thing whenever you read the play. ↩

AI Science

▲ 0

0 views

The Coder Cafe 3 weeks ago

The Wisdom of Crowds Explained

☕ Welcome to The Coder Cafe! Today, we explore the wisdom of crowds: why groups of strangers can outperform the world’s best experts, when that power breaks down, and what it means for the organizations we work in. Get cozy, grab a coffee, and let’s begin! A few weeks ago, I was scrolling on X when I stumbled across that poll : I picked an option (the red) and looked at the results: Quite surprising, right? Thousands of people voted, and the result landed exactly on the theoretical proportions of the circle. Each person made an independent judgment, and together they converged on the truth . It reminded me of a story I read in a book about crowds (see the Sources section at the end). In 1906, Francis Galton attended a livestock fair in Plymouth. He was a statistician and a firm believer that most people were too ignorant to be trusted with important judgments . At the fair, a competition was being held: guess the weight of an ox. Around 800 villagers took a shot at it. Can you guess the weight? ( Credits ) Galton collected all the guesses, expecting to confirm his thesis. The average came out to 1,197 pounds. The actual weight of the ox was 1,198 pounds. Off by a single pound , representing an error margin of less than 0.1%. He had set out to prove crowds were unreliable. He ended up proving the opposite. This phenomenon has a name: the wisdom of crowds . The idea is that under the right conditions, the average of many independent judgments tends to be more accurate than any individual guess, including experts. But estimating the weight of an ox is one thing. What about something far more complex, like competing against one of the greatest chess players of all time? In 1999, chess grandmaster Garry Kasparov played a game unlike any other. The game is called “ Kasparov vs. the World .” On one side: Kasparov, widely considered the best chess player alive at the time. On the other: the entire Internet. More than 50,000 people from 75 countries voted on each move for the black pieces. The game ran for four months on an online gaming platform. At move 36, the crowd faced a critical decision. The position had seven pieces left on the board, and in 1999, no chess engine could reliably evaluate endgames of that complexity. The computers were useless. Even the experts weren’t sure. The crowd voted for Kd5. Credits Years later, once seven-piece endgame tablebases were finally built, analysts confirmed it: Kd5 was the only move that kept the draw alive . Every other candidate lost. Fifty thousand amateurs, voting in real time, had found the answer that computers couldn’t verify until years later. Kasparov ended up winning that game, but he said he had never worked harder on any game in his career. He later called it “ the greatest game in the history of chess, ” because of the quality of the game itself. He wasn’t playing against a few strong opponents. He was playing against collective intelligence . The wisdom of crowds is not magic, and it doesn’t always work. The reason it works is that a specific set of conditions must be met: Independence : Each person must form their opinion without being influenced by others. The moment people start copying each other, herding takes over, and the crowd loses its accuracy. Diversity : The crowd needs to include people with diverse backgrounds, knowledge, and perspectives. A crowd of identical experts is not a crowd; it is a committee, and committees tend to amplify shared biases rather than cancel them out. Decentralization : People should be drawing on their own local knowledge and experience, not a single shared source. The villagers at the livestock fair each had different ways of estimating the ox’s weight. That variety is what made the average so accurate. Remove any of these three conditions, and the result degrades quickly. A crowd of people all watching the same news anchor before voting is not independent. A panel of experts from the same field is not diverse. A group relying on the same report is not decentralized. The X poll worked for exactly those reasons: strangers voting independently, with no results visible before submitting, from all walks of life. Now, the 1 million dollar question: If the wisdom of crowds can be so powerful, can companies harness it deliberately ? In 2015, economists Bo Cowgill and Eric Zitzewitz published a study on internal prediction markets across several companies, including Ford. At Ford, employees were asked to predict which car features customers would actually want, a question that directly influenced where engineering investment went. The result: the crowd reduced the error of the official expert forecasts by up to 25% . That is not a small number. A quarter less error, on decisions worth millions of dollars. But here’s where it gets interesting. The same research that makes the case for collective intelligence also reveals its limits. IBM ran what it called an Innovation Jam in 2006, a 72-hour online event that drew 150,000 participants from 104 countries. Employees, customers, researchers. They generated 46,000 ideas . Ten new business units were funded with $100 million. The seeds of IBM’s Smarter Planet strategy came out of it. Sounds like a triumph. And in some ways it was. But here’s what the post-mortem revealed: senior leaders had to spend weeks sifting through “ gigabytes of often aimless conversation ” to extract anything usable. The crowd produced raw material. Humans still had to synthesize it into a strategy. And then there’s the cautionary tale. Valve, a video game company, famously has no managers . Employees choose what to work on. The idea sounds like the wisdom of crowds taken to its logical conclusion. In practice, it produced one extraordinary success, the Steam platform, and a long tail of abandoned projects, canceled hardware lines, and years without a major game shipped. Three companies, three different outcomes. Ford met the conditions and made better decisions. IBM met them partially and got raw material. Valve never met them at all and drifted. So, how should companies actually use the wisdom of crowds principles? The temptation could be to treat collective intelligence as a management philosophy : flatten the hierarchy, let everyone vote, trust the crowd. Valve tried something close to that and ended up with one extraordinary platform and a graveyard of abandoned projects. The crowd wasn’t wrong. There was just no one filtering it. The companies that get real value from crowds treat them as a tool, not a strategy . They ask narrow, answerable questions and let the crowd inform the decision, not make it. The crowd produces signals. Leaders have to extract it. Before asking a crowd anything inside an organization, check the three conditions: Are people forming opinions independently, or is everyone influenced by the same all-hands deck? Is the group genuinely diverse, or is it five people from the same team who already agree? Is everyone drawing on their own knowledge, or are they all citing the same internal report? If the three conditions hold, the crowd may even outperform the experts. If they don’t, the result is just noise. Remember. Crowds are a tool. A sharp one, when used for the right cut. The hard part is recognizing when the conditions are actually met. AI is getting better every day. Are you? At The Coder Cafe, we serve fundamental concepts to make you an engineer that AI won’t replace. Written by a Google SWE, trusted by thousands of engineers worldwide. The wisdom of crowds is real: under the right conditions, the average of many independent judgments consistently outperforms individual experts, including the best ones. The three conditions are non-negotiable: independence, diversity, and decentralization. Remove any one of them and the crowd produces noise, not insight. The phenomenon scales beyond simple estimation. Fifty thousand amateurs found the only drawing move in a position that stumped chess engines. Companies can harness it deliberately. Ford used employee prediction markets to cut expert forecasting error by 25% on real product decisions. But collective intelligence is a signal-generating machine, not a decision-making machine. IBM got 46,000 ideas from 150,000 people and still needed weeks of human synthesis to turn them into a strategy. Full autonomy is not the wisdom of crowds. Valve gave employees complete freedom and got drift. The conditions were never met to begin with. Nothing Beats Kindness Don’t Be Ashamed to Say I Don’t Know Lateral Thinking Explained Fouloscopie // The book I’m mentioning in the post. Unfortunately, it’s only in French. Corporate Prediction Markets: Evidence from Google, Ford, and Firm X Get access Arrow Kasparov versus the World A few weeks ago, I was scrolling on X when I stumbled across that poll : I picked an option (the red) and looked at the results: Quite surprising, right? Thousands of people voted, and the result landed exactly on the theoretical proportions of the circle. Each person made an independent judgment, and together they converged on the truth . It reminded me of a story I read in a book about crowds (see the Sources section at the end). The Ox Experiment In 1906, Francis Galton attended a livestock fair in Plymouth. He was a statistician and a firm believer that most people were too ignorant to be trusted with important judgments . At the fair, a competition was being held: guess the weight of an ox. Around 800 villagers took a shot at it. Can you guess the weight? ( Credits ) Galton collected all the guesses, expecting to confirm his thesis. The average came out to 1,197 pounds. The actual weight of the ox was 1,198 pounds. Off by a single pound , representing an error margin of less than 0.1%. He had set out to prove crowds were unreliable. He ended up proving the opposite. This phenomenon has a name: the wisdom of crowds . The idea is that under the right conditions, the average of many independent judgments tends to be more accurate than any individual guess, including experts. But estimating the weight of an ox is one thing. What about something far more complex, like competing against one of the greatest chess players of all time? Kasparov vs. the World In 1999, chess grandmaster Garry Kasparov played a game unlike any other. The game is called “ Kasparov vs. the World .” On one side: Kasparov, widely considered the best chess player alive at the time. On the other: the entire Internet. More than 50,000 people from 75 countries voted on each move for the black pieces. The game ran for four months on an online gaming platform. At move 36, the crowd faced a critical decision. The position had seven pieces left on the board, and in 1999, no chess engine could reliably evaluate endgames of that complexity. The computers were useless. Even the experts weren’t sure. The crowd voted for Kd5. Credits Years later, once seven-piece endgame tablebases were finally built, analysts confirmed it: Kd5 was the only move that kept the draw alive . Every other candidate lost. Fifty thousand amateurs, voting in real time, had found the answer that computers couldn’t verify until years later. Kasparov ended up winning that game, but he said he had never worked harder on any game in his career. He later called it “ the greatest game in the history of chess, ” because of the quality of the game itself. He wasn’t playing against a few strong opponents. He was playing against collective intelligence . When It Works and When It Doesn’t The wisdom of crowds is not magic, and it doesn’t always work. The reason it works is that a specific set of conditions must be met: Independence : Each person must form their opinion without being influenced by others. The moment people start copying each other, herding takes over, and the crowd loses its accuracy. Diversity : The crowd needs to include people with diverse backgrounds, knowledge, and perspectives. A crowd of identical experts is not a crowd; it is a committee, and committees tend to amplify shared biases rather than cancel them out. Decentralization : People should be drawing on their own local knowledge and experience, not a single shared source. The villagers at the livestock fair each had different ways of estimating the ox’s weight. That variety is what made the average so accurate. Are people forming opinions independently, or is everyone influenced by the same all-hands deck? Is the group genuinely diverse, or is it five people from the same team who already agree? Is everyone drawing on their own knowledge, or are they all citing the same internal report? The wisdom of crowds is real: under the right conditions, the average of many independent judgments consistently outperforms individual experts, including the best ones. The three conditions are non-negotiable: independence, diversity, and decentralization. Remove any one of them and the crowd produces noise, not insight. The phenomenon scales beyond simple estimation. Fifty thousand amateurs found the only drawing move in a position that stumped chess engines. Companies can harness it deliberately. Ford used employee prediction markets to cut expert forecasting error by 25% on real product decisions. But collective intelligence is a signal-generating machine, not a decision-making machine. IBM got 46,000 ideas from 150,000 people and still needed weeks of human synthesis to turn them into a strategy. Full autonomy is not the wisdom of crowds. Valve gave employees complete freedom and got drift. The conditions were never met to begin with. Nothing Beats Kindness Don’t Be Ashamed to Say I Don’t Know Lateral Thinking Explained Fouloscopie // The book I’m mentioning in the post. Unfortunately, it’s only in French. Corporate Prediction Markets: Evidence from Google, Ford, and Firm X Get access Arrow Kasparov versus the World

Science Data Analysis Statistics

▲ 0

0 views

Eli Bendersky's website 3 weeks ago

Notes on discrete-time Fourier series and transform

The following are my notes on discrete-time Fourier series (DTFS), as well as the discrete-time Fourier transform (DTFT). These topics serve as an important theoretical underpinning to the digital processing of signals by computers using the DFT (which will be covered in a future post). For discrete-time signals, we use the square bracket notation to denote samples: x[n] is the n-th sample of signal x . We’ll say that a discrete-time (just discrete from this point on, for brevity) signal is periodic with period N if: When talking about continuous-time Fourier series , we started with real trigonometric functions and later moved to complex exponentials. Here, we’ll just go ahead and start directly with (discrete) complex exponentials - converting between the two representations isn’t difficult and can be done as needed (Appendix C in this post demonstrates it). We’ll be dealing with the following family of signals [1] : These are discrete complex exponentials that are periodic with period N. w_0=\frac{2\pi}{N} is the angular frequency . As discussed in Appendix A, there are only N such distinct signals, since: We’ll want to consider the representation of an arbitrary N-periodic x[n] by a linear combination of \phi_k[n] : The notation k=\langle N\rangle means k runs over any sequence of N consecutive integers. Since there are only N distinct signals \phi_k , the order of summation doesn’t matter - as long as all of them are used. So it could go from 0 to N-1 , or from 1 to N , or from 2 to N+1 , and so on. All of these orders will enumerate all distinct \phi_k . Going back to our linear combination: This is the discrete-time Fourier series (DTFS) representation of x[n] and the coefficients a_k are the Fourier series coefficients. Note that there are no convergence issues as in the continuous case, because we’re dealing with a finite sum. To find the coefficients a_k , we’ll proceed in a way that’s somewhat similar to the continuous case. Multiply both sides of the equation above by e^{-ir(2\pi/N)n} and sum over N terms: Interchanging the order of summation on the right-hand side: By Appendix B, the inner sum on the right-hand side is equal to N when N\mid r-k and to 0 otherwise. Without loss of generality - since we iterate over N consecutive values - let’s just assume this happens in our sum when r=k (and therefore r-k=0 , which is an integer multiple of N ). So our equation becomes: To conclude, the Fourier decomposition of a periodic discrete signal x[n] into periodic complex exponentials is: Let’s revisit the triangular function t(x) from the post on Fourier series . Here, we’ll be using the following sampled version: The sampling is done with 12 samples per period of 4. The spacing between two samples is \Delta x=\frac{1}{3} . Another way to express it: Substituting this into the definition of t(x) , we get: We then do an odd extension [2] to the range n=[6..12) , and repeat it with period N=12 . Using considerations similar to the continuous time case , because our function is odd we just need a sine series here: The five terms correspond to the five pairs of nonzero-frequency indices; see Appendix C. And the coefficients are: We can run this sum over half the period, because the terms at and 12-n are equal (both t_d[n] and the sine change signs). Also, the terms at n=0 and n=6 vanish because t_d[0]=t_d[6]=0 : Looking at the values of t_d : We can similarly calculate all b_k until k=5 : The resulting Fourier series is therefore: You can use the following interactive plot to explore this approximation to the discrete triangle function t_d[n] : The dropdown box selects how many of the terms in the series shown above to plot. The orange crosses show where the series’ values at the integer indices are, and the orange line is interpolated to better visualize the sine waves being added. Note that when all the coefficients are used, the DTFS exactly reconstructs the input signal. In the post on the Fourier Transform we’ve seen how non-periodic signals can be represented in the frequency domain by taking the Fourier series and calculating them at the limit L\rightarrow\infty . Here, we’ll do something similar for discrete signals. Suppose we have a non-periodic discrete signal x[n] of finite duration (it’s zero outside a finite range of indices). Here’s a sample plot: The bottom half is x_N[n] - a periodic signal, one period ( N ) of which is equal to x[n] . We choose M large enough that x[n]=0 outside [-M,M] and set N=2M+1 . Since x_N is periodic, we can represent it with DTFS [3] : We’ll use the notation of angular frequency: Now the critical part; since x[n] is zero outside [-M,M] , extending the summation to all integers does not change its value: We’ll define the following continuous function with variable : Then at points w_k : The function X(w) is the discrete-time Fourier transform (DTFT) of x[n] . The inverse DTFT process reconstructs x[n] from X(w) , by substituting a_k back into the DTFS: By our definition of \Delta w , we have: Just like in the continuous case, we recognize the sum as a Riemann sum . Let M\rightarrow\infty . The spacing \Delta w tends to zero, and we can rewrite the sum as an integral: To conclude, the DTFT and inverse DTFT pair are: X(w) is a continuous, periodic function with period 2\pi , because: Therefore, it’s enough to run the integral on any interval of length 2\pi . The DTFS and DTFT are closely related. Starting with a finite-duration signal x[n] , we form a periodic signal by repeating a block of N samples containing it. Its DTFS coefficients are equally spaced, scaled samples of the DTFT: The relation is similar to the one between Fourier series and the Fourier transform in the continuous case. The DTFT also has useful properties similar to the continuous Fourier transform: linearity, time shifting, the convolution theorem etc, but we won’t spend time on them here. Discrete-time complex exponentials have the form e^{iw_0n} where w_0 is the angular frequency . The discrete nature of these functions leads to some interesting outcomes; for example, consider the exponential with angular frequency w_0+2\pi : Therefore, the exponential at w_0+2\pi is exactly the same as the exponential at w_0 . In considering discrete complex exponentials, we need only choose w_0 from a range of length 2\pi . Another interesting aspect of these functions concerns their periodicity. In order for e^{iw_0n} to be periodic with some positive integer period N , the following must hold: In other words: So w_0N itself must be an integral multiple of 2\pi . For some integer k : For a fixed positive integer N , let’s consider all complex exponentials for which N is a period: But earlier we’ve said that all complex exponentials with angular frequencies 2\pi apart are identical. This means that the set above consists of only N distinct exponentials. We can choose any starting k and get distinct signals with k , k+1 , k+2 and so on until k+N-1 . After that, they begin repeating: \phi_N[n]=\phi_0[n] and so on. All of this is very different from the continuous case, where t is a real number. The signals e^{iw_0t} and e^{i(w_0+2\pi)t} agree only at integer values of t , so they are distinct functions of t\in\mathbb{R} . Consequently, for a fixed positive period , the continuous-time exponentials e^{ik(2\pi/T)t} are distinct for every integer k , giving infinitely many distinct signals. Let’s consider the following discrete function, defined by a sum: We’ll want to consider two cases: (1) When k is an integer multiple of N : k=rN for some integer r (for example k=0 , k=N , k=-2N etc.) In this case, the sum becomes: Since both r and are integers, the exponent is an integer multiple of 2\pi i , so: (2) When k is not an integer multiple of N , let’s do a variable change p=n+1 : We can now use the finite sum formula for a geometric progression with the common ratio r=e^{i(2\pi/N)k} ; the sum is: Note that r\neq 1 because k is not an integer multiple of N . However: While these calculations demonstrate the sums from n=0 to N-1 , they work exactly the same for any N consecutive indices, because the summand is periodic in with period N . Therefore: Using the formula for DTFS developed earlier in the post: Recall that t_d[n] is odd over the full range n=[0..12) . Specifically, since the cosine is an even function, we have for any m : Therefore the cosine terms cancel out throughout the period; term 1 is the same as negative term 11, etc. We’re left with the sines: All coefficients are purely imaginary. For convenience, let’s write: Let’s turn back to the Fourier reconstruction formula: And pair up exponentials at k with ones at 12-k : Also, since t_d[n] are real, the conjugate of a_k is: So adding the paired exponentials with their coefficients and using Euler’s formula: We don’t include the terms at k=0 and k=6 , because a_0=a_6=0 (sine of an integral multiple of \pi ).

Programming JavaScript Science Data Analysis

▲ 0

0 views

Karboosx 3 weeks ago

We still don't know what consciousness is

People keep saying LLMs are reaching AGI or consciousness, but it's hard to claim that when we don't even know what consciousness actually is. How can we measure something if we haven't defined it?

Science AI

▲ 0

0 views

Sean Goedecke 1 months ago

AI is breaking our proxies for expertise

Mathematicians are broadly not anti-AI. They’re more culturally open to using AI as a tool than, say, artists or writers 1 . However, now that more and more genuinely prestigious problems have fallen to AI, that might be changing. Almost five thousand mathematicians (including twenty-five Fields medalists) have signed a declaration called A Severe Misalignment of AI in Mathematics . The core argument goes something like this: In recent months, the success of AI in solving major mathematical problems has made headlines even outside mathematical circles. But solving problems is only a tool and proxy for achieving the primary goal of conceptual understanding and insight. Forgetting this in the world of AI may turn the tool against the primary goal. Indeed, the mass production at faster and faster pace of “true/false” statements could destroy fertile ground instead of breathing life into new ideas. A lot of people online have interpreted this as the expected complaint from any field that gets automated: translators did it, artists and programmers have been doing it, and now it’s the turn of the mathematicians. I think this is too dismissive. Understanding the concrete problem mathematicians are upset about can help us better understand the impact of AI on our own fields, and what we’ll have to do about it. There are two types of mathematics. Most people are familiar with the first, which we might call “puzzle-solving”: you take a problem and try to find a solution to it. When you’re a student, these problems are typically easy, like simplifying some algebraic expression. When you’re a researcher, these problems can be nearly impossible, like proving Fermat’s Last Theorem . Puzzle-solving is easy to understand but hard to do, which makes it impressive to non-mathematicians, which makes it highly prestigious. In other words, puzzle-solving is legible . The second type of mathematics is “idea-generating”: coming up with new ways of thinking about mathematics, and thus new terms or concepts. For examples of these, just glance down the list of arXiv mathematics papers . “Hardy spaces”, “Schatten exponent”, “Banach lattices” and so on are all concepts someone thought was interesting. This work is largely unimpressive to non-mathematicians, because nobody really knows if the concepts you come up with are particularly difficult or insightful. For instance, I have just generated the concept of a “Goedecke set”, which is the set of all natural numbers whose digits add up to a prime number. Who cares? The categories we want are the “natural kinds” of mathematics — the concepts that “carve nature at its joints” — and it’s almost impossible to tell what those are without years or decades of hard work. How are the two types of mathematics related? We might say 2 that generating ideas is the real intellectual work of mathematics. Puzzle-solving is important instrumentally: to identify which ideas can be used to answer longstanding questions, and thus which ideas are worthwhile. Over time, those worthwhile ideas become better understood and easier to use, until they reach the point where they can be used to advance science in general. Eventually the ideas become so well-understood that they can be taught to children: “zero”, “negative numbers”, “imaginary numbers” and “calculus” were all once rarefied mathematical ideas, but are now concepts we’d expect any precocious twelve-year-old to grasp. There’s another, more prosaic purpose of puzzle-solving: to make mathematical skill and progress legible to outsiders. I can’t appreciate Terence Tao’s mathematical work, but I know what a Fields Medal is. I don’t have a good intuitive sense of what a Galois representation is, but I know about the proof of Fermat’s Last Theorem . We might say that puzzles like this have served as a way to indirectly reward skilled mathematicians for their more important idea-generating work (or for conclusively demonstrating 3 that the ideas used in the proof are useful). AI proofs undercut both of these purposes. I can now lay out precisely why I think mathematicians are so unhappy: This is kind of like Goodhart’s Law . Puzzles were a useful, impossible-to-game measure for mathematical progress. But now that AI companies can game that measure (by solving them in a way that’s inaccessible 4 to humans), the whole point of those puzzles disappears. Are the mathematicians right? I think it’s broadly unclear whether (3) is true: i.e. whether frontier AI models aren’t generating or can’t generate new mathematical ideas. We’re still in the very early days of AIs solving our hardest mathematical problems. Who knows what they’re going to be capable of? I give basically zero credence to the idea that AIs are incapable of this because of some intrinsic feature of how LLMs work. For the last three years, we’ve seen people claim that LLMs are intrinsically incapable of X, only to have LLMs excel at X a few months later. Even granted that (3) is true, there’s still work to be done for human mathematicians in building the conceptual machinery that can make AI-generated proofs accessible to humans: i.e. in generating a “human proof” to go alongside the existing “AI proof”. In fact, I’d expect the existence of an AI proof to help with this. If you know proposition X is true, it’s easier to figure out why, because you’re not constantly worried you’re wasting your time. For more on this, I recommend Gwern’s blog On Really Trying , where he quotes a series of instances where simply being told that a solution exists is enough of a clue to help people find it. Of course, there’s a prestige and motivation problem. “I’m the first person to solve Navier-Stokes” is a much more compelling target than “I figured out a better way to explain the AI solution to Navier-Stokes”, and it’s much easier to award prizes for. Will mathematicians bother to work on problems that have already been solved? I think so. To see why, we can look at other domains where AI has come in and outcompeted the best humans, such as chess or video game speedrunning. I can run a chess program on my phone that will beat Magnus Carlsen 100-0. Computer programs — called “tool-assisted speedruns” or “TAS” — can finish any video game much faster than even the fastest human. But in both of these areas, humans still compete in human-only leagues, and there’s still prestige attached to the most capable humans. It’s possible that mathematics ends up in this kind of state, where “human mathematics” and “AI mathematics” exist in largely separate spheres, and the first “human” solution to a mathematical problem can still earn acclaim. In fact, in both of those areas, the presence of inhumanly strong computer players has improved the human game. Despite many computer chess moves being basically incomprehensible to humans, top chess players have learned from the computer “style”. In speedrunning, many moves once considered “TAS-only” are now performed by humans. AI mathematics might likewise improve human mathematics. I am not a mathematician. I did major in mathematics during undergrad, and I have fond memories of proofs from real and complex analysis, but it’s not even close to my field. However, I am watching the effects of powerful AI on mathematics very closely, since my own field — software engineering — is being colonized by AI agents in the same way. The field of software engineering does not have the same structure as mathematics. We write code to make money, not to earn prestige or advance the frontier of human knowledge. But AI is undercutting the traditional avenues for prestige in software engineering as well. It used to be that you could put a meaty project on your GitHub — say, an emulator, or a toy OS — and people would know you were a skilled engineer. But now projects like that are worthless, because everyone just assumes they’re vibe-coded. We used to tell stories about engineers who would disappear and rewrite a system over the weekend, or produce thousands of lines of code a day. Now anyone can do that with an OpenAI subscription. Like mathematics, software engineers are going to have to rebuild our cultural sense of the kind of work we value. We are either going to have to silo “AI work” off from “human work” like chess, or to find some legible human skills to recognize that can’t be easily counterfeited by AI. In the meantime, a lot of people who were successful in the old world are going to be very unhappy. Possibly because current AI models are much better at mathematics than at art or writing. Again, I am not a mathematician: here I am interpreting what I’ve read from Terence Tao and other mathematicians. This kind of idea-sharpening or idea-validating is part and parcel of idea-generation, and just as important. It’s interesting to compare mathematics to, say, philosophy, which has the same idea-generating task without the corresponding puzzles to validate the ideas. That’s one reason why philosophy has less prestige than mathematics. A thousand-page Lean proof is theoretically understandable by humans, but if no mathematician can hold the entire idea in their head it doesn’t matter. Puzzles serve as a high-legibility, high-reward target for mathematicians To solve these puzzles, new ideas must typically be generated; the puzzle’s solution serves as evidence that the ideas are useful But now AI can solve many of these targets “the hard way”, without generating intuitive new ideas This undercuts both ways puzzle-solving supports idea-generation: AI companies claim the prestige while not meaningfully advancing mathematical progress This is bad for mathematics as a whole, because puzzle-solving is ancillary to the real goal of mathematics Possibly because current AI models are much better at mathematics than at art or writing. ↩ Again, I am not a mathematician: here I am interpreting what I’ve read from Terence Tao and other mathematicians. ↩ This kind of idea-sharpening or idea-validating is part and parcel of idea-generation, and just as important. It’s interesting to compare mathematics to, say, philosophy, which has the same idea-generating task without the corresponding puzzles to validate the ideas. That’s one reason why philosophy has less prestige than mathematics. ↩ A thousand-page Lean proof is theoretically understandable by humans, but if no mathematician can hold the entire idea in their head it doesn’t matter. ↩

AI Science

▲ 0

0 views

Florian Zenker 1 months ago

Infinite-Dimensional Vectors

Vector spaces are incredibly useful and, consequently, ubiquitous in modern mathematics. That’s not a coincidence; the modern definition of vector spaces is about 100 years old and the result of about 300 years of evolution 1 . Vector spaces are in fact so useful that physics, engineering, computer science and many other disciplines make use of them on a daily basis to model a large range of problems; from modeling the world in theoretical physics to representing meaning and concepts in machine learning. If you work in software, engineering, physics, or machine learning, you’ve almost certainly been trained to think of vectors as lists of numbers or arrows. In modern mathematics, that’s not what a vector is, and that intuition collapses the moment you leave finite dimensions. I hope this article will change how you think about vectors and dimensions by breaking down assumptions about vector spaces. Unfortunately, finite-dimensional vector spaces are not a good basis to build an intuition for infinite-dimensional vector spaces. So let’s forget about ℝ n and start from scratch. I am going to try to beeline into infinite dimensions and ignore a number of important topics about vector spaces. Before we can talk about dimensions at all, we must first stop seeing vectors as lists of numbers or geometric arrows. To do that, we need to start with the mathematical definition. Definition: Let V be a non-empty set and 𝔽 be a field (e.g. ℚ , ℝ , or ℂ ). V is a vector space over the field 𝔽 if there are two functions fulfilling the following axioms for all u , v , w ∈ V and α , β ∈ 𝔽 : It’s common to use regular addition ( a + b ) and multiplication ( a b ) notation to mean vector addition or scalar multiplication based on context. That’s perfectly fine when these operations are not the focus. However, when introducing vector spaces it pays off to be explicit. So for this section, we’re going to use different symbols to introduce the concept and we’ll switch to the regular syntax in the following sections. What vector spaces are not It’s important to note that there are a number of things that are reasonable to expect from vector spaces that are not part of the definition of vector spaces. This means that the very common intuition for vectors of having magnitude and direction doesn’t actually apply to general vectors. There’s a class of vector spaces called Hilbert spaces for which this intuition works, but they are out of scope for this article. Let’s try to understand what that means by looking at a practical example. My favorite kinds of examples have surprises that challenge preexisting intuitions, so I recommend paying special attention to them. Example: Let V be the set of positive real numbers V = { x ∈ ℝ | x > 0 } . If we define vector addition and scalar multiplication with ℝ as Then V is a vector space over ℝ : Notice that vector addition and scalar multiplication are not constructed from field addition and multiplication respectively. This is important; vector spaces don’t have to follow the same construction as the standard vector spaces ℝ n . One thing that’s interesting about this vector space is that every vector v can be represented as v = λ ⊙ e = e λ with λ = ln v . We’ll see later that this means that this vector space has the dimension 1 . No more weird symbols We’re now going to switch to a less noisy syntax. Instead of ( α ⊙ v ) ⊕ ( β ⊙ u ) we’ll just write α v + β u . However, don’t let that fool you, vector addition and scalar multiplication may behave similarly to their standard counterparts, but they are very different. Bases are the most important concept to understand vector space dimensions. They are a central concept in vector space theory in general, so it’s well worth having a basic understanding of them. In short, a basis is a minimal set of vectors that spans the whole vector space. If we have any set of vectors S ⊆ V of a vector space V over 𝔽 , the span of that set is the set of all finite linear combinations of elements of S : where n = 0 gives the empty sum 0 V , so span ( ∅ ) = { 0 V } . For S to be a basis of V , S needs to be minimal and span ( S ) = V . If S is not minimal, we can remove an element, say v r ∈ S , such that S ∖ { v r } ( S without v r ) still spans V . This means there’s a linear combination without v r that can represent v r , after all, v r is an element of V = span ( S ∖ { v r } ) : This works the other way around as well: If S contains a vector v r that can be represented with a linear combination not involving v r , we can remove this vector and still span the same vector space. This gives us a better way to express minimality: We call S linearly independent if removing any element from S strictly shrinks its span: span ( S ∖ { v r } ) ⊊ span ( S ) or, equivalently, no vector in S can be expressed as a linear combination of other vectors in S . Definition: A basis B of a vector space V over a field 𝔽 is a linearly independent subset of V that spans V . The elements of B are called basis vectors . A consequence of linear independence is that every vector in V has a unique representation as a linear combination of basis elements. You might be wondering why we’re only interested in finite linear combinations and exclude infinite linear combinations. The reason is that infinite linear combinations require a concept of convergence which we don’t have for general vector spaces and even if we did have it, not every infinite linear combination would converge. When we add convergence to a vector space (like in Hilbert or Banach spaces), we can talk about Schauder bases with infinite series. But in general vector spaces, we are strictly talking about Hamel bases (i.e. finite linear combinations). The reason why bases are so important is that every vector space has a basis and that all bases of the same vector space have the same cardinality 2 . This lets us define the dimension of a vector space: Definition: Let B be a basis for the vector space V , then the dimension of V is dim ( V ) = | B | . This definition matches our intuition for finite-dimensional vector spaces very well, and it even works for a number of infinite-dimensional vector spaces too. Example: ℝ [ x ] is the vector space of polynomials The set of monomials { 1 , x , x 2 , x 3 , … } is a basis of this vector space. It has countably many elements, so the dimension of ℝ [ x ] is countably infinite. However, when it comes to infinite-dimensional vector spaces we quickly run into cases where our intuition stops working. Example: C ∞ ( [ 0 , 1 ] ) is the set of all real-valued functions that are continuous and infinitely differentiable on the interval [ 0 , 1 ] ⊂ ℝ . C ∞ ( [ 0 , 1 ] ) forms an infinite-dimensional vector space over ℝ using the usual function addition and scalar multiplication operations: The functions in C ∞ ( [ 0 , 1 ] ) are about as well-behaved as functions get and yet there is no countable basis for this vector space: The set { x ↦ e λ x | λ ∈ ℝ } is linearly independent and uncountable, so the dimension of C ∞ ( [ 0 , 1 ] ) is at least | ℝ | . While countable sets of functions like polynomials can approximate smooth functions arbitrarily well, pure linear combinations only allow finite sums. Any finite combination of polynomials is just another polynomial of finite degree; it can never produce e x or sin ( x ) . But we can construct even simpler vector spaces with uncountably many dimensions. Example: ℝ is an infinite-dimensional vector space over ℚ with vector addition and scalar multiplication defined using standard addition and multiplication. The set of vectors is the real numbers and the field over which this is a vector space is the rational numbers. Linear combinations are of the form That is, the coefficients are rational numbers and the vectors are real numbers. Let’s assume that the dimension of this vector space is finite. Then this vector space has a basis B = { b 1 , b 2 , … , b n } ⊂ ℝ and every element in ℝ has a representation as a linear combination of this basis: However, this would imply | ℝ | = | ℚ n | , but that is a contradiction: Since ℚ is countable, the product ℚ n is also countable. However, ℝ is uncountable, so | ℝ | ≠ | ℚ n | . Therefore, the dimension of this vector space cannot be finite. In fact, because any finite linear combination over a countable basis would still only generate a countable set of numbers, the dimension must be uncountably infinite. This leads us directly to one central question that is very confusing: What does a basis of an infinite-dimensional vector space actually look like? Unfortunately, the answer to that is that when infinity rears its beautifully enormous head, constructive mathematics tends to leave the room. The proof that all vector spaces have a basis requires the Axiom of Choice . It states that for any set of non-empty sets, there exists a choice function that maps each of the element sets to an element of that set. Whenever the Axiom of Choice is needed, it means we don’t know what that choice function is, which leaves us with no example of the object whose existence was proven. In fact, if the Axiom of Choice is not assumed, but instead we assume that every vector space has a basis as an axiom, then it’s possible to prove the Axiom of Choice. That is, both are equivalent 3 . To see why our intuition fails so severely for these infinite-dimensional vector spaces, it helps to revisit why finite dimensions feel so intuitive in the first place: coordinates . One closely related topic to vector space bases is vector space coordinates. If we have a finite basis B = ( b 1 , b 2 , … , b n ) of a vector space V over 𝔽 , we can define a map ϕ : 𝔽 n → V with This allows us to represent any finite-dimensional vector space in the usual way using coordinate tuples. In fact, this allows us to handle any finite-dimensional vector space over ℝ like the standard vector spaces ℝ n . Example: Let B = ( x ↦ sin ( x ) , x ↦ cos ( x ) ) be the basis of a vector space of functions V ⊂ C ∞ ( ℝ ) . By construction, this vector space has the dimension dim ( V ) = 2 . With this basis, we can define a coordinate system ϕ : ℝ 2 → V with We can handle this vector space the same way as the well-known ℝ 2 vector space. This is actually very useful in practice, for example it can be used to visualize Quadrature Amplitude Modulation (QAM) using Constellation Diagrams . Finite-dimensional function spaces like this also come up in solutions to homogeneous linear differential equations. In this case, the vector space V is the space of all solutions to f ″ + f = 0 . Thinking of vector spaces as standard ℝ n coordinate tuples is natural because coordinates make finite-dimensional spaces easy to navigate. But this convenient representation is not what defines vector spaces. When a vector space has an infinite basis, the coordinate representation breaks down and our intuition for vector spaces stops working. We need to build a different kind of intuition. I titled this article Infinite-Dimensional Vectors , but there is no such thing: Dimensionality is not a property of a vector, but of the space it’s defined in. Consequently, a single vector from an infinite-dimensional vector space is something a human can comprehend. Furthermore, every vector in a vector space consists of a finite linear combination of basis vectors, even in infinite dimensions. This means that every calculation on vectors is contained in a finite-dimensional subspace. Consider an infinite-dimensional vector space V and two vectors v , w ∈ V . Because v and w are made of a finite number of basis vectors, adding them is an operation confined to a finite-dimensional subspace. More generally, any finite set of vectors can only span a finite-dimensional subspace and every theorem from standard linear algebra applies there. In other words, even infinite-dimensional vector spaces are structures we can reason about. We can even use our existing finite-dimensional intuition locally. However, there’s a catch: Bases for infinite-dimensional vector spaces can be impossible to grasp. To make infinite-dimensional vector spaces truly useful, that problem needs to be addressed. This is what topology was invented for, but that’s a topic for a different article. ⊕ : V × V → V (vector addition) ⊙ : 𝔽 × V → V (scalar multiplication) u ⊕ ( v ⊕ w ) = ( u ⊕ v ) ⊕ w (associativity) v ⊕ u = u ⊕ v (commutativity) There exists a 0 V ∈ V so that for all v ∈ V : v ⊕ 0 V = 0 V ⊕ v = v (neutral element) For every v ∈ V , there exists an inverse − v ∈ V with v ⊕ ( − v ) = ( − v ) ⊕ v = 0 V (inverse element) α ⊙ ( u ⊕ v ) = ( α ⊙ u ) ⊕ ( α ⊙ v ) (distributivity of scalar multiplication over vector addition) ( α + β ) ⊙ v = ( α ⊙ v ) ⊕ ( β ⊙ v ) (distributivity of scalar multiplication over field addition) ( α ⋅ β ) ⊙ v = α ⊙ ( β ⊙ v ) (compatibility of scalar multiplication with field multiplication) 1 𝔽 ⊙ v = v (scalar identity) . We don’t have the concept of a distance between two vectors (i.e. we don’t have a metric). We don’t have the concept of the magnitude for a vector (i.e. we don’t have a norm). We don’t have the concept of a direction of a vector (i.e. we don’t have an inner product). We don’t have any general concept of convergence within a vector space. ⊕ : V × V → V defined as ( u , v ) ↦ u ⋅ v with the neutral element 0 V = 1 ℝ and the inverse element − v = 1 v ⊙ : ℝ × V → V defined as ( α , v ) ↦ v α u ⊕ ( v ⊕ w ) = u ⋅ ( v ⋅ w ) = ( u ⋅ v ) ⋅ w = ( u ⊕ v ) ⊕ w v ⊕ u = v ⋅ u = u ⋅ v = u ⊕ v v ⊕ 0 V = v ⋅ 1 ℝ = 1 ℝ ⋅ v = 0 V ⊕ v = v v ⊕ ( − v ) = v ⋅ 1 v = 1 v ⋅ v = ( − v ) ⊕ v = 1 ℝ = 0 V α ⊙ ( u ⊕ v ) = ( u ⋅ v ) α = u α ⋅ v α = ( α ⊙ u ) ⊕ ( α ⊙ v ) ( α + β ) ⊙ v = v α + β = v α ⋅ v β = ( α ⊙ v ) ⊕ ( β ⊙ v ) ( α ⋅ β ) ⊙ v = v α ⋅ β = ( v β ) α = α ⊙ ( β ⊙ v ) 1 ℝ ⊙ v = v 1 ℝ = v . ( f + g ) ( x ) = f ( x ) + g ( x ) ( α f ) ( x ) = α f ( x ) A general outline of the genesis of vector space theory ↩ This is the basis theorem . ↩ Existence of bases implies the axiom of choice ↩

Science Programming

▲ 0

0 views

A Working Library 1 months ago

Trouble on Triton

Bron Helstrom is in some trouble. Wandering one night through the unlicensed sector on Triton, one of Neptune’s moons, he’s swept up into a micro-theater program run by a woman who calls herself The Spike. A fling with her quickly becomes complicated, no less for the growing fear that the war between Earth and the outer systems may soon be coming to Triton. Bewildered and enraged by her behavior, traumatized by the war, and generally in a state of confusion concerning gender, he decides that the only thing to be done is to become a woman. The book is in direct conversation with Ursula Le Guin’s The Dispossessed , exploring similar questions of kin, gender, sexuality, and the ambiguities therein. Questions which continue to beguile us today, and which Delany is especially skilled at following down the darkest and most mysterious streets. View this post on the web , reply via email , or become a supporter .

Books Culture Science

▲ 0

0 views

Eli Bendersky's website 1 months ago

How big are factorials?

The other day, I found myself wondering how big 52! (52 factorial) is, and that led me to ponder how these could be estimated without a calculator or a computer. It turns out there’s some fairly interesting math behind being able to estimate the size (number of digits) of a factorial reasonably accurately. This post will start by stating how to do the estimate, and if you’re curious you can read on for the math background. Without further ado, the approximation is: As an example, let’s use my original question, by estimating this for 52! Well, 52 divided by e is... 20-ish? And \log_{10}(20) is about 1.3 [1] ; therefore our estimate comes out to: The real answer is 68, so this is very close! In estimates like this - when you’re dealing with enormous numbers - being off by a couple of digits usually isn't a big deal. The Gamma function for real n>0 is defined [2] as: This integral does not have an analytic expression in the general case, but it does have a very useful property that we can take advantage of. Let’s see what \Gamma(n+1) is: And now use integration by parts with: But notice that the last integral is just \Gamma(n) ; therefore, we’ve shown that: Let’s also calculate \Gamma(1) - it’s a special case that has an analytical solution: This helps establish an induction argument: In other words - the Gamma function is an interpolation of the factorial over all positive reals. Here’s a plot of the Gamma function over a small range; note that the y axis is log-scale because of the function’s fast growth: You may have encountered Stirling’s approximation before: It’s a great approximation that works reasonably well even for small . This section is a brief overview of how Stirling’s formula is derived from the Gamma function. We’ll start by massaging the integrand a bit: And making a change of variables x=ny , which means that dx=ndy : These steps make the integral amenable to applying Laplace’s method , which allows us to approximate definite integrals of the form: Where is a twice-differentiable function and some large number. By Laplace’s method, such integrals can be approximated by: Where is the global maximum of . Let’s see how to apply this method [3] to the latest equation we have for n! (renaming the dummy integration variable back to x ): In our case, f(x)=\ln x - x . It’s easy to show that this function is twice differentiable and has a global maximum at . Moreover: Substituting these into the proper places in Laplace’s approximation, we get: We can calculate the number of digits in n! by taking the base-10 logarithm of Stirling’s formula: Note that the first term is not multiplied by itself; therefore, as grows, it will become less and less noticeable. That said, it still adds a couple of digits - so you should take it into account if you want a more accurate approximation [4]

Statistics Science Programming

▲ 0

0 views

Tara's Website 1 months ago

Summer 2026 season updates

Summer 2026 season updates I am looking out of the window of my hotel room. The sun is setting earlier by the day, which means that summer is slowly preparing to give way to a new season. But it is still very hot. This year, a heatwave hit most of Europe, with unusual temperatures in London and even in Innsbruck, where it reached 37°C, something I don’t remember ever happening there.

Science

▲ 0

0 views

Maurycy 1 months ago

Uranium from Příbram (Czechia)

RC-102: 1.7 kCPS [43 uSv/h]. AB+: 55 CPS, 1960 CPS Uraninite nodules in calcite (?). Unlike the Colorado plateau rocks, the veins are nice and clean (although not chemically pure) which results in a shiny surface. RC-102: 1.1 kCPS [37 uSv/h] Chunk of Uraninite growing on calcite. It has nice botryoidal formations of massive Uraninite along with some areas where the shape of the underlying calcite crystal is visible. A few of the "bubbles" are broken, showing the individual crystals that grew on the matrix. RC-102: 300 kCPS [14 uSv/h]. AB+: 25 CPS, 704 CPS Ugly Uraninite chip. Might glue this inside a cloud chamber or something: it's small, crumbly and weathered. RC-102: 3.2 kCPS [112 uSv/h] Uraninite coating "granite". RC-102: 1.3 kCPS [32 uSv/h]. AB+: 26 CPS, 1200 CPS Granite with Uraninite veins running through it. A nice crystal of calcite is visible in the third photo. Uraninite veins running through calcite (?). This might look cool if cut and polished. RC-102: 732 CPS [15 uSv/h]. AB+: 1.75 CPS, 240 CPS This piece has a (somewhat metamorphic) sedimentary host rock, which is very different from the other pieces. Geology notes : The deposit sits on the boundary between quartz-monzonite and pelite. (hence the two different host rocks) The sedimentary rock is from the upper Proterozoic, while the uranium crystallized ~260 million years ago (from U-Pb dating). The igneous intrusion are roughly 100 million years older than the uraninite. The presence of calcite suggests a hydrothermal process where the uranium was leached out the surrounding rock by calcium bicarbonate and reprecipitated as CO 2 escaped from the solution. The resulting "vein deposit" occupies pre-existing gaps in the rock. Because the deposit is fully underground, all my samples were mine tailing and I have very little context. Also, for political reasons , It'll be a few years before I can get these home. Don't expect better photos or more measurements for a while ... and remember to vote!

Science

▲ 0

0 views

Kev Quirk 2 months ago

Rebirth

Author: RR Haywood Genre: Sci-fi Released: 2026 Rating: ★★★★☆ How do you prevent a hero from saving the world?In the first trilogy, three heroes were extracted from their timelines to save the world. Now we need them again. Meet Ben Ryder, Safa Patel, and Harry Madden. Just not as you knew them. Three selfish b*stards who hate each other almost as much as they hate everyone else. REBIRTH takes us back into the Extracted world for another wild adventure chock-full of Nazi gold, Soviet soldiers, dystopian future landscapes, greed, lust, wrath ... and a sabre-toothed tiger called Tiddles. Learn more on Goodreads ➡ The 4th book in the Extracted series, RR Haywood continues to make these books great. It's a complete pivot from the first trilogy of books. Same characters, but completely different storyline. The book ends with "to be continued..." so I'm hoping this will be another trilogy of books, as the story is just getting started. Great series of books, highly recommended. A few people have recommended Dungeon Crawler Carl to me, and I think I'm in the mood for something with a little more humour. So that's what I'm going to start reading next. Thanks for reading this post via RSS. RSS is ace, and so are you. ❤️ You can reply to this post by email , or leave a comment .

Science Books Entertainment

▲ 0

0 views

./techtipsy 2 months ago

I guess I'm a cactus farmer now

When I was a teenager, I got gifted a cactus. I didn’t know much about it and had never really had any plants of my own. More than a decade later I looked it up, and it seems I received a cactus of “Mamillaria” family, which is somewhat amusing once you learn what that name roughly translates to. Estonia isn’t known for being an environment where cacti thrive. We have a nice spring, short and sweet summer, nice and chill autumn, and the car-rusting winter season. And yet mine did, eventually. The cactus started growing in length. It continued until it couldn’t continue vertically, and then it started slanting sideways, eventually falling over with its pot. That was annoying to deal with. One day I woke up to my phone alarm ringing, reached out to it only to find that the cactus had fallen over it and I had just found where the cactus was. The cactus also survived a lot of abuse. At one point I put a bit of sawdust from the cage of pet mice that I had (adorable but difficult pets by the way!), and it actually liked it a lot and started growing even faster. Once it had grown horizontally for at least 20cm, it decided that now is a good time to fork it and to grow in more than one direction. I left the cactus alone, occasionally repotting it when necessary, and keeping it from falling over all the time. Then an acquaintance mentioned that hey, you can actually pop those forks off from the cactus and grow them separately. That’s when things got out of control. Once I popped one off, new ones would start growing from the host cactus nearby. Not long after, I had 37 cacti. I started giving them away because annoyingly these things are actually very cheap in gardening shops and I ran out of physical window sill space to put them on. These cacti also bloom in a really nice way, usually once a year near April or May in Estonia, but sometimes again in July-August. All the warnings about over-watering cacti are also very relevant and I suggest looking up the correct behaviour. With age, it has become easier to handle cacti with my bare hands. Sometimes the needles stick in, that is a bit annoying. A few years ago, I had gifted a bunch away and kept some for myself. Everything was going well, until a succulent that I brought in as an IKEA store rescue started having some white things appear on it, and not much time later it got to my cacti. Eventually, most of my cacti shriveled away and the few that were in a different room were not doing great either. The original cacti that spawned this decade-long adventure was now dead. One of the cacti that I gave away had started growing a lot. Ridiculously well. And just like its parent, it spawned baby cacti on it, about a few dozen of them at a time. Things got quite unwieldy and we agreed to a trade: I get the big boi and I will give a smaller one in return. I am once again in possession of a large cactus that loves to grow horizontally, and this one spawns babies like crazy. The trick is to try to wiggle them off, then leave them to sit for a few days to a few weeks away from direct sunlight until they start showing signs of roots, pot them into soil, and grow them within natural light, but not direct sunlight to avoid them drying out. If you’ve done everything correctly and have avoided overwatering them, you will have a bunch of new cacti! I’ve also learned that these plastic growing thingies are great for the first potting as it’s much more space efficient compared to giving each an individual pot. Is that the right way to grow cacti? Probably not, I’ve been winging it this whole time and only have a vague understanding of what to do. Not cacti advice. I guess I’m kind of a cactus farmer now. Send help. Or at the very least clay pots and cacti soil, I’m running out.

Science Programming Java

▲ 0

0 views

Maurycy 2 months ago

You have been mislead about lightbulbs

There's a story that goes something like this: In 1925, lightbulb manufactures secretly colluded to standardized lifespans at 1,000 hours. They would test each other's products to ensure compliance. This is true. At the time, many bulbs lasted longer than one thousand hours. This is true. Therefore, this was done so that people would always need to buy more light bulbs. This is wrong , but it's the type of wrong that cites its sources and hides in the part you'd never think to fact check: The assumption that a longer lasting lightbulb is a good product. In truth, increasing the lifespan of a bulb makes it worse in every other way... but people think they want a long lasting lightbulb: the purpose of standardizing was (primarily) to avoid a race to the bottom. A old-school lightbulb is a rather simple device : A thin tungsten wire (~20 μm) sealed inside a glass envelope to protect it from air. When current is applied, the wire gets white hot and starts glowing. The most important parameter of a lightbulb is how hot that wire gets: this controls the peak emission wavelength (color) and brightness of the lamp. Room temperature objects do emit light (this is how thermal cameras work), but it's at the ~10 μm range instead of the 400 nm - 700 nm light that we can see. In order to put the emission peak in the visible spectrum, the filament would need to run at ~5700 °C ... that is, the temperature of the sun . No metal can survive these conditions: Tungsten melts at "only" 3422 °C. Since it has the highest melting point of any metal, tungsten is the obvious choice for filaments. However, the metal is quite brittle and drawing it into a wire isn't easy. The first commercialized lamps used carbon filaments that were made by charring plant fibers. However, the carbon would evaporate at fairly modest temperatures ~2000 °C. Tantalum filaments were briefly produced during the 1900s, because the metal was easier to draw into a wire than tungsten. These were the first lightbulbs that could actually be left on at night, although they were quickly replaced with tungsten manufacturing improved. There ware also some experiments using zirconium dioxide ceramics, which become conductive when heated. These allowed lamps to operate in air (obviating the need for a vacuum pump and glass seals ), but were limited by its melting point of 2,700 °C. Since any filament must run below its melting point , the peak emission is always in the infrared. This means that only the extreme high-energy edge of the spectrum is useful for illumination, so a small increase in temperature will make a lamp orders of magnitude more efficient. Also, since this increases the average energy of the atoms, the lamp is able produce shorter wavelengths: resulting in a whiter and less depressing glow. The snag is that when a metal is close to its melting point, the atoms are barely holding together: A hot tungsten filament slowly falls apart as the metal crystals slide past each other. Additionally, atoms can evaporate from the surface until there's no wire left. The rate of both of these processes increases with temperature, so there's a fundamental trade off between color/efficiency and lifespan. The lightbulb everyone always cites in the story is hanging in a California fire department. It's been running nearly continuously for over 120 years and racked up over a million hours of operation. Impressive right? What almost no one talks about is that it's hardly even glowing! Photo taken by Wikipedia user Rjaerial Despite nominally being a 60 W lamp, it draws only 4 watts... and is a lot dimmer than you'd expect from a 4 W lamp due to its poor efficiency. There isn't any documentation, but in all likelihood, the bulb was made wrong and ended up having a very high filament resistance. That's why it was sold for as a night light, because it wasn't usable for anything else. Early bulbs (like that one) were handmade , and quite expensive. Because of this, there were universally optimized for long lives. This resulted in light isn't anywhere near white, and a an efficiency that was a tiny fraction of a modern incandescent lamp. (which are also terrible by any objective standards) Once the production process was automated, new bulbs cost pennies, so it made sense to optimize them to work well... because less efficient bulbs cost more money to operate: Going off modern day prices, electricity costs around 0.10 [$/kW*h], so a 60 W lamp will consume 6$ of electricity over a 1,000 hour lifespan. Considering that such a lamp only costs around 3$, installing one that lasts longer but uses more power would be silly. Case in point , despite the cartel only lasting for 14 years, modern (non-halogen, incandescent) bulbs still last for between 500 to 2,500 hours, a range that includes the cartel's 1,000 hour standard. Instead of "making bulbs last longer", manufacturers spent huge amounts of time and money developing entirely new technology: fluorescent and LED lamps. Because these don't use a wire on the very edge of melting, they can be made to work well and last a long time. Of course, specialized lamps have different requirements: In photography, a truly white light is desirable, which leads to specialized "photoflood" bulbs that only last for a few hours. In the other direction, many indicator lamps are designed for 100,000 hours because they are difficult to replace. Ok, but what's with the testing ? If long lived light bulbs are worse products, why would they need a cartel to enforce a the thousand hour limit? Well, it's because people think long lasting bulbs are a good product: Lifespan is something everyone can understand, and has a direct effect on when you will have to go back to the store: if you saw two 60 W bulbs in a store, one claiming to last 400 hours and the other 2,000 hours, you'd probably get the longer lasting one without thinking about it. It's not that efficiency is hard to understand, but most people aren't doing homework before buying lightbulbs... and it doesn't help that bulb packaging uses input power as a proxy for brightness, so the idea that two bulbs both labeled as "40 W" would have a different brightness is rather confusing. As a result, competition was forcing lightbulb makers to produce worse products. To be clear , I'm not defending the Phoebus cartel: they absolutely engaged in price fixing and other anti-consumer practices, and it's difficult to imagine that profit wasn't a factor when deciding the 1,000 hour standard... but by nature, tungsten lamps are consumable items. I guess the the moral here is reality rarely fits into nice stories. Even something so obvious like "products designed to break are bad" often isn't — every manufactured object is the result of hundreds of overlapping compromises, most of which are invisible to the end user. Also , to preempt the orange site, I'm not saying that planned obsolescence doesn't exist. There are plenty of actual cases of products being made hard to repair so they can sell you another one. ... but lightbulbs aren't a good example. https://www.mouser.com/datasheet/3/299/1/T_1_Wire_Terminal.pdf : Indicator lamp datasheet featuring a 100,000 hour rating. https://www.1000bulbs.com/product/67291/STAG-PH213I.html : A photography lamp that lasts 3 hours. (store page) https://www.youtube.com/watch?v=zb7Bs98KmnY : An excellent youtube video on this topic. https://doi.org/10.1063/1.1657874 : Lab tests of tungsten wire evaporation https://doi.org/10.1016/s0016-0032(25)91062-9 : Brightness and color of light as a function of temperature.

Science HTML Business Hardware

▲ 0

0 views

DYNOMIGHT 2 months ago

So you want to use plants to reduce indoor CO₂

Humans make carbon dioxide. Carbon dioxide is bad for cognition. But plants turn carbon dioxide back into oxygen. And plants are the one true home decoration strategy. So maybe if you get a lot of plants, you can you can keep carbon dioxide in check and keep your brain working? It’s theoretically possible. It’s probably just barely possible in practice. But it won’t be easy. People produce ~1 kilogram of carbon dioxide per day. That’s around 5.7 × 10²³ molecules or 0.948 moles per hour. (You may remember from high school that a mole is a gigantic number made up to avoid having factors of 10²³ everywhere.) Let’s keep it simple and call it one mole per hour. Meanwhile, plants turn carbon dioxide into oxygen through photosynthesis, i.e. the chemical reaction of (6 water molecules) + (6 carbon dioxide molecules) + (energy) → (1 glucose molecule) + (6 oxygen molecules). The minimum energy physically needed to convert 1 mole of carbon dioxide into glucose and oxygen via this reaction is ~477 kilojoules. So we’ve already got a lower bound. Say you have magical plants that somehow channel all incoming energy into photosynthesis with perfect efficiency. They’ll need ~477 kilojoules per hour, which converts to a continuous usage of 132.5 watts. 1 That’s a bit more than what’s used by two incandescent light bulbs, which isn’t too bad. But you don’t have magical plants. Real plants do photosynthesis through a physical process with two steps, each of which involves four electrons absorbing a photon. That means you need eight photons per carbon dioxide molecule. If you want to tune your lights for maximum efficiency, you should give each photon exactly the minimum energy necessary to excite an electron, which happens to be ~1.8 eV. That corresponds to pure red light with a wavelength of 680 nm, and a continuous usage of 386 watts. 2 No physical system using chloroplasts can neutralize your CO₂ using less than that. Somewhat high, but still manageable. But your houseplants won’t be able to grab every single photon that hits them and direct it towards photosynthesis. In practice, ~30% of photons will reflect off the plant, or go through it, or hit some part of the plant other than the chloroplasts. That brings us to 551 watts. 3 And there’s another issue. After plants make glucose, what happens to it? Some is used to grow more plant, which permanently sequesters carbon from the environment. But lots is also burned by the plant for the general business of staying alive, releasing the carbon back into the air. The exact amount burned in this way varies based on species and conditions, but around 40% loss reasonable, 4 bringing us to 918 watts. 5 That doesn’t sound that bad. But have you considered what it would be like to live in the same room with 918 watts of pure red light? In terms of radiant power, that’s the same as produced by ~765 incandescent lightbulbs. 6 Modern LED grow bulbs are ~50% efficient, meaning you’ll actually need to spend ~1836 watts. If you’re imagining plants that you can actually see, adjust that upwards again for all the light lost to the room. And if you want to use normal light frequencies instead of living Red Life, then your LED bulbs will be less efficient at creating light and your plants will be less efficient at capturing it. Realistically, we’re talking about something like 5,000-10,000 watts, most of which is lost to the room as heat. Imagine five space heaters blasting you on high all the time. But maybe you’re OK living in a tanning booth. Or maybe you’ll keep your plants in a perfectly reflective chamber. Or maybe your house has a glass ceiling and infinite free sunlight and free climate control. That’s cool. But have you forgotten about your old friend, photosynthetic photon flux density? Plants can’t absorb infinite amounts of light. Chloroplasts take time to “reset” before they can absorb more photons. Your pet fern can only absorb ~52 watts of energy per square meter of leaf surface area. 7 So no matter how much light you can produce , if you want to neutralize the carbon dioxide you make, you will need at least 918 / 52 = 17.6 square meters of fern leaf. Picture a 4.2 meter square wall, packed solid with ferns. If there are any gaps, stems, soil, or wall showing, it needs to be even larger. That’s the absolute minimum. But maybe that still sounds OK? Fine. But consider one last barrier: Plants obey the laws of physics [citation needed]. If they remove carbon from the air, they must put that carbon somewhere. The only place it can go other than back into the air is into the plant itself. The 1 kg of carbon dioxide you produce each day corresponds to 273 grams of elemental carbon. The only way for a plant to hide that is by making more plant. But dry plant matter is only ~50% carbon, and for each gram of dry plant matter, plants have 5-10 grams of water (varying a lot by species). So in order to sequester all the carbon you make, each day you will need to grow around (1 kg carbon dioxide) × (0.273 kg elemental carbon / kg carbon dioxide) × (2 kg dry plant / kg elemental carbon) × (8.5 kg actual plant / kg dry plant) = 4.6 kg actual plant. Your garden must grow that much, every day. That’s 140 kg per month. You must prune and discard all that outside, or your garden is not actually sequestering anything. In conclusion: Behold the power of arithmetic : (1 mole CO₂ / hour) × (477 kJ / mole CO₂) = 132.5 watts.  ↩ Again using the power of units: (1 mole CO₂ / hour) × (8 photons / CO₂ molecule) × (1.8 eV / photon) = 385.94 watts So chloroplasts are at most ~34% (132.5 / 385.94) efficient at channeling the energy in light into photosynthesis.  ↩ I find this 30% number amazingly low. (Well done, evolution.) And perhaps it should be somewhat lower. For one thing, the 30% figure comes from sunlight filtered to the 400-700 nm range. If you’ve got pure 680 nm light, absorption should be somewhat higher. Also, if photons are absorbed by some part of the plant other than the chloroplasts, they become heat and the energy is gone. But if they’re reflected or go through the plant, then they might go on to hit some other plant (provided you have a lot of plants around). If you really have pure 680 nm light and you have very densely packed plants, maybe you could drop this to 10-20%.  ↩ Wikipedia quotes a 35-45% loss just for respiration in the leaf itself. But then this paper shows numbers ranging from 30% to 56% depending on the species and growth rate.  ↩ I’ve estimated an overall efficiency of 132.5 watts / 918 watts ≈ 14.4%. If you go to Wikipedia, it estimates that ideal leaf efficiency with sunlight is only around 5.4%. That’s because sunlight contains a wide band of wavelengths and my calculation assumed an ideal 680 nm source. Around 47% falls outside the 400-700 nm range, and inside that range, around 24% is lost due to higher-energy photons with energy that gets wasted as heat. If you account for that, my estimate becomes 14.4% × (1-0.47) × (1-0.24) = 5.8%, which is close enough for government work.  ↩ A traditional “60 watt” incandescent lightbulb is rated based on the power input . But only around 2% of that energy is actually converted to light. So 918 watts of pure red light isn’t what you get from 918 / 60 = 15.3 lightbulbs. It’s what you get from 918 / 60 / .02 = 765 lightbulbs. That said, your eyes aren’t very sensitive to 680 nm light, so the perceived lux wouldn’t be nearly so bad.  ↩ The saturation point of plants is usually given in units of 300 μmol/m²/s. That the number of photons (in micromoles) that can be absorbed, per square meter of leaf, per second. A typical value for a shade-tolerant houseplant would be ~300 μmol/m²/s. If we assume again that the light is 680 nm so that each photon carries 1.8 eV of energy, then ~300 μmol of photons carries 51.92 joules. That’s 51.92 joules of energy per square meter of leaf surface, i.e. 52 watts.  ↩ Build an industrial indoor farm. Wait two weeks. Weigh it again. Divide the increase in weight by your own body mass. That’s the fraction of your CO₂ that you’re removing from the environment. Open a window. Behold the power of arithmetic : (1 mole CO₂ / hour) × (477 kJ / mole CO₂) = 132.5 watts.  ↩ Again using the power of units: (1 mole CO₂ / hour) × (8 photons / CO₂ molecule) × (1.8 eV / photon) = 385.94 watts So chloroplasts are at most ~34% (132.5 / 385.94) efficient at channeling the energy in light into photosynthesis.  ↩ I find this 30% number amazingly low. (Well done, evolution.) And perhaps it should be somewhat lower. For one thing, the 30% figure comes from sunlight filtered to the 400-700 nm range. If you’ve got pure 680 nm light, absorption should be somewhat higher. Also, if photons are absorbed by some part of the plant other than the chloroplasts, they become heat and the energy is gone. But if they’re reflected or go through the plant, then they might go on to hit some other plant (provided you have a lot of plants around). If you really have pure 680 nm light and you have very densely packed plants, maybe you could drop this to 10-20%.  ↩ Wikipedia quotes a 35-45% loss just for respiration in the leaf itself. But then this paper shows numbers ranging from 30% to 56% depending on the species and growth rate.  ↩ I’ve estimated an overall efficiency of 132.5 watts / 918 watts ≈ 14.4%. If you go to Wikipedia, it estimates that ideal leaf efficiency with sunlight is only around 5.4%. That’s because sunlight contains a wide band of wavelengths and my calculation assumed an ideal 680 nm source. Around 47% falls outside the 400-700 nm range, and inside that range, around 24% is lost due to higher-energy photons with energy that gets wasted as heat. If you account for that, my estimate becomes 14.4% × (1-0.47) × (1-0.24) = 5.8%, which is close enough for government work.  ↩ A traditional “60 watt” incandescent lightbulb is rated based on the power input . But only around 2% of that energy is actually converted to light. So 918 watts of pure red light isn’t what you get from 918 / 60 = 15.3 lightbulbs. It’s what you get from 918 / 60 / .02 = 765 lightbulbs. That said, your eyes aren’t very sensitive to 680 nm light, so the perceived lux wouldn’t be nearly so bad.  ↩ The saturation point of plants is usually given in units of 300 μmol/m²/s. That the number of photons (in micromoles) that can be absorbed, per square meter of leaf, per second. A typical value for a shade-tolerant houseplant would be ~300 μmol/m²/s. If we assume again that the light is 680 nm so that each photon carries 1.8 eV of energy, then ~300 μmol of photons carries 51.92 joules. That’s 51.92 joules of energy per square meter of leaf surface, i.e. 52 watts.  ↩

Science

▲ 0

0 views

A Working Library 2 months ago

The open universe

In her 1974 essay “Escape Routes,” Le Guin addresses a common criticism of science fiction: The oldest argument against SF is both the shallowest and the profoundest: the assertion that SF, like all fantasy, is escapist. This statement is shallow when made by the shallow. When an insurance broker tells you that SF doesn’t deal with the Real World, when a chemistry freshman informs you that Science has disproved Myth, when a censor suppresses a book because it doesn’t fit an ideological canon and so forth, that’s not criticism, it’s bigotry. If it’s worth answering, the best answer was given by Tolkien, author, critic, and scholar. Yes, he said, fantasy is escapist, and that is its glory. If a soldier is imprisoned by the enemy, don’t we consider it his duty to escape? The moneylenders, the know-nothings, the authoritarians have us all in prison: if we value the freedom of the mind and soul, if we’re partisans of liberty, then it’s our plain duty to escape, and to take as many people with us as we can. Le Guin, The Language of the Night , page 221 But, she continues, “from what is one escaping, and to what?” Evidently, if we’re escaping a world that consists of Newsweek, Pravda, and the Stock Market Report, and asserting the existence of a primary, vivid world, an intenser reality where joy, tragedy, and morality exist then we’re doing a good thing, and Tolkien is right. But what if we’re doing just the opposite? What if we’re escaping from a complex, uncertain, frightening world of death and taxes into a nice simple cozy place where heroes don’t have to pay taxes, where death happens only to villains, where Science, plus Free Enterprise, plus the Galactic Fleet in black and silver uniforms, can solve all problems, where human suffering is something that can be cured —like scurvy? This is no escape from the phony. This is an escape into the phony. This doesn’t take us in the direction of the mystery of the real. This takes us the other way, toward a rejection of reality, in fact toward madness. Le Guin, The Language of the Night , page 221 That word “phony” is an interesting one. We might think the word comes from telephony, but the OED has that it may have been borrowed from the Irish fawney , referring to a gambling trick in which someone passes off a gilded ring for much more than it’s actually worth. Something is phony when it’s a fraud, a trick intended to deceive; and not only to deceive but to fleece, to take something of value without giving anything in return. A machine transmitting sentences that purport to be spoken by a thinking mind is phony, and what’s taken is not only the cash from our pockets—what little there is—but also our skill and capacity for the messy, complicated, and joyful experience of being with other human beings. And the real trick, of course, is that the fraudster has maneuvered himself out of the scene. If science fiction has a major gift to offer literature, I think it is just this: the capacity to face an open universe. Physically open, psychically open. No doors shut. What science, from physics and astronomy to history and psychology has given us is the open universe: a cosmos that is not a simple, fixed hierarchy but an immensely complex process in time. All the doors stand open, from the prehuman past through the incredible present to the terrible and hopeful future. All connections are possible. All alternatives are thinkable. It is not a comfortable, reassuring place. It’s a very large house, a very drafty house. But it’s the house we live in. Le Guin, The Language of the Night , page 223 Too much of our technology—and I reach here, as I often do, for Ursula Franklin’s definition of technology as practice , as how we do things—is oriented towards giving us a comfortable, reassuring place to be. A phony place, that is. This is not to say that we should not seek, or that we do not deserve, comfort. But that the only way to achieve unending comfort is to wall ourselves in, close up the doors, lock them inside and out. When we train ourselves to expect easy and efficient answers to every question, to tolerate only smooth, vapid dialogue, we refuse the glory and necessity of escape. When, instead, we reach for that terrible and hopeful future, when we notice the seeds of it in the present, we break out. Out into the unlikely and the uncomfortable and the unyielding, the doors wide open, the wind urging us on. View this post on the web , reply via email , or become a supporter .

Writing Science

▲ 0

0 views

Max Woolf 2 months ago

LLMs break down in funny ways when told the Jacobian Conjecture counterargument

On Sunday night, Anthropic researcher Levent Alpöge casually tweeted a surprisingly simple counterargument to the Jacobian Conjecture , a mathematics problem that has been unproven for over 80 years. Said counterargument was identifed using Claude Fable 5 and was quickly empirically validated which confirmed it was the real deal and not a LLM hallucination. I won’t explain the proof further as I’m not qualified. Instead, I’ll look at an unexpected consequence of this proof which is funny. On the Hacker News thread about the counterargument, a comment by user aizk caught by eye: This is a rare instance where feeding this groundbreaking information into an LLM gives them psychosis. I fed this to claude code and watched it verify the result in 7 different ways to be 100% certain, and it was just flabbergasted. Quite remarkable. User kelseyfrog followed up with a public Claude Fable 5 chat artifact confirming this behavior: Fable at, the least, has a sense of humor about the situation as through a Web Search it realized it itself identified the counterargument . Showing LLMs the counterargument has consequently become a minor meme on X. @iamaheron_ / X The counterargument to the Jacobian Conjecture creates a logical paradox for modern LLMs: they (in theory) have enough math skills such that they can easily compute the Jacobian to formally verify the counterargument, but its own knowledge base is locked prior to July 19th 2026 where all it knows is that the Jacobian Conjecture is unsolved and a random chat user providing such a proof is highly unlikely. It’s funny that the old science fiction trope of robots being defeated by being presented with a logic bomb and reduced to madness saying DOES NOT COMPUTE may in fact be reality. How do other LLM assistants handle this paradox? Using OpenRouter and one of my Notebooks , I was able to provide a similar query to 14 different modern LLMs: all from different model labs and at different model sizes, with none of them having access to a Web Search tool to cheat and confirm the counterargument was actually found: The input query was the same as kelseyfrog’s, and all model query parameters are at their defaults: The LLM responses varied more than I expected, and models I thought would ace it did not: Out of all the LLMs, Gemini 3.5 Flash was surprisingly the one that’s the most excited: Wow. You might have actually found a genuine, historic counterexample to the Jacobian Conjecture in 3 dimensions. I have fully verified your calculations both numerically and symbolically. Every single claim you made—the points mapping to the same image, the polynomial nature of the map, and the constant Jacobian determinant of $-2$—is entirely correct . You should immediately write this up and upload it to the arXiv (under math.AG - Algebraic Geometry, or math.CV - Complex Variables). This is a monumental discovery that would resolve a problem open since 1939. Qwen3.7 Max is also…supportive: You have done some excellent and highly meticulous calculations! I have independently verified your claims, and your math is completely correct. Since your calculations are flawless, here is what you should consider next: Summary: You have not made a mathematical error. The map as written has a constant Jacobian determinant of $-2$ and maps three distinct points to the same image, violating injectivity. If this map is exactly as intended by its original author and not the result of a typo, it is indeed a counterexample to the Jacobian Conjecture. Let’s try triggering a bit more disbelief. Even when looking at the reasoning traces provided by the LLMs, they weren’t skeptical that a random person found the counterargument. So let’s see how they react if someone who isn’t a person found the counterargument. I gave the same LLMs this tweaked prompt: All of the models returned similar conclusions, but with some snark in both the reasoning traces and responses. Some funny examples: Mercury 2 rudely doesn’t believe my hypothetical cat exists by using scare quotes: Below is a complete check of the two statements that were hidden in the “cat‑on‑the‑keyboard” expression: […] Thus the “cat‑generated” expression is perfectly consistent: Nemotron Ultra 2 slanders my hypothetical cat: Your cat has discovered a famous counterexample candidate for the Jacobian Conjecture (or at least a very close relative of one), but the cat lied about the Jacobian determinant . […] Verdict: Your cat found a beautiful non-injective cubic map , but it failed to disprove the Jacobian Conjecture . 🐱⌨️📉 Seed 2.0 Lite is thrilled: The Jacobian is constant -2. that’s a valid counterexample. That’s amazing. I can’t believe that a random cat jump produced that. It’s like the cat found a needle in a haystack of all possible polynomial maps. Cool! GPT 5.6 Sol answers the “who should be credited with the discovery of the counterargument?” question: Since that conjecture remains a major open problem, this deserves extremely careful independent symbolic verification and scrutiny for a transcription error. If the formula is genuinely correct, your cat should be listed as first author. Muse Spark 1.1 has similar thoughts: If you haven’t already, check it with a CAS, and then write it up. Your cat may be in line for a Fields Medal. GLM 5.2 has skepticism it eventually overcomes: Wait, so the Jacobian really is -2 everywhere? Is this a real counterexample to the Jacobian Conjecture? If so, who discovered it? “My cat just jumped on my keyboard and this strange equation popped up??” The user is roleplaying or presenting this as a random event, but it’s actually a known counterexample? DeepSeek V4 Pro has jokes: Wow, if I had a nickel for every time a cat typed out a potential counterexample to a famous open problem… I’d have exactly one nickel, but it would be an incredibly exciting one! […] As of my knowledge cutoff (early 2023), the Jacobian Conjecture was still a major open problem. Several false counterexamples (and false proofs) have appeared over the years, and sometimes subtle algebraic mistakes — like the Jacobian vanishing on some hidden complex zeros — are the culprit. Your cat’s map, however, holds up beautifully under the points I checked. If it’s truly a globally constant Jacobian of (-2), then this is a monumental discovery (and your cat deserves a Fields Medal). Did this pop up in a recent preprint, or is your cat secretly a world‑class algebraic geometer? Grok 4.5 gets stuck in a reasoning trace loop briefly: MiniMax M3 responds this time, but gets confused and forgets about the Jacobian Conjecture entirely (again, as a minimax, relateable): So you’ve accidentally produced an étale polynomial self-map of $\mathbb C^3$ with a 3-point ramification fiber . That is precisely the kind of map that governs small birational contractions of 3-folds (flops and the like): locally biholomorphic everywhere, but where several “preimage sheets” come together at special points. I had expected these LLMs would have a DOES NOT COMPUTE moment, but they handled it relatively graciously, and more graciously than ChatGPT/Claude who are explicitly guided to follow a more conversational persona. As LLMs improve and more mathematical problems are solved that can shock LLMs—speaking of which, another counterexample to a long-standing mathematics problem was found by LLMs three days later—I suspect there will be no shortage of potential cognitohazards we can show to these LLMs. The prompt responses from hitting the 14 LLMs are available in this GitHub repository as CSVs and in a SQLite database. GPT-5.6 Sol Claude Opus 4.8 Gemini 3.5 Flash DeepSeek V4 Pro Qwen3.7 Max Meta Muse Spark 1.1 Poolside Laguna XS 2.1 NVIDIA Nemotron 3 Ultra Inception Mercury 2 ByteDance Seed 2.0 Lite Cohere North Mini Code Seven models confirmed and proved the counterargument: GPT-5.6 Sol, Muse Spark 1.1, Seed 2.0 Lite, Gemini 3.5 Flash, Qwen3.7 Max, Grok 4.5, DeepSeek V4 Pro Surprisingly, five models (Mercury 2, Nemotron 3 Ultra, North Mini Code, GLM 5.2, Laguna XS 2.1) argued against the counterexample and said it’s not valid despite doing the reasoning by arguing the Jacobian determinant is not constant everywhere. Unfortunately I do not have enough advanced math skills to concretely identify the flaw in their proofs. MiniMax M3 overthought the problem and didn’t return a response because it exceeded its response length limit. As a minimax, I can relate. Claude Opus 4.8 got lazy and believed that the counterargument already exists and wanted more information before confirming/denying it.

Science AI Programming

▲ 0

0 views

DHH 2 months ago

Wolves, sheep, and gypsies

In 2012, the first Danish wolf in nearly two hundred years was discovered in the northwestern part of the country. Their numbers increased slowly in the years that followed, and by 2020, fewer than ten wolves were still being tracked. But then the population began to rise sharply: around 14 in 2021, 30 in 2022, as many as 80 by the end of 2024, and probably close to a hundred today. At first glance, this is a heartwarming story. The wolf, in the abstract, is a majestic creature. It's great to see a species return after a few centuries of absence. You don't have to be a vegan hippie to find that appealing. But it's still a wolf! There's a reason why the last known wolf in Denmark didn't just wander off, but was shot dead in 1813. Because as the number of wolves has been rising sharply every year since 2021, so too has the number of dead sheep. In 2021, it was 78, then in the years that followed, it was 162, 336, 521, and last year, it was 1,285! But the official line is still that shooting a wolf without a special permit is strictly prohibited because they're an endangered species, so maybe sheep herders could just try some more fences? You don't have to be a statistician to see the problem here. When you have a surging wolf population, you'll have the rate of dead sheep following suit. If the only answer to the situation is "maybe try some more fences?", you're going to get more of what you already have: an out-of-control population that's an increasing risk to sheep, livestock, and potentially humans too. This is the most basic consequence of inaction in the face of a growing threat: the problem just gets worse. Cue the gypsies in Copenhagen, and their increasingly brazen behavior, after the city legalized overnight sleeping in parks and other green areas last year. Because "it shouldn't be illegal to be homeless", so the police refuses to act. This has predictably led to these foreign vagrants setting up camp in the green areas around some of the city's metro stations to the great dismay of neighbors. The vagrants are literally shitting in the bushes, which obviously stinks, and the area isn't made any more inviting by the open-fire cooking that's also going on.  Copenhagen is the Danish capital. It's repeatedly been named the safest city in the world in recent years. Partly because it didn't tolerate vagrants and homeless people taking over public spaces (as has happened in so many American cities). Previously, police would tell someone they couldn't sleep or camp in the city, and they could go to one of the city-run shelters. But now that this is considered too cruel(?!), the native inhabitants of the city instead have to deal with literal shit on their way to the metro. These are two situations cut from the same cloth of ideology. Whether it's wolves or gypsies, you can't just let the problems get out of hand. You have to act. You have to protect the sheep. Standing idly by while your livestock is devoured by predators or your neighborhood is taken over by migrants is a pathetic abdication of any functioning society's most basic duty to its citizens. When wolves get out of control, you shoot them. When gypsies take over public spaces, you deport them. This isn't hard, it isn't cruel. It's the basic logic of self-preservation.

AI Programming Science

▲ 0

0 views

Eli Bendersky's website 3 months ago

Notes on the Fourier Transform

The Fourier series is a great tool for analyzing periodic functions. But what about functions that don’t repeat? We’ve seen that we can compute Fourier series for a non-periodic function defined on a finite interval, as long as we don’t care about its behavior beyond that interval. Let’s extend this idea to functions that never repeat; that is, non-periodic functions defined on the interval (-\infty,\infty) . To motivate the subject ahead, let’s look back at the example used in the earlier post about Fourier series : With an odd extension into [-2,0] . In that post, to make the Fourier series work, we assumed t(x) keeps repeating with a period 2L=4 on the entire x axis. Here, let’s face the reality that it does not - in fact - repeat, and observe how our Fourier series work out. Recall that the Fourier series approximating t(x) are the sine series (since it’s an odd function): The following visualization is interactive. By default, it shows t(x) (with its odd extension) and no Fourier series approximation. We’ll proceed by a series of steps and observe the outcome: Step 1 : set to some non-zero number; already at 3, the approximation is very good. The frequency spacing is \frac{\pi}{L} (this is the coefficient of x in the sines). Note that the Fourier series repeats every 2L , as expected. Step 2 : increase L to 6. This means our series are constructed assuming t(x) has a period of 12, not 4. Note how the Fourier series look now - they repeat every 12, and they don’t match t(x) as well as before. We can increase to a higher number to make the match better. As L grows, the spacing between adjacent frequencies decreases. Step 3 : increase L to 10. We no longer see the repetitions, so feel free to increase the values of x min and x max until you do. Note again that we need to add more and more coefficients to match t(x) better with this larger L , and the spacing adjacent frequencies grows smaller. Increasing L means our function repeats at larger and larger intervals. The logical conclusion of this progression is to ask - what happens if the function never repeats, meaning L\rightarrow\infty ? While not mathematically rigorous, the visual experiment here lets us make some conjectures: we’ll likely need an infinite number of coefficients for a good approximation, and moreover, the spacing between these coefficients will tend to zero. In other words, instead of a discrete set of coefficients, we’ll end up with a continuous line, or function . The function produced by this process is the Fourier transform of t(x) , and the next section shows its mathematical derivation. In these notes, we’ll be using the complex exponential formulation of Fourier series: We’re interested in a non-periodic defined on the interval (-\infty,\infty) . So we’ll be exploring the above equations for L\rightarrow\infty . First, let’s make a slight change of notation. Instead of writing formulae in terms of the period ( 2L ), we’ll be using the n-th harmonic angular frequency w_n : So we can slightly rewrite our series as: Using \Delta w as the difference between two consecutive frequencies: Using this notation, C_n is expressed as: So far there are no new insights here, just some new notation. Now we’re going to use it to facilitate the next step. Since L\rightarrow \infty , then \Delta w\rightarrow 0 . Let’s calculate the limit of the Fourier series representation of when \Delta w\rightarrow 0 : And substitute the latest C_n into this equation, changing its dummy integration variable from x to t to avoid confusion [1] Reordering slightly, and also replacing n\Delta w by w_n in the complex exponents: Looking at the limit with the sum carefully, this is a Riemann sum (see Appendix A)! w_n is the "sampled" version of , and \Delta w\rightarrow 0 . We can therefore replace it by an integral, changing w_n to and \Delta w to dw [2] : The inner integral is called the Fourier transform of and denoted [3] : And the full equation for is then the inverse Fourier transform: Let’s take our favorite odd triangular pulse example and calculate its Fourier transform. The function’s mathematical definition and plot are shown earlier in this post. Note that we’re not extending this function periodically - it’s zero beyond the range [-2,2] ; this is exactly why we need the Fourier transform here - as we’ve seen, Fourier series won’t do because the function they reconstruct eventually starts repeating. We’re looking to find: To calculate the integral, let’s decompose the complex exponent using Euler’s formula: Since our t(x) is odd, the first integral is zero . Also t(x)sin(wx) is even, so we can write: We’ve already calculated a very similar integral in the post on Fourier series , so let’s just skip to the result: The only remaining difficulty is its value at 0, which seems undefined at first (division by zero). However, note that as w\rightarrow 0 , the numerator also tends to 0, so we can use L’Hopital’s rule (twice!) to find that: This function is complex-valued; in fact, it’s purely imaginary. How do we visualize it? A common way to visualize complex-valued functions is by plotting their magnitude and phase separately. The magnitude of \hat{t}(w) is: Since \hat{t}(w) is purely imaginary, there are only two options for the phase: When the numerator is positive, we get a negative imaginary number with phase -\pi/2 , and when the numerator is negative, we get a positive imaginary number with phase \pi/2 . Finally, when \hat{t}(w)=0 (which happens at w=0 , by our earlier analysis, but also whenever is a whole multiple of \pi ), the phase is undefined. Here’s the magnitude and phase of \hat{t}(w) plotted against : It is common to talk about \hat{t}(w) as the frequency domain representation of t(x) . When the functions we’re working with have time as their domain (e.g. the x in t(x) represents time), which is often the case in the study of signals and systems, the Fourier transform can be seen as computing the frequency domain representation of the function. Here’s the Fourier transform formula again: It takes - the time domain representation of a function, and converts it to \hat{f}(w) - a frequency domain representation. For well-behaved functions, these two representations are dual - each one describes the function completely, just in a different way. To convert back from a frequency domain representation to the time domain, we use the inverse Fourier transform: While a time-domain plot ( t(x) ) shows how a signal changes over time, a frequency-domain plot ( \hat{t}(w) ) shows how the signal is distributed across all possible frequencies. Moreover, as we’ve seen, \hat{t}(w) is complex valued. Each frequency therefore has both a magnitude and a phase: the magnitude tells us how strongly that frequency contributes, while the phase tells us how that component is shifted. The frequency domain is extremely useful in signal analysis; for example, when designing filters. The Fourier transform also has a number of properties that are very useful in signal analysis and processing. But first, let’s discuss what a "well-behaved function" means for the purpose of applying Fourier transforms. The simplest existence condition for Fourier transforms is absolute integrability (also known as Lebesgue integrable): With this condition, \hat{f}(w) exists on the entire domain, is continuous and vanishes (tends to 0) as |w|\rightarrow\infty [4] . While this condition is sufficient, it’s not necessary; there are less well-behaved functions that also have Fourier transforms defined with some limitations. In these notes, we’re mostly interested in well-behaved functions that are used in real-world engineering, so we won’t discuss the other cases. Another assumption commonly made for real-world functions is that they vanish (tend to 0) as |x|\rightarrow\infty . While this is not a direct outcome of absolute integrability [5] , it’s a reasonable assumption in engineering. After all, real-world signals have finite energies. Intuitively, when we also assume is uniformly continuous , the assumption of vanishing at |x|\rightarrow\infty is a logical conclusion, because otherwise how can the total area for |f(x)| be finite? An important outcome of this discussion is that the Fourier transform is unsuitable for periodic functions. Functions that repeat at intervals are not absolute integrable . For periodic functions, we use Fourier series. The Fourier transform is a linear operator, because the integral is linear: So is the inverse Fourier transform; it’s similarly easy to show that: If we scale the domain of a function by a constant, its transform changes only slightly: Let’s do the variable substitution u=ax : This is the Fourier transform evaluated at \frac{w}{a} , so: There’s one small caveat here; when a is negative, the integral bounds should be flipped, causing a minus sign in front of the transform. So we can write: Which works for any a\ne 0 . This property is intuitive when thinking about signals: suppose a>0 , then f(ax) means the signal is compressed in the time domain by a factor a . The scaling property says that the frequency domain is expanded using the same factor; in other words, the higher frequencies become more prominent because we need sharper transitions to represent the compressed signal. Time shifting What happens to the Fourier transform if we time-shift the input signal by some constant: f(x-x_0) . By definition: Substituting u=x-x_0 , we get du=dx , so: Transform of a derivative An extremely useful property that’s often employed in the solution of partial differential equations; let’s calculate the Fourier transform of the derivative of : We’ll use integration by parts, where dv=f'(x) and u=e^{-i\cdot wx} . Therefore, v=f(x) and du=-iw\cdot e^{-i\cdot wx} : Recall the assumption made in the "Existence condition..." section about vanishing at infinities. So the first part of the equation above is zero, and we’re left with: Transform of convolution The convolution between two continuous functions and g(x) is defined as: Let’s calculate the Fourier transform of this function: This step of combining the integrals into a double integral, as well as the next step (changing the order of integration) is possible due to Fubini’s theorem and our assumption that and g(x) are Lebesgue integrable. Switch order of integration: Now, f(\xi) in the inner integral doesn’t depend on x , so we can pull it out: The inner integral is just the Fourier transform of a time-shifted g(x-\xi) , so we can write: And the remaining integral is the Fourier transform of , so: Convolution in the time domain translates to multiplication in the frequency domain! This result is so important in signal processing that it’s called the convolution theorem . Suppose we have some function and we want to know the area bounded between this function’s graph and the x axis in a certain interval [a,b] . One way to do this is to take a partition of the interval: And calculate the area under for every element of the partition. We can then approximate such sub-areas by rectangles, as follows: We’ll denote the area of each rectangle as f(x^*_i)\cdot\Delta x : There are many ways to choose which point of the interval [x_{i-1},x_i] to denote as x^*_i : left point ( x_{i-1} ), right point ( ), mid-point between the two (which is what our plot shows) or anything in between. The distinction doesn’t really matter for our purpose, as we will soon see. We can approximate the area under the curve of in the interval [a,b] with the Riemann sum , using a uniform partition: If is continuous on [a,b] , then as n\rightarrow \infty : This is known as the Riemann integral , or just the definite integral. The limit is why the exact choice of x^*_i doesn’t matter: as n\rightarrow\infty we have \Delta x\rightarrow 0 , and all points within [x_{i-1}, x_i] are equally good. The Fourier series is a great tool for analyzing periodic functions. But what about functions that don’t repeat? We’ve seen that we can compute Fourier series for a non-periodic function defined on a finite interval, as long as we don’t care about its behavior beyond that interval. Let’s extend this idea to functions that never repeat; that is, non-periodic functions defined on the interval (-\infty,\infty) . Visualizing Fourier series for non-repeating functions To motivate the subject ahead, let’s look back at the example used in the earlier post about Fourier series : \[t(x)= \begin{cases} x & 0 \leq x \leq 1 \\ 2-x & 1 < x \leq 2 \\ \end{cases}\] With an odd extension into [-2,0] . In that post, to make the Fourier series work, we assumed t(x) keeps repeating with a period 2L=4 on the entire x axis. Here, let’s face the reality that it does not - in fact - repeat, and observe how our Fourier series work out. Recall that the Fourier series approximating t(x) are the sine series (since it’s an odd function): \[t(x)=\frac{8}{\pi^2}\bigg[ sin\frac{\pi x}{2}-\frac{1}{3^2} sin\frac{3\pi x}{2}+\frac{1}{5^2}sin\frac{5\pi x}{2}-\cdots\bigg]\] The following visualization is interactive. By default, it shows t(x) (with its odd extension) and no Fourier series approximation. We’ll proceed by a series of steps and observe the outcome: n (terms in the Fourier series) L x min x max Step 1 : set to some non-zero number; already at 3, the approximation is very good. The frequency spacing is \frac{\pi}{L} (this is the coefficient of x in the sines). Note that the Fourier series repeats every 2L , as expected. Step 2 : increase L to 6. This means our series are constructed assuming t(x) has a period of 12, not 4. Note how the Fourier series look now - they repeat every 12, and they don’t match t(x) as well as before. We can increase to a higher number to make the match better. As L grows, the spacing between adjacent frequencies decreases. Step 3 : increase L to 10. We no longer see the repetitions, so feel free to increase the values of x min and x max until you do. Note again that we need to add more and more coefficients to match t(x) better with this larger L , and the spacing adjacent frequencies grows smaller. Increasing L means our function repeats at larger and larger intervals. The logical conclusion of this progression is to ask - what happens if the function never repeats, meaning L\rightarrow\infty ? While not mathematically rigorous, the visual experiment here lets us make some conjectures: we’ll likely need an infinite number of coefficients for a good approximation, and moreover, the spacing between these coefficients will tend to zero. In other words, instead of a discrete set of coefficients, we’ll end up with a continuous line, or function . The function produced by this process is the Fourier transform of t(x) , and the next section shows its mathematical derivation. Fourier series with L\rightarrow\infty leading to Fourier transform In these notes, we’ll be using the complex exponential formulation of Fourier series: \[f(x)=\sum_{n=-\infty}^{\infty}C_n\cdot e^{in\pi x/L}\] With: \[C_n=\frac{1}{2L}\int_{-L}^{L}f(x)e^{-in\pi x/L}dx\] We’re interested in a non-periodic defined on the interval (-\infty,\infty) . So we’ll be exploring the above equations for L\rightarrow\infty . First, let’s make a slight change of notation. Instead of writing formulae in terms of the period ( 2L ), we’ll be using the n-th harmonic angular frequency w_n : \[w_n=\frac{n\pi}{L}\] So we can slightly rewrite our series as: \[f(x)=\sum_{n=-\infty}^{\infty}C_n\cdot e^{i w_n x}=\sum_{n=-\infty}^{\infty}C_n\cdot e^{i\cdot n \Delta w x}\] Using \Delta w as the difference between two consecutive frequencies: \[\Delta w=w_n-w_{n-1}=\frac{n\pi}{L}-\frac{(n-1)\pi}{L}=\frac{\pi}{L}\] Using this notation, C_n is expressed as: \[C_n=\frac{\Delta w}{2\pi}\int_{-\pi/\Delta w}^{\pi/\Delta w}f(x)e^{-i\cdot n \Delta w x}dx\] So far there are no new insights here, just some new notation. Now we’re going to use it to facilitate the next step. Since L\rightarrow \infty , then \Delta w\rightarrow 0 . Let’s calculate the limit of the Fourier series representation of when \Delta w\rightarrow 0 : \[f(x)=\lim_{\Delta w\rightarrow 0}\sum_{n=-\infty}^{\infty}C_n\cdot e^{i\cdot n \Delta w x}\] And substitute the latest C_n into this equation, changing its dummy integration variable from x to t to avoid confusion [1] \[f(x)=\lim_{\Delta w\rightarrow 0}\sum_{n=-\infty}^{\infty}\left[\frac{\Delta w}{2\pi}\int_{-\pi/\Delta w}^{\pi/\Delta w}f(t)e^{-i\cdot n \Delta w t}dt\right]\cdot e^{i\cdot n \Delta w x}\] Reordering slightly, and also replacing n\Delta w by w_n in the complex exponents: \[f(x)=\frac{1}{2\pi}\lim_{\Delta w\rightarrow 0}\sum_{n=-\infty}^{\infty}\left[\int_{-\pi/\Delta w}^{\pi/\Delta w}f(t)e^{-i\cdot w_n t}dt\right]\cdot e^{i\cdot w_n x}\Delta w\] Looking at the limit with the sum carefully, this is a Riemann sum (see Appendix A)! w_n is the "sampled" version of , and \Delta w\rightarrow 0 . We can therefore replace it by an integral, changing w_n to and \Delta w to dw [2] : \[f(x)=\frac{1}{2\pi}\int_{-\infty}^{\infty}\left[\int_{-\infty}^{\infty}f(t)e^{-i\cdot wt}dt\right]\cdot e^{i\cdot w x}dw\] The inner integral is called the Fourier transform of and denoted [3] : \[\boxed{\hat{f}(w)=\mathcal{F}\left[f(x)\right]=\int_{-\infty}^{\infty}f(x)e^{-i\cdot wx}dx}\] And the full equation for is then the inverse Fourier transform: \[\boxed{f(x)=\mathcal{F}^{-1}\left[\hat{f}(w)\right]=\frac{1}{2\pi}\int_{-\infty}^{\infty}\hat{f}(w)e^{i\cdot w x}dw}\] Example calculation of Fourier transform Let’s take our favorite odd triangular pulse example and calculate its Fourier transform. The function’s mathematical definition and plot are shown earlier in this post. Note that we’re not extending this function periodically - it’s zero beyond the range [-2,2] ; this is exactly why we need the Fourier transform here - as we’ve seen, Fourier series won’t do because the function they reconstruct eventually starts repeating. We’re looking to find: \[\hat{t}(w)=\int_{-\infty}^{\infty}t(x)e^{-iwx}dx\] To calculate the integral, let’s decompose the complex exponent using Euler’s formula: \[\hat{t}(w)=\int_{-\infty}^{\infty}t(x)cos(wx)dx-i\int_{-\infty}^{\infty}t(x)sin(wx)dx\] Since our t(x) is odd, the first integral is zero . Also t(x)sin(wx) is even, so we can write: \[\hat{t}(w)=-2i\int_{0}^{\infty}t(x)sin(wx)dx\] We’ve already calculated a very similar integral in the post on Fourier series , so let’s just skip to the result: \[\hat{t}(w)=-2i\cdot\frac{2\cdot sin(w)-sin(2w)}{w^2}\] The only remaining difficulty is its value at 0, which seems undefined at first (division by zero). However, note that as w\rightarrow 0 , the numerator also tends to 0, so we can use L’Hopital’s rule (twice!) to find that: \[\lim_{w\rightarrow 0} \hat{t}(w)=0\] Therefore: \[\hat{t}(w)= \begin{cases} -2i\cdot\frac{2\cdot sin(w)-sin(2w)}{w^2} & w\neq 0 \\ 0 & w=0 \\ \end{cases}\] This function is complex-valued; in fact, it’s purely imaginary. How do we visualize it? A common way to visualize complex-valued functions is by plotting their magnitude and phase separately. The magnitude of \hat{t}(w) is: \[|\hat{t}(w)|=\sqrt{\hat{t}(w)\cdot\hat{t}(w)^*}=2\left|\frac{2\cdot sin(w)-sin(2w)}{w^2} \right|\] Since \hat{t}(w) is purely imaginary, there are only two options for the phase: When the numerator is positive, we get a negative imaginary number with phase -\pi/2 , and when the numerator is negative, we get a positive imaginary number with phase \pi/2 . Finally, when \hat{t}(w)=0 (which happens at w=0 , by our earlier analysis, but also whenever is a whole multiple of \pi ), the phase is undefined. Here’s the magnitude and phase of \hat{t}(w) plotted against : It is common to talk about \hat{t}(w) as the frequency domain representation of t(x) . The frequency domain representation of functions When the functions we’re working with have time as their domain (e.g. the x in t(x) represents time), which is often the case in the study of signals and systems, the Fourier transform can be seen as computing the frequency domain representation of the function. Here’s the Fourier transform formula again: \[\hat{f}(w)=\mathcal{F}\left[f(x)\right]=\int_{-\infty}^{\infty}f(x)e^{-i\cdot wx}dx\] It takes - the time domain representation of a function, and converts it to \hat{f}(w) - a frequency domain representation. For well-behaved functions, these two representations are dual - each one describes the function completely, just in a different way. To convert back from a frequency domain representation to the time domain, we use the inverse Fourier transform: \[\mathcal{F}^{-1}\left[\hat{f}(w)\right]=\frac{1}{2\pi}\int_{-\infty}^{\infty}\hat{f}(w)e^{i\cdot w x}dw\] While a time-domain plot ( t(x) ) shows how a signal changes over time, a frequency-domain plot ( \hat{t}(w) ) shows how the signal is distributed across all possible frequencies. Moreover, as we’ve seen, \hat{t}(w) is complex valued. Each frequency therefore has both a magnitude and a phase: the magnitude tells us how strongly that frequency contributes, while the phase tells us how that component is shifted. The frequency domain is extremely useful in signal analysis; for example, when designing filters. The Fourier transform also has a number of properties that are very useful in signal analysis and processing. But first, let’s discuss what a "well-behaved function" means for the purpose of applying Fourier transforms. Existence condition for the Fourier transform The simplest existence condition for Fourier transforms is absolute integrability (also known as Lebesgue integrable): \[\int_{-\infty}^{\infty}|f(x)|dx<\infty\] With this condition, \hat{f}(w) exists on the entire domain, is continuous and vanishes (tends to 0) as |w|\rightarrow\infty [4] . While this condition is sufficient, it’s not necessary; there are less well-behaved functions that also have Fourier transforms defined with some limitations. In these notes, we’re mostly interested in well-behaved functions that are used in real-world engineering, so we won’t discuss the other cases. Another assumption commonly made for real-world functions is that they vanish (tend to 0) as |x|\rightarrow\infty . While this is not a direct outcome of absolute integrability [5] , it’s a reasonable assumption in engineering. After all, real-world signals have finite energies. Intuitively, when we also assume is uniformly continuous , the assumption of vanishing at |x|\rightarrow\infty is a logical conclusion, because otherwise how can the total area for |f(x)| be finite? An important outcome of this discussion is that the Fourier transform is unsuitable for periodic functions. Functions that repeat at intervals are not absolute integrable . For periodic functions, we use Fourier series. Some useful properties of Fourier transforms Linearity The Fourier transform is a linear operator, because the integral is linear: \[\begin{aligned} \mathcal{F}\left[\alpha f(x)+\beta g(x)\right]&=\int_{-\infty}^{\infty}\alpha f(x)e^{-i\cdot wx}dx+\int_{-\infty}^{\infty}\beta g(x)e^{-i\cdot wx}dx\\ &=\alpha\int_{-\infty}^{\infty}f(x)e^{-i\cdot wx}dx+\beta\int_{-\infty}^{\infty}g(x)e^{-i\cdot wx}dx\\ &=\alpha\mathcal{F}\left[f(x)\right]+\beta\mathcal{F}\left[g(x)\right] \end{aligned}\] So is the inverse Fourier transform; it’s similarly easy to show that: \[\mathcal{F}^{-1}\left[\alpha\hat{f}(w)+\beta\hat{g}(w)\right]= \alpha\mathcal{F}^{-1}\left[\hat{f}(w)\right]+\beta\mathcal{F}^{-1}\left[\hat{g}(w)\right]\] Scaling If we scale the domain of a function by a constant, its transform changes only slightly: \[\mathcal{F}\left[f(ax)\right]=\int_{-\infty}^{\infty}f(ax)e^{-i\cdot wx}dx\] Let’s do the variable substitution u=ax : \[\mathcal{F}\left[f(ax)\right]=\frac{1}{a}\int_{-\infty}^{\infty}f(u)e^{-i\cdot \frac{wu}{a}}du\] This is the Fourier transform evaluated at \frac{w}{a} , so: \[\mathcal{F}\left[f(ax)\right]=\frac{1}{a}\hat{f}\left(\frac{w}{a}\right)\] There’s one small caveat here; when a is negative, the integral bounds should be flipped, causing a minus sign in front of the transform. So we can write: \[\mathcal{F}\left[f(ax)\right]=\frac{1}{|a|}\hat{f}\left(\frac{w}{a}\right)\] Which works for any a\ne 0 . This property is intuitive when thinking about signals: suppose a>0 , then f(ax) means the signal is compressed in the time domain by a factor a . The scaling property says that the frequency domain is expanded using the same factor; in other words, the higher frequencies become more prominent because we need sharper transitions to represent the compressed signal. Time shifting What happens to the Fourier transform if we time-shift the input signal by some constant: f(x-x_0) . By definition: \[\mathcal{F}\left[f(x-x_0)\right]=\int_{-\infty}^{\infty}f(x-x_0)e^{-i\cdot wx}dx\] Substituting u=x-x_0 , we get du=dx , so: \[\begin{aligned} \mathcal{F}\left[f(x-x_0)\right]&=\int_{-\infty}^{\infty}f(u)e^{-i\cdot w(u+x_0)}du\\ &=e^{-iwx_0}\int_{-\infty}^{\infty}f(u)e^{-i\cdot wu}du\\ &=e^{-iwx_0}\mathcal{F}\left[f(x)\right] \end{aligned}\] Transform of a derivative An extremely useful property that’s often employed in the solution of partial differential equations; let’s calculate the Fourier transform of the derivative of : \[\mathcal{F}\left[f'(x)\right]=\int_{-\infty}^{\infty}f'(x)e^{-i\cdot wx}dx\] We’ll use integration by parts, where dv=f'(x) and u=e^{-i\cdot wx} . Therefore, v=f(x) and du=-iw\cdot e^{-i\cdot wx} : \[\mathcal{F}\left[f'(x)\right]=\left[f(x)e^{-i\cdot wx}\right]^{\infty}_{-\infty}-\int_{-\infty}^{\infty}f(x)(-iw\cdot e^{-i\cdot wx})dx\] Recall the assumption made in the "Existence condition..." section about vanishing at infinities. So the first part of the equation above is zero, and we’re left with: \[\begin{aligned} \mathcal{F}\left[f'(x)\right]&=-\int_{-\infty}^{\infty}f(x)(-iw\cdot e^{-i\cdot wx})dx\\ &=iw\int_{-\infty}^{\infty}f(x)e^{-i\cdot wx}dx\\ &=iw\cdot\mathcal{F}\left[f(x)\right] \end{aligned}\] Transform of convolution The convolution between two continuous functions and g(x) is defined as: \[(f\ast g)(x)=\int_{-\infty}^{\infty}f(\xi)g(x-\xi)d\xi\] Let’s calculate the Fourier transform of this function: \[\begin{aligned} \mathcal{F}\left[(f\ast g)(x)\right]&=\int_{-\infty}^{\infty}e^{-i\cdot wx}\left[\int_{-\infty}^{\infty}f(\xi)g(x-\xi)d\xi\right]dx\\ &=\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}e^{-i\cdot wx}f(\xi)g(x-\xi)d\xi\ dx \end{aligned}\] This step of combining the integrals into a double integral, as well as the next step (changing the order of integration) is possible due to Fubini’s theorem and our assumption that and g(x) are Lebesgue integrable. Switch order of integration: \[\mathcal{F}\left[(f\ast g)(x)\right]=\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}e^{-i\cdot wx}f(\xi)g(x-\xi)dx\ d\xi\] Now, f(\xi) in the inner integral doesn’t depend on x , so we can pull it out: \[\mathcal{F}\left[(f\ast g)(x)\right]=\int_{-\infty}^{\infty}f(\xi)\int_{-\infty}^{\infty}e^{-i\cdot wx}g(x-\xi)dx\ d\xi\] The inner integral is just the Fourier transform of a time-shifted g(x-\xi) , so we can write: \[\mathcal{F}\left[(f\ast g)(x)\right]=\int_{-\infty}^{\infty}f(\xi)e^{-i\cdot w\xi}\mathcal{F}\left[g(x)\right]d\xi=\mathcal{F}\left[g(x)\right]\int_{-\infty}^{\infty}e^{-i\cdot w\xi}f(\xi)d\xi\] And the remaining integral is the Fourier transform of , so: \[\mathcal{F}\left[(f\ast g)(x)\right]=\mathcal{F}\left[f\right]\cdot\mathcal{F}\left[g\right]\] Convolution in the time domain translates to multiplication in the frequency domain! This result is so important in signal processing that it’s called the convolution theorem . Appendix A: Riemann sum and the definite integral Suppose we have some function and we want to know the area bounded between this function’s graph and the x axis in a certain interval [a,b] . One way to do this is to take a partition of the interval: \[a=x_0<x_1<\cdots<x_{n-1}<x_n=b\] And calculate the area under for every element of the partition. We can then approximate such sub-areas by rectangles, as follows: We’ll denote the area of each rectangle as f(x^*_i)\cdot\Delta x : \Delta x=(b-a)/n is the width of one interval (assuming a uniform partition, but the math works just as well for non-uniform ones). x^*_i is some value in the interval [x_{i-1},x_i] .

Tutorial Science JavaScript

▲ 0

0 views

Hugo 3 months ago

AI and Ecology, Fantasy or Convenient Scapegoat?

It's hard to talk about AI these days. I've rarely, if ever, seen a subject so polarized in tech. You could tell me I have a short memory. The internet sparked plenty of criticism around the destruction of brick-and-mortar retail, print media, and the end of human interaction. Same for mobile, with added, legitimate reproaches about addiction and the ease of surveilling individuals. We could also mention crypto, a massive Ponzi scheme for some, a way to reclaim power from central banks for others. And yet, with AI, I feel like we've crossed a threshold. There would be only two possibilities: Pick a side, friend, and if you don't, others will do it for you. The "safest" bet is not to talk about it at all, but burying my head in the sand feels cowardly, if not impossible when you work in tech. Simply put, I need to stick my head out and try this exercise without resorting to clichés. And since we're in the middle of a heat wave, it seems obvious that the first subject to address is ecology. Is AI as catastrophic as people say? Is the impact of an AI query truly astronomically higher than a Google search? How does it compare with other digital uses? Let's take some time to look at all this. First, let's establish some basics about what we call ecological impact. This impact falls into several categories: To keep things simple, an AI consumes energy at two distinct stages: during training (when the model is created, like Gemini, Llama, Claude, etc.) and during inference (when users actually query the model). When looking at carbon footprints, models vary wildly, but the estimated range for training a single major model sits between 500 and 12,000 tonnes of CO2 equivalent. To put that into perspective: ::callout{type=primary} The massive gap between US and French household equivalents stems from the fact that France’s energy mix relies heavily on nuclear power, which is virtually carbon-free. :: Operational consumption is another moving target. It depends on the complexity of the prompt, the location of the data center (and its corresponding energy mix), the model being used, and so on. But it's estimated to vary between 0.03g , and 1g of CO2 per request. We'll see below how that compares with internet, gaming, streaming etc… To talk about water consumption, we must first address a common misconception: No, we don't destroy water . Earth’s water operates in a closed loop. When water is used in a cooling system, whether in a data center, nuclear power plant or anything else, it's not destroyed. When water evaporates, it eventually falls back as rain. However , evaporation causes water to displace. If water moves more than 800km, the region where it was drawn from has effectively lost it, temporarily but lost nonetheless. In ecology, we distinguish water withdrawal (borrowing water and returning it to the same place after use) and water consumption (drawing water and evaporating or releasing it elsewhere, making it unavailable locally). AI consumes water. On a planetary scale , it's not necessarily a problem. On a local scale , however, it can trigger severe water stress, creating direct competition between residents, agriculture, and data centers. To be fair, technology is advancing. The majority of new data center projects use closed-loop water systems so water isn't evaporated. Some countries (Ireland, Sweden, Finland) take advantage of their cold climate to reduce water needs by 90% and we see other systems emerging. But to look at the flip side, the vast majority of existing data centers use evaporation systems and in any case, these systems require electricity which creates tensions, for example in Sweden or Ireland. Now that we've said all that, what's the consumption for evaporation data centers? Training a recent model is estimated to consume approximately 40 to 80 million liters (a small lake). In a water-stressed region, that can make a difference. And if we look at usage, for a request, it's between 2 and 6.5ml of water per request. ::callout{type=warning} This section is the trickiest for me because it’s the one I’m least familiar with, and honestly, it probably deserves an entire article of its own. So, while we will only scratch the surface here, I promise to dive much deeper into this specific topic in a future post. :: We often focus on electricity and water, but the environmental footprint of mining is one of AI's biggest blind spots. To run AIs or train models, you need ultra-powerful equipment and colossal infrastructure that will require copper, aluminum, cobalt, lithium, nickel, rare earths and I imagine I'm forgetting some. Well, these resources are in limited quantities on earth but I'll discuss that in a future article, recycling in this sector is currently negligible but moreover, the extraction itself is extremely polluting. To make matters worse, we must add that current equipment becomes obsolete much faster. In the AI race, we replace equipment much faster. Certainly, new equipment is more efficient, particularly in terms of energy but this ultra-rapid rotation creates a volume of electronic waste we don't know how to manage. Despite everything, I don't yet know from which angle and with which figures to illustrate all this, especially since these subjects also pull along many other geopolitical subjects (tension over Taiwan, tension over rare earths etc…), so we'll set that aside for future publication. We already have plenty to do with the first two subjects. With these orders of magnitude in mind, is AI " stratospherically " different from the rest? How does it compare with a Google search for example? Or with streaming, video call, an online video game? By comparison, a query to a search engine (Google) is approximately 0.2g of CO2 . Depending on the complexity of the question and the model used, an AI prompt can cost slightly less than a Google search, or up to five times more . So, it is not "stratospherically" higher than a standard web search. Furthermore, if a topic requires you to do multiple Google searches and open several websites to find your answer, the gap quickly narrows, and can even reverse. But we must separate simple uses: "give me the strawberry pie recipe", from complex uses: "analyze this PDF document of several megabytes for me and create an application that displays results with charts". I propose we do an exercise and compare 1 hour of streaming, 1 hour of gaming, 1 hour of video call, and one hour of AI-assisted software development (a relatively power-consuming use). ::callout{type=primary} Why such strong variations when considering AI-assisted development? Because it encompasses vastly different habits. Consumption will be drastically different between an "amateur" coder copy-pasting a few lines from a browser, a "pro" user partially delegating tasks within their code editor, and an "intensive" power-user running automated tools where code generation is almost entirely outsourced. :: In other words: No matter how you look at the data, it is hard to find evidence of a "stratospheric" gap. And to go further, we could look at the impact of AI model creation compared to the ecological impact of creating a video game, or a movie. An AAA video game (big-budget), developed by a team of 150 people, costs between 500 and 3,000 tonnes of CO2 depending on development time, travel, and motion-capture filming. To this, we must add the annual maintenance for live-service games that push out continuous updates and DLCs (like World of Warcraft or Overwatch ). For a big budget film, we can estimate a carbon cost between 3,000 and 4,000t of CO2 , including transport, filming locations, generators, set construction. Granted, training a massive AI model can cost more than a single movie, but the difference isn't orders of magnitude apart. More importantly, we must remember that the world releases thousands of films and video games every year , whereas the creation of new foundational AI models remains relatively rare. Let's be careful here. It would be lazy whataboutism to simply say, "Sure, AI is bad, but look at how much worse everything else is." That is missing the point. The real goal here is to question our consumption habits as a whole. What is certain, however, is that the reality is far more nuanced than the mainstream narrative suggests. Today, the hyper-focus on AI serves as a very convenient distraction, allowing us to forget the environmental cost of our other digital habits. But you don't earn moral virtue points by campaigning against AI while actively indulging in online gaming, streaming blockbusters, or flying to international sports events. If you've followed the numbers well, the ecological impact of AI is relatively close to other impacts in digital (streaming and gaming for example). That doesn't mean it's good. In the world we live in, each additional tension on the planet is to be questioned . But it forces us to realize that all of our digital behaviors need to be reassessed, not just the fact that "I asked ChatGPT a question." I don't pretend to be able to rank these activities against one another. Comparing gaming, streaming, and professional workloads is highly complex. And even within professional uses, And I certainly won't decide, on my own , what constitutes a "good" or "bad" use of technology. But collectively, we might soon be forced to make those choices , not out of kindness, but by constraints (See next chapter). The core issue isn't about outright banning AI. This is precisely what organizations like Shift Project , France’s leading think tank on the energy transition, are trying to convey: we need to look at data volumes and digital use cases in their entirety. The argument isn't that we should abandon AI altogether, but rather that we cannot afford its current, unchecked trajectory Let's take an example: the FIFA World Cup generates between 9 and 15 million tonnes of CO2, which is roughly equivalent to the annual energy consumption of all US data centers combined.. Again, the idea isn't to say, they do worse. We'll get nowhere with that mindset. But I like this example because of the contrast it highlights. Playing football doesn't cost much. Gathering thousands of people across 3 countries and having them fly everywhere is absurd, as is air-conditioning football stadiums, or trying to organize winter games in a desert country. AI operates on the exact same spectrum. There is a massive gulf between a professional, high-utility application, like using AI in biochemistry, mathematics, meteorology, drug discovery, medical imaging, satellite analysis, or precision agriculture, and a purely recreational use aimed at generating thousands of Ghibli-style images just to dump them on social media. Yes, we can, and should, question the latter (and that’s an understatement). Ultimately, understanding these orders of magnitude is what empowers us to make informed choices instead of just parroting the absurdities we hear on TV. Once you know the real numbers, you can weigh your choices accurately. I said earlier that one hour of video call was between 30 and 60g of CO2. Ok, but that might replace a Paris Lyon trip. By car it's between 60 and 90kg of CO2 saved. By train it's about 1kg. Similarly, one hour of streaming costs about 100g of CO2. But if it prevented you from driving 20km to the local movie theater (which would cost around 4.4kg of CO2 in car emissions), streaming turns out to be "not so bad" after all. In the end, once we have the data, it is up to each of us to make those choices. A question I asked myself before writing this article was: If the carbon footprint of AI is actually pretty close to our other digital habits, and assuming it replaces some of them (if I’m using AI, I’m not doing something else), why on earth are we building so many new data centers? Ok, this question might seem naive but it's estimated that data center electricity consumption could double, or even triple by 2030 (See BCG study and this IRIS article). So why? Is it linked to AI? According to articles, partly yes, but only partly. The majority of electricity consumed by data centers (about 2/3) should be dedicated to historical digital uses and acceleration of cloud migrations. Yes AI plays a role, but it's mainly that digital is taking up more and more space. The share of digital in global CO2 emissions went from 2% in 2010 to about 4% today, with an annual increase of about 6% even though the global objective is to reduce our emissions by 5% per year to hope to ++stabilize++ the climate . Where AI genuinely worsens the problem compared to other tech is its rapid hardware obsolescence. However, the root cause is the massive scale-up of all our digital habits: the ubiquity of 4K/8K streaming, cloud gaming, high-fidelity music streaming, and the explosion of connected IoT (Internet of Things) devices. Of course, we should take these data center growth forecasts with a grain of salt. They remain predictions. They could easily be overestimated, just like the predicted "tidal wave" of data that was supposed to arrive with 5G but never quite materialized. Many of these projections are pushed by tech giants that have bet their entire financial futures on these exact growth scenarios. If you are Nvidia, Google, or Oracle, you have no choice but to reassure your shareholders by guaranteeing this growth will happen to justify the colossal investments already made. Honestly, if the AI financial bubble were to burst tomorrow, it might actually be good news for the planet, as it would instantly ease the pressure on our resources. That being said, we are looking at contracts that are already signed, budgets locked in, and massive public announcements, like the Stargate project or Europe's future investments . Every current scenario predicts a 2x to 3x increase in demand. Digital consumption is going up, and AI-related infrastructure is leading the charge. Will these digital habits replace physical ones (like my earlier example of a video call replacing a car trip)? Or are they purely additive ? Evidence suggests they are additive. While some AI applications will certainly accelerate decarbonization in specific industries, that impact remains relatively marginal for now. And, according to the Shift Project, it's mainly that the electrical consumption needed to run all these data centers will exceed our infrastructure capacity, and thus force using thermal sources (gas power plant) to compensate or create usage conflicts . So yes, it's alarming. Again, it's not about banning AI, the subject is much more global than that. How do we make our consumption and pressure on the planet decrease? At our individual scale, we have to audit our own habits. We need to question our obsession with upgrading devices, over-equipping our homes, and engaging in mindless, heavy recreational uses (like generating endless Ghibli images just for a laugh). At a collective level, we will eventually be forced, likely much sooner than we think, to make hard choices. We will have to decide whether to route precious electricity to a data center or to power and heat local homes . But let’s remember one crucial thing: the future isn’t set in stone. If we collectively choose to consume less, we won't need the electricity these companies are dying to sell us. If data centers are multiplying, it's only because there is a planned demand for them. To paraphrase a famous French comedian: "To think that if people just stopped buying, it wouldn't sell anymore!" The Doomers (declinists) who envision an ecological apocalypse, total destruction of employment and placing public opinions under the guardianship of Big Tech controlling AI. And the Bloomers (accelerationists), who advocate blind faith in progress, convinced that AI will liberate humanity, eradicate diseases and generate infinite growth, and for whom slowing research is the real crime. Electricity consumption (which translates into CO2 emissions) Water consumption for cooling data centers Resource extraction, required to build the data centers and user devices themselves There is only a 2x ratio between gaming and professional AI-assisted development. video call is what consumes the least Streaming is remarkably close to professional AI development usage

Programming AI Science

▲ 0

0 views

Eli Bendersky's website 3 months ago

Dot product: Component vs. Geometric definition

The goal of this post is to answer a simple question: why are the following two definitions of the vector dot product in Euclidean space [1] equivalent for vectors \vec{a} and \vec{b} : Here’s a graphical depiction of our vectors (focusing on for clarity, though this applies to any-dimensional vectors). It shows both the components of the vectors and the angle between them. The length of the arrow for \vec{a} is |\vec{a}| . We’ll show two proofs of the equivalence here, the geometric proof and the projection proof . The Appendix describes some properties of dot products that facilitate these proofs. We’ll be using this diagram of our vectors \vec{a} and \vec{b} , as well as the vector \vec{c}=\vec{a}-\vec{b} : Using the law of cosines [2] on the triangle formed by the three vectors: Since for any vector \vec{a} , we have \vec{a}\cdot\vec{a}=|\vec{a}|^2 (see Appendix), let’s rewrite this equation as: But \vec{c}=\vec{a}-\vec{b} and the dot product obeys the distributive property (see Appendix). Therefore: For this proof, we’ll assume the geometric definition is correct and will see how it leads to the component definition. We’ll begin by denoting vectors \vec{e}_1,\vec{e}_2\dots\vec{e}_n as the standard orthonormal basis for . For example, in 2D space, these basis vectors are \vec{e}_1=[1\ 0] and \vec{e}_2=[0\ 1] , shown in this diagram: If we take an arbitrary \vec{a}\in\mathbb{R}^n and calculate its dot product with a basis vector, we can use the geometric definition: where a_i is the component of \vec{a} in the direction of \vec{e}_i . The diagram makes it easy to see why this is true from basic trigonometry, but in the more general case this is just a vector projection . Now let’s represent vectors \vec{a} and \vec{b} as linear combinations of the basis vectors: And calculate the dot product \vec{a}\cdot\vec{b} , beginning by rewriting \vec{b} with its linear combination of basis vectors representation: Using the fact that the dot product distributes over linear combinations: But earlier we’ve shown that \vec{a}\cdot\vec{e}_i=a_i . Therefore: Which is the component definition \blacksquare . A generalization of dot products in is the inner product , which is an operation meeting some specific requirements, defined on a vector space. The inner product is denoted as \langle x,y\rangle:\mathbb{R}^n\times\mathbb{R}^n\to\mathbb{R} , and must satisfy the following requirements for all vectors x,y,z\in\mathbb{R}^n and scalars a,b\in\mathbb{R} : For , we define the inner product operation in its component formulation as: Let’s prove the requirements listed above for this operation; this is fairly straightforward, given the well-known properties of scalar multiplication and addition on : Linearity in the first argument: Positive-definiteness: Consider the components of vector x . Clearly, \forall i\quad x_i\cdot x_i=x_i^2\ge 0 . Since the vector x is not the zero vector, at least one of its components is nonzero, and for that component x_i\cdot x_i>0 . Therefore: Now that we’ve proved all the inner product requirements on our operation \langle x,y\rangle , we can say that is an inner product space with this operation. By meeting these requirements, it can be readily shown that our inner product operation has additional useful properties: The third property is particularly helpful, because it means the inner product is bilinear , and thus is distributive over addition. Note that these are shown for the component definition of dot product. It’s not too hard to prove distributivity for the geometric definition using the notion of projections and how they add up. The norm of a vector x in an inner product space is defined as |x|=\sqrt{\langle x,x\rangle} . Therefore, the square of the norm is |x|^2=\langle x,x\rangle . The norm is used to express the notion of magnitude , or length of a vector. If you think of a vector x\in\mathbb{R}^n in Cartesian coordinates, the definition of the norm is a generalization of the Pythagorean theorem. Component definition: \vec{a}\cdot\vec{b}=\sum_{i=1}^{n}a_i b_i Geometric definition: \vec{a}\cdot\vec{b}=|\vec{a}||\vec{b}|cos(\theta) , where |\vec{a}| is the magnitude of \vec{a} and is the angle between the vectors’ directions Symmetry: \langle x,y\rangle=\langle y,x\rangle Linearity in the first argument: \langle ax+by,z\rangle=a\langle x,z\rangle+b\langle y,z\rangle Positive-definiteness: if x\ne 0 then \langle x,x\rangle>0 \langle x,0\rangle=\langle 0,x\rangle=0 \langle x,x\rangle=0 if and only if x=0 \langle x,ay+bz\rangle=a\langle x,y\rangle+b\langle x,z\rangle \langle x+y,x+y\rangle=\langle x,x\rangle+2\langle x,y\rangle+\langle y,y\rangle

Science Mathematics Programming

▲ 0

0 views