Yeah, not gonna mess app servers. I do find the JVM was kind of made for a different era though. It’s basically designed to act like a VM on top of which all your apps run. So, startup time isn’t really a problem, and it wants to grab as much memory as it can by default. But nowadays everybody just makes small self contained apps that you can scale horizontally, so this whole model the JVM is tailored for isn’t really used outside big enterprise. And Jenkins was alight, I used to use it back in the day too. For the most part, I do find GitHub actions are an improvement though. You just make a script and magic happens.

there’s certainly a good case for that too

same, would be fun if it took off

why would I want to interact with imbeciles more?

I recall looking at mold, but it didn’t end up helping much in my case. And haven’t looked at sccache, that might actually help if it can cache compilation incrementally in an intelligent way.

I mean if it’s going against Anthropic pricing that’s steal a good deal.

It’s around 100k loc sized project, so from what I’ve seen that’s about what you can expect with Rust.

d’aww so adorable of you to start projecting

On a related note, it turns out that CO2 emissions also impair cognition. So, by the time the crisis peaks everybody is going to be too cognitively impaired to actually do anything about it.

https://www.colorado.edu/mechanical/2020/04/21/continued-co2-emissions-will-impair-cognition

Says dumb shit in public, acts all offended when called out on said dumb shit. Go clutch your pearls elsewhere loser.

indeed

Right, you basically have to start building a rube goldberg machine for compiling your app just so you don’t have to wait for minutes on end. And to me it is absolutely bananas that you can make a compiler this slow on modern hardware. Coming from largely working with Clojure where I have a live application I can interactively load code into as I edit it, this whole process feels insane.

haha this reminds me of my days working with websphere :)

I’ve got a non trivial project in Rust, and it takes like 5 min to compile on my machine. Personally, I don’t know how anybody can call this fun. I find it insane to have to wait minutes to see the changes and to iterate. And like sure you can break shit up into crates to speed up compilation, but to do that you already have to have a design you’re happy with and that’s stable.

Instead of shrinking components, manufacturers have been focused on developing new hardware that is tailored to AI. This has provided occasional incremental improvements in performance, but none has come close to keeping up with the exponential curve of AI’s increasing demands. Ultimately, inefficiency may be of little concern to the people within the tech industry who believe that they are replicating intelligence itself. There is an almost-religious conviction among many in Silicon Valley that something mindlike could arise from LLMs, which are ultimately just statistical language-generating software—this, despite the software’s inability to recall basic facts, its lack of common sense, and its complete dissimilarity to a biological brain. Even Yann LeCun, one of AI’s “godfathers,” told The New York Times recently that “LLMs are not a path to superintelligence or even human-level intelligence.” But the mythological lure of AI is so strong that many engineers believe that nothing should stand in their way. Not even the basic task of writing efficient software.

Seems like might be problem on your end, perhaps your ISP is blocking it?

As they scramble to keep their systems online, AI companies are making things expensive for the rest of us. Large language models such as ChatGPT and Claude are so resource-hungry that tech companies may be purchasing 70 percent of the world’s supply of high-end computer memory, causing a shortage. As a result, the prices of computer memory and storage are skyrocketing: Hard drives that I bought for my reporting two years ago for $350 each were $800 when I checked two weeks ago, and are now out of stock. The prices of some laptops have gone up as much as 50 percent, and low-cost computers are being hit the hardest. Affordable entry-level computers may “disappear by 2028” according to one forecast. And the memory shortage is expected to continue for years. The memory is being put into data centers, which tech firms are expanding at incredible speed. They are planning to multiply total U.S.-data-center capacity by a factor of eight over the next few years. The demand for electricity at these sites is already so great that some companies are repurposing jet engines to power them. The problem is not simply that AI is being deployed so widely or quickly. Other computer technologies have seen similarly massive growth without triggering such a large spike in electricity or a shortage of computer components: Video and music are now streamed around the globe, accounting for many terabytes of internet traffic daily; the smartphone boom required the manufacturing of billions of devices that are now transferring huge amounts of data; billions of household devices are also now part of the Internet of Things; and whole industries have moved their operations to cloud software, which is hosted not in the sky but in, yes, data centers. Read: The $10,000 MacBook Pro is here The problem with generative AI, in the industry’s own jargon, is that it does not scale. The cost of growing from, say, a thousand users to a million is a key factor that venture capitalists examine when they evaluate start-ups. They want to see that the cost of adding each new user decreases over time, so that the company can support millions of users and make increasing profits. This is achieved partly through the careful engineering of computer systems that can efficiently handle more users who want to post photos, hail Ubers, or stream music. With generative AI, the work of building efficient, scalable systems has not been done. And the problem is exacerbated by the ever-larger generative-AI models, which have grown from 175 billion parameters in 2020 to more than 1 trillion today, according to independent estimates (the actual sizes of the models powering products such as Claude and ChatGPT are secret). The large in large language model should not be a selling point. But the industry’s observation that bigger models tend to outperform smaller ones has given rise to a totemic belief in “scaling laws” that suggest any problem can be solved by simply making models bigger. “Maybe with 10 gigawatts of compute, AI can figure out how to cure cancer,” OpenAI CEO Sam Altman wrote on his blog in September. Recommended Reading

A photograph of children on and around a dining-room table, making a mess
The Myth That Gets Men Out of Doing Chores
Joe Pinsker
Bottom half of woman's face with a reflection in a mirror
How Not to Be Your Own Worst Enemy
Becca Rashid and Arthur C. Brooks
An illustration of a husband and wife sitting on opposite sides of a bed that's floating in open water.
Dear Therapist: Can I Still Trust My Husband?
Lori Gottlieb

Yet the returns are diminishing. The bigger an AI model is, the less it improves with each added parameter, and so it must be made bigger at a faster rate just to sustain steady progress. I asked a few AI researchers whether they could name any other real-world software that scales so poorly. None of them could think of any. Even outside the world of software, it’s hard to find a comparable example, given that economy of scale is the principle that has made light bulbs, cars, and clothing so affordable. By economic and engineering measures, generative AI might be the worst technology ever deployed. Read: Welcome to a multidimensional economic disaster But with the massive investment behind the current bloated approach, there may not be much will to change. Ilya Sutskever, a co-founder and former chief scientist at OpenAI, said in a November interview that companies take the brute-force approach “because it gives you a very low-risk way of investing your resources.” It’s harder, he argued, to invest in research that would reengineer a product currently accruing trillion-dollar valuations. Those who suspect we are in an AI-driven bubble economy have pointed out that the profitability of these companies remains an open question, largely because of the high cost and inefficiency of the technology. Efficiency is a core principle of computer science. One of the first things undergraduates learn is that writing a program that sorts a list of 50 words is easy. But if you give that program 50 million words, it will likely run out of memory or take hours to finish. Much of computer science is learning the clever coding techniques that prevent this from happening. Many of these techniques take advantage of repeating patterns in the data so that as the program receives more input, it takes less time or memory to process each additional bit. Such efficiency is one reason that modern smartphones and computers are so capable and affordable. This is called logarithmic scaling, and it looks like this when you graph it: graph showing what tech companies want to see when they scale Large language models do not scale logarithmically. When they’re given more words to process, they get slower and use more memory—the time and resources increase faster as the input grows. In technical terms, LLMs scale quadratically. Any computer-science student knows that this is very bad. Epoch AI, an organization that tries to determine the costs of operating AI models, published a graph last year, which is reproduced here with permission. It shows the exponentially increasing costs of serving more “tokens”—the words users type to chatbots—with several public AI models. graph showing what AI companies are actually seeing as they scale LLMs AI does not have to be built this way. Traditionally, the goal of AI was to solve problems in ways that simulated human mental processes. Researchers observed their own thinking and tried to implement their mental habits in code. This approach has mostly been abandoned, partly due to the difficulty of discerning and articulating the rules of human thought, but it did have the benefit of consuming far fewer resources and data. Today’s approach to AI doesn’t try to describe the rules of human thought; instead, it gives a computer millions of examples to imitate. The huge quantity of examples is one reason that large models can perform better than small ones when generating language, images, and music—they have more material to draw from. Some researchers want to bring back the old, more efficient approach and combine it with the modern approach, but so far these projects have not drawn nearly as much attention or funding as the models that power chatbots. Chatbot companies are aware that their products are inefficient. Some have found techniques for improving performance, but they have not yielded significant gains. Occasionally, companies claim to have made breakthroughs—Anthropic CEO Dario Amodei has called them “compute multipliers”—but they are usually described in vague terms, and there is no evidence that the basic problems of quadratic scaling and exploding model size have been overcome. (Anthropic declined to comment on the record when I reached out to ask about this.) Some researchers are working on extremely small models that require fewer data and less computing power. I spoke with Alexia Jolicoeur-Martineau, an AI researcher at Microsoft who independently designed one of these, and asked her about the industry’s brute-force approach. “It’s a bit insane,” she told me. “At some point you have to learn to be a bit more efficient.” Last year, Jolicoeur-Martineau won a $50,000 prize for her paper on a “tiny recursive model” that doesn’t consume huge quantities of computing resources. “The idea that one must rely on massive foundational models trained for millions of dollars by some big corporation in order to achieve success on hard tasks is a trap,” she wrote. Her model isn’t a substitute for an LLM—it’s designed to solve logic problems in fields such as biology and electrical engineering, rather than generate language—but it can do some of the tasks that much larger AI models are currently being used for. Yet we seem stuck with LLMs, perhaps because they have been so aggressively marketed. They are now being added to everything, whether you want them or not. In 2024 and 2025, they were integrated into both Windows and MacOS, which means that more computing power is now needed to run a basic personal computer. Smartphones are also being sold with upgraded hardware as companies anticipate new AI features. Inefficient AI is also being added to common programs such as Adobe Photoshop and Microsoft Word, meaning that computers need to be more powerful to run this software. Ted Chiang: No, artificial intelligence is not conscious This is all especially bad because computers are no longer improving at the rate they used to. Since the 1950s, manufacturers have learned to make microchips steadily faster, smaller, and cheaper, a trend known colloquially as Moore’s Law. But in the past few years, components have gotten so small that manufacturers have run into molecular-level limitations on shrinking them any further, which has slowed progress significantly.

I think the big picture here is that the difference in quality is largely subjective at this point, while US companies are burning through orders of magnitude of cash which is obviously not sustainable. We shouldn’t underestimate the power of developing things in the open. Chinese open models benefit from the wisdom of an entire global research community while American engineers working on proprietary closed models are working in their own insular silos. It should be no surprise that the scientific community at large would pull ahead of these small teams. On top of that, doing research in the open amortizes the cost. Incidentally, this is exactly the same logic that led open source to dominate in recent years.

The energy consumption trope in particular hasn’t been true for a while now. You can literally run models on your laptop for most tasks, and these are models that have capability that needed a data centre literally less than a year go. Meanwhile, the whole notion of people offloading their cognitive capacity to models is based on a handful of studies with tiny samples. So yes, you are wrong, and you just run around uncritically repeating nonsense thinking you’re being really profound.

Thank you for taking your valuable time away from huffing gas to grace us all with your insights.