cross-posted from: https://feddit.org/post/24404539
If we combine these two sets of data^1 we obtain a fascinating result^2.
- 46% of all code out there, in every app, is maintained by hobbyists
- 13,8% is maintained by “I sometimes get a bit of pocket money for my code”
- 40% of all code out there is maintained by an industry-paid person
So, nearly 60% of all code being actively shipped in an app or product in the wild is hobbyist-maintained open-source.
See also this discussion on lobste.rs on the economics of the average (as in median) open source project:
https://lobste.rs/s/ftwkvo/hobbyist_maintainer_economic_gravity
To sum up, apparently most open source projects are small, and aren’t funded as paid work. And they matter because of their number, which has the effect that they make up a large part of all software in use.
The insight that a majority of open source projects are small contributions by hobby developers, and that it is their summed joint effort what matters, is very interesting.
This is part of a discussion that FLOSS development should be “funded better” to secure digital infrastructure.
What if the majority of contributors
(1) are not motivated by money in the first place, and
(2) don’t have time to work more?
Another thought: I think that one reason why the proportion of open source code grows is also software quality:
Companies would love to own all their code. So, when they employ people who work on proprietary code, the amount of proprietary code should grow, shouldn’t it?
Except that companies have mostly very short-term goals. And this affects quality: A lot of proprietary code has quite shit quality and is not really maintainable. Which has the effect that either the project dies, or becomes very slow to develop further, because of tons of technical debt.
FOSS projects do not have this constraint on short-term returns, so they often have better quality. Which makes it more likely that these projects live and prosper a bit longer. The short-term difference might not be even large - but the process goes year for year, round for round, and it becomes an evolutionary advantage.
In the end, everyone uses that Finnish students former hobby kernel project, and nobody uses Windows 95 - or wants to use its shitty successors.
(And this is why I also think that Guix will win in the long term: The capability to automatically re-produce all components of a program or system from freely available source is, in the long run, an overwhelming evolutionary advantage.)
The insight that a majority of open source projects are small contributions by hobby developers, and that it is their summed joint effort what matters, is very interesting.
The vast majority of open source projects are by hobby developers but how much of those projects make up the 77% of the open source included in apps mentioned in the study?
The author assumes an even distribution but I challenge that.
The most popular (Top listed by Github, Gitlab, etc) open source languages (python, typescript, etc), frameworks (rails, flutter, react, etc), and databases (postgres, mongo, redis etc) are all either directly corporately funded (Google, Microsoft, Meta, etc) and/or have robust foundations and sustainability plans.
I would expect these to make up the vast majority of the open source code in modern apps.
If an app includes 50 well-known big projects and 1000 small projects, the sum result can still be that small projects make up for a large fraction of the code.
It does not need to be an even distribution, it can be a “long tail” distribution.
And FOSS code is inherited often. Some years ago, a bug in a string-to-floating point connversion routine was found. This affected, if I remember correctly, PHP. But it turned out that many more languages were affected.
Similar with the TimSort algorithm, which was written by Python’s Tim Peters.
If an app includes 50 well-known big projects and 1000 small projects, the sum result can still be that small projects make up for a large fraction of the code.
I understand your point that this is possible. It is an assumption to assume it is most likely the case however.
I would expect the Fat Head of most used open source projects to make up the vast majority of the open source code included in apps. It is not a common practice to include 1000 small projects into a code base for an app, or even 100.
Is it not reasonable then to expect that the 77% of app code from open source is because the most popular app building blocks are open source? Aren’t the popular open source languages, frameworks, and databases are themselves big enough to exceed the number lines of internally written code for the app business logic most of the time?
For example, if I make a “small” electron app its going to be 90% or more open source because the electron base is so large already.
I would expect the Fat Head of most used open source projects to make up the vast majority of the open source code included in apps. It is not a common practice to include 1000 small projects into a code base for an app, or even 100.
Not usually 1000. But nowadays apps really do have a lot of dependencies - often more than 100.
An article about this:
https://wiki.alopex.li/LetsBeRealAboutDependencies
Rust apps also have been criticized for this. The thing is that when building a Rust app, every direct and indirect dependency is fetched and built locally. That makes them very visible. But the same happens e.g. with Debian systems with installing C or Python libraries, which bundle many pre-compilef dependencies, which is faster, and not as visible.
Yes, 100 projects from the Fat Head would make sense.
Do you have any way to establish that these 100 more often come from the Long Tail?
I think it is very likely a few very big components and many, many very small ones.
This is a very common pattern. But that is hard to verify without examining all these components.
I think the OP’s analysis might have made a bit of a jump from overall levels of hobbyist maintainers to what percentage of shipping code is maintained by people in their spare time.
While the experiences of OpenSSL and xz should certainly drive us find better ways of funding underlying infrastructure you do see a higher participation rates of paid maintainers where the returns are more obvious. The silicon vendors get involved in the kernel because it’s in their underlying interests to do so - and the kernel benefits as a result.
I maintain a couple of hobbyist packages on my spare time but it will never be a funded gig because comparatively fewer people use them compared to DAYJOB’s project which can make a difference to companies bottom lines.
Well, hobbyist projects are surely not the only pillar of the open source systems, and big projects like the Linux kernel matter immensely, too. But the article author does not deny that. He makes a point that the hobbyist projects are very important, too. Without them, there would be very little desktop software. I’d guess that much of KDE is hobbyist-powered.
And apart from that, financial support for projects important for infrastructure is a popular talking point. But I don’t see that happen much. Where are the SW engineering jobs for maintainers and contributors of real time Linux, messaging middleware, things like Ceph and file systems, FLOSS browsers, conference software, and so on? And now there are calls that the FLOSS community should care for security in infrastructure and industrial applications. If this were serious, one could simply pay the people who already do that (and massively hire more of them).
Well, hobbyist projects are surely not the only pillar of the open source systems
Your hunch is correct, they are, because the differentiator between open source and walled garden projects is freedom, and freedom will spontaneously generate projects based on an unfulfilled need. A paid market by itself will not.
In my early days of programming (late 80s), I was copying code from books and magazines. Then came windows and mac, and these were far less friendly to devs, and became more and more so.
Most of these tools were born of need and want, not because any infrastructure existed to pay them. Look at the list of apps in frdroid; most are very obviously solving a problem unique to the dev.
And there is one more thing to account for: for all the apps and scripts you see in a public code repo, there are many times more than that living on someone’s HDD that will never see the public eye.
The point you’ve ignored in your article is that this is simply the split free market creates. We’ve had this issue since the invention of transmissible ideas.
