It’s very far from a bulletproof solution and it can cause its own issues, but if used properly, there are a lot of issues that it can help prevent.
Put it this way: Production is on the Eclipse Temurin flavour of JDK, but your new developer downloads the Oracle one off java.com instead. Ideally both should function the same, they implement the same Java spec after all! But there might be a bug in one or the other that changes how something works.
Now you could just provision machines with a particular environment setup, and not let your developers install random-ass packages, but most smaller companies I’ve worked for don’t really practice that for developers (less technical employees are another story). You get full admin access to your machine and you can customize your environment.
Instead, your new developer builds the same docker container that runs in your kubernetes cluster. Same base image, same dependencies installed, etc. Day one and the project builds fine (assuming the code itself works).
However, it’s not perfect. For one, sometimes an older version of a distro won’t support a newer version of some really low-level dependency for some weird-ass reason. It’s a fixed bug in newer versions, but your new employee decided to install Debian Stable on their machine so now they have to install docker from somewhere other than the main Debian repo. Another issue is the caching I mentioned beforehand.
If you’ve never used docker before, I’ll give you a quick rundown on how building a docker image works, just in case.
Generally you first pull from a base image. Depending on your use case, it might just be ubuntu 24.04, or maybe you take an image that already includes the toolchain you need (e.g a jdk image for java stuff). Your base image itself could also be based on another image, with extra stuff added. There could be tons of layers, the ‘base’ image is just the last layer you import from elsewhere before you start doing your own stuff. Then in the dockerfile, you can use different commands to build your own custom image, such as copying files from some directory on your system to the new image (like your code), or you can literally run bash commands as well. Say you used a Debian or Ubuntu base and you need some library from their repo, you just sudo apt-get install it. Or similarly you install your npm or pip or whatever packages before building. The thing is, and you can disable this, to save time, and bandwidth, docker will cache the results of each step and use cached versions up until the first step where it detects changes (usually to the files you copy to your image). But if you want it to be performant, you do all your dependency installs before your filesystem stuff so those steps don’t get re-run each time you change your code. So the first time you build an image on a particular computer, it pulls the dependencies and then until you do a clean build it might never pull them again. That’s where you can run into issues.
And yes, you can just include the dependencies in a base image that gets built automatically in the CI and only add the code and compile steps in the image you build on the local machine. But you still have to pull the new base image!
I appreciate the info and the time and effort you took to pass it along. I certainly might get around to using Docker. If you work with node you might be interested in create-api-lite, an npm package (which I haven’t used yet) that generates a basic node project structure, including dockerfile.
Oh I keep the hell away from all things javascript as much as possible, I’ve only had to work with node in a couple of projects, mostly did bugfixes to other people’s old code and migrated some APIs away from node to Kotlin as that was the company’s main tech stack and we didn’t have a single backend engineer who wanted to do Node, those APIs were built by someone who’d long left.
These days if I create a new API for a personal project, it’s going to be in Rust and for professional projects it depends on what I’m being paid to use. And for personal projects, since Rust is so heavily focused on the one single build environment that is Cargo, I don’t really see the need for docker - usually the pain points are people not having postgres or openssl installed on their machines, but you can just stuff it in the readme that it’s necessary, and you get build time errors rather than runtime errors if they’re not present. I always prefer build time errors myself.
For work I still use docker, but my main client has a fairly robust setup, where I don’t really need to worry about much, but when there is an issue, I’m usually the person who fixes it because the CTO is busy and most internal employees’ time has been scheduled already, whereas I get the “oh we need this today, can you do it?” stuff
Tbh I’ve never used Docker, but the idea that it’s a perfect bulletproof solution to environmental issues has always seemed overly rosy to me.
It’s very far from a bulletproof solution and it can cause its own issues, but if used properly, there are a lot of issues that it can help prevent.
Put it this way: Production is on the Eclipse Temurin flavour of JDK, but your new developer downloads the Oracle one off java.com instead. Ideally both should function the same, they implement the same Java spec after all! But there might be a bug in one or the other that changes how something works.
Now you could just provision machines with a particular environment setup, and not let your developers install random-ass packages, but most smaller companies I’ve worked for don’t really practice that for developers (less technical employees are another story). You get full admin access to your machine and you can customize your environment.
Instead, your new developer builds the same docker container that runs in your kubernetes cluster. Same base image, same dependencies installed, etc. Day one and the project builds fine (assuming the code itself works).
However, it’s not perfect. For one, sometimes an older version of a distro won’t support a newer version of some really low-level dependency for some weird-ass reason. It’s a fixed bug in newer versions, but your new employee decided to install Debian Stable on their machine so now they have to install docker from somewhere other than the main Debian repo. Another issue is the caching I mentioned beforehand.
If you’ve never used docker before, I’ll give you a quick rundown on how building a docker image works, just in case.
Generally you first pull from a base image. Depending on your use case, it might just be ubuntu 24.04, or maybe you take an image that already includes the toolchain you need (e.g a jdk image for java stuff). Your base image itself could also be based on another image, with extra stuff added. There could be tons of layers, the ‘base’ image is just the last layer you import from elsewhere before you start doing your own stuff. Then in the dockerfile, you can use different commands to build your own custom image, such as copying files from some directory on your system to the new image (like your code), or you can literally run bash commands as well. Say you used a Debian or Ubuntu base and you need some library from their repo, you just sudo apt-get install it. Or similarly you install your npm or pip or whatever packages before building. The thing is, and you can disable this, to save time, and bandwidth, docker will cache the results of each step and use cached versions up until the first step where it detects changes (usually to the files you copy to your image). But if you want it to be performant, you do all your dependency installs before your filesystem stuff so those steps don’t get re-run each time you change your code. So the first time you build an image on a particular computer, it pulls the dependencies and then until you do a clean build it might never pull them again. That’s where you can run into issues.
And yes, you can just include the dependencies in a base image that gets built automatically in the CI and only add the code and compile steps in the image you build on the local machine. But you still have to pull the new base image!
I appreciate the info and the time and effort you took to pass it along. I certainly might get around to using Docker. If you work with node you might be interested in create-api-lite, an npm package (which I haven’t used yet) that generates a basic node project structure, including dockerfile.
Oh I keep the hell away from all things javascript as much as possible, I’ve only had to work with node in a couple of projects, mostly did bugfixes to other people’s old code and migrated some APIs away from node to Kotlin as that was the company’s main tech stack and we didn’t have a single backend engineer who wanted to do Node, those APIs were built by someone who’d long left.
These days if I create a new API for a personal project, it’s going to be in Rust and for professional projects it depends on what I’m being paid to use. And for personal projects, since Rust is so heavily focused on the one single build environment that is Cargo, I don’t really see the need for docker - usually the pain points are people not having postgres or openssl installed on their machines, but you can just stuff it in the readme that it’s necessary, and you get build time errors rather than runtime errors if they’re not present. I always prefer build time errors myself.
For work I still use docker, but my main client has a fairly robust setup, where I don’t really need to worry about much, but when there is an issue, I’m usually the person who fixes it because the CTO is busy and most internal employees’ time has been scheduled already, whereas I get the “oh we need this today, can you do it?” stuff