In a lot of projects, this is usually done via README. It tells you what running dependencies to install and how to run certain commands.
This can get harder to maintain as things change, the project grows, and complexity increases.
I see two parts to automate here: actually setting up the environment, and running the application or orchestrating multiple apps.
How do you address this?
- direnv
- nixpkgs (with flakes enabled)
- committed flake.nix and flake.lock
- Makefile or several shell scripts in
bin/for common tasks
My life changed once I discovered dev containers in VS Code. Basically developing within a bootstrapped Docker container.
We’re following the same. It’s been a life changer for us.
Dev containers are the shit. We did the readme instructions style at my last job and it took new hires like a full day to set up, propagating changes was a nightmare and shit was always going wrong. We use dev containers now. Everyone gets the exact same version of everything with almost zero opportunity to screw it up. If anything gets messed up, it’s fixed by a rebuild almost every time.
How’s the filesystem performance? Whenever I’ve mounted something into a Docker Container, the performance has suffered. For example, things like NPM/MVN suddenly take way longer.
It depends on how you set it up. You’re going to take a performance hit using a bind mount. The docs recommend putting your workspace into an actual docker volume for better performance, but I haven’t tried that myself cause so far the bind mount has performed “good enough” for me.
In my experience it’s greatly dependent on how much memory you dedicate to Docker. I’ve never ran any performance benchmarks, this is just anecdotal.
I use “Dev Containers” in Visual Studio Code.
- Step 1 - open the root directory of your project in Visual Studio Code
- Step 2 - say yes when prompted to launch the dev container
- Step 3 - there is no Step 3.
You do need to create various config files/etc, but it’s pretty straight forward and only needs to be done once for each project. Those config files are all kept in version control.
The dev container setup generally matches the production system pretty closely. Depending on the project that ranges from dead simple to exceedingly complex with several servers all of which are built, configured and launched automatically when you open the project. If you need to change a container you just edit the config file and run a command to rebuild all of them. It’s surprisingly fast - thanks to caching at every step to avoid rebuilding the entire environment.
I work primarily on the JVM & the projects (personal/corporate) I work w/ can be summarised as below:
- Building & running the repo is done on the host using an SCM (software configuration management tool) such as Gradle or SBT.
- The external dependencies of the repo, such as Redis, are managed via a
docker-compose.yml. - The README contains a short series of commands to do different tasks RE (1)
However one approach that I’ve always been fond of (& apply/advocate wherever I can) is to replace (3) w/ a
Makefilecontaining a bunch of standard targets shared across all repos, egtest,integration-test. Then Makefiles are thinly customised to fit the repo’s particular repo.This has proven to be very helpful wrt congnitive load (and also CI/CD pipelines): ALL projects, regardless of the toolchain, use the same set of commands, namely
make testmake integration-testmake compose-upmake run
In short (quoting myself here):
Don’t repeat yourself. Make Make make things happen for you!
How do you manage JVM versions? We have many older projects that use 8, and some newer ones using 17, for example.
I’ve been using sdkman for about a decade now and am totally pleased w/ it. It does a very good job of managing JDK versions for you and much more, eg SBT, Gradle, Scala, Groovy, Leiningen, SpringBoot, …
Now, technically you could use sdkman in your CI/CD pipeline too but I’d find it a strong smell. I’ve always used dedicated images pre-configured for a particular JDK version in the pipeline.
Nix/NixOS with flakes, direnv, and lorri.
There is also https://devenv.sh/ which is based on nix.
Nix! And then you can also auto-create your CI/CD environment from Nix. They share a common source, which eliminates whole classes of problems. It’s supsr fast and very awesome when it works, but it has a learning curve. Worth the effort though.
Which programming language(s) do you have in mind? Many already have built-in support for this (Go, Rust, Nim), while others have external tools you could use (Python->Poetry). Otherwise, if you want a “fast” (easy to understand) solution, a shell-script might be a fine solution.
If you want some real power, you could use the nix package manager (as already stated by other comments). It’s easy to install, but you need to learn how to use it, and with that you can easily share dev environments.
I like using vagrant and/or docker to provision the environments, when possible.
Docker compose to get started quickly. Although docker can be slow on windows so other methods are also documented.
For my local team: Generally a container (docker) for local dev. My team uses go so sometimes a Makefile without docker is enough. For other teams i’ve mostly i see docker.
for multiple apps this can get more complicated, docker compose, or skaffold is what i generally reach for (my team is responsible for k8s clusters so skaffold is pretty natural). I’ve seen other teams use garden.
hashicorp makes something called waypoint which i’ve never used. Nix people seem to be well liked as well.
Typically using Lando which is a frontend for docker-compose which makes it easier for the users unfamiliar with docker to use it to spin up their environments.
Well-documented bash scripts.
Orchestration and containerization are heavy dependencies. I prefer few and simple requirements, especially on the environment.
That only works well with tech with a defined or prevalent environment. Then it’s a matter of keeping docs up to date - like any doc.
Using small scripts if necessary, and splitting off non central dev workflows helps keeping it simple and focused.
Containerization is only heavy outside of Linux, and orchestration only makes sense when manual orchestration becomes too tedious (it’s easy to orchestrate a single app).
Keeping docs for those things is very troublesome imo. You can’t feasibly consider everyone’s different environment, C library used, their system’s package manager and how it may package software differently than yours, and the endless array of things they may have already installed that may effect your app in some way. Sure, it’s not super common, but it’s hell when it does.
But I suppose if your use case is very simple, like “just have nodejs installed and run npm start” then sure. But things can get ugly very easily.
For toolchains like rust, go, c#, typescript/nodejs how would “things get ugly very fast” when making the toolchain an env dependency?
