Shami's Blog

DevOps because uptime is not optional

Development Environments With Podman

I switched to NixOS a while back. Then I tried to learn Rust. Turns out, Rust does not play nicely with Nix. So the journey of finding an alternative began.

Experimentation started with a few solutions: distrobox, Devbox, Dev Containers, and DevPod. They all had their pros and cons, but none of them felt right.

The solution that ended up working is a custom development container with podman and mise-en-place. The benefits of this setup are:

  • Extremely flexible, can simulate almost any environment. This example uses Debian slim, but any base image that suits your needs works.
  • Shared UID/GID between the host and container; files can be edited from either the host or container.
  • Perfect for use with Neovim, with custom configurations/LSPs per container.
  • Allows usage over SSH, even from a phone or tablet.
  • Isolates changes from the host.
  • Gives least amount of access to the tools running inside the container. Useful when using tools like pi.
  • Unlike Docker, podman doesn’t mess with iptables and doesn’t have to run as root.

To achieve this, 2 wrapper scripts and a small configuration file were used:

  1#!/bin/bash
  2
  3# devenv-buildimage
  4# Builds the development container image
  5
  6set -euo pipefail
  7
  8# Default configuration file
  9CONFIG_FILE="./.devenv"
 10
 11# Check if a custom config file is provided
 12if [ $# -ge 1 ]; then
 13    CONFIG_FILE="$1"
 14fi
 15
 16# Check if the config file exists
 17if [ ! -f "$CONFIG_FILE" ]; then
 18    echo "Error: Configuration file '$CONFIG_FILE' not found." >&2
 19    exit 1
 20fi
 21
 22# Read the image name from the config file
 23IMAGE_NAME=""
 24while IFS='=' read -r key value; do
 25    # Skip comments and empty lines
 26    [[ "$key" =~ ^[[:space:]]*#.*$ || -z "$key" ]] && continue
 27
 28    # Trim whitespace
 29    key=$(echo "$key" | xargs)
 30    value=$(echo "$value" | xargs)
 31
 32    case "$key" in
 33        Image|image)
 34            IMAGE_NAME="$value"
 35            ;;
 36    esac
 37done < "$CONFIG_FILE"
 38
 39# Validate required field
 40if [ -z "$IMAGE_NAME" ]; then
 41    echo "Error: 'image' is required in the config file." >&2
 42    exit 1
 43fi
 44
 45BASE_IMAGE="docker.io/debian:trixie-slim"
 46
 47# Use the same UID and GID as current user
 48MyUID=$(id -u)
 49MyGID=$(id -g)
 50
 51echo "==> Creating working container from $BASE_IMAGE..."
 52CONTAINER=$(buildah from "$BASE_IMAGE")
 53echo "    Container: $CONTAINER"
 54
 55cleanup() {
 56    echo "==> Removing working container..."
 57    buildah rm "$CONTAINER" 2>/dev/null || true
 58}
 59trap cleanup EXIT
 60
 61echo "==> Running setup commands..."
 62buildah run \
 63    --env MyUID="$MyUID" \
 64    --env MyGID="$MyGID" \
 65    "$CONTAINER" -- bash -c '
 66set -e
 67export DEBIAN_FRONTEND=noninteractive
 68
 69apt-get update
 70apt-get install --no-install-recommends -y curl fish build-essential ripgrep unzip fd-find luarocks locales tree-sitter-cli sudo git ca-certificates openssh-client
 71update-ca-certificates
 72
 73luarocks --lua-version 5.1 install jsregexp
 74
 75locale-gen en_US.UTF-8
 76echo "en_US.UTF-8 UTF-8" | tee /etc/locale.gen
 77locale-gen
 78dpkg-reconfigure locales
 79update-locale LANG=en_US.UTF-8
 80
 81curl https://mise.run | MISE_INSTALL_PATH=/usr/local/bin/mise sh
 82
 83rm -rf /var/lib/apt/lists/*
 84
 85useradd -m -u $MyUID -g $MyGID devuser
 86
 87# Passwordless sudo for devuser
 88echo "devuser ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/devuser
 89chmod 0440 /etc/sudoers.d/devuser
 90
 91mkdir -p /home/devuser/.config /home/devuser/.local/share/mise/installs
 92chown -R devuser:users /home/devuser
 93'
 94
 95echo "==> Configuring image defaults (user, workdir, env)..."
 96buildah config \
 97    --user devuser \
 98    --workingdir /home/devuser \
 99    --env HOME=/home/devuser \
100    --env LANG=en_US.UTF-8 \
101    "$CONTAINER"
102
103echo "==> Committing image as $IMAGE_NAME..."
104buildah commit "$CONTAINER" "$IMAGE_NAME"
105
106echo "==> Done. Image built: $IMAGE_NAME"
  1#!/bin/bash
  2
  3# devenv-createcontainer
  4# Creates a container using the image, volumes and ports defined
  5
  6# Default configuration file
  7CONFIG_FILE="./.devenv"
  8
  9# Check if a custom config file is provided
 10if [ $# -ge 1 ]; then
 11    CONFIG_FILE="$1"
 12fi
 13
 14# Check if the config file exists
 15if [ ! -f "$CONFIG_FILE" ]; then
 16    echo "Error: Configuration file '$CONFIG_FILE' not found." >&2
 17    exit 1
 18fi
 19
 20# Initialize variables
 21VOLUMES=()
 22PORTS=()
 23ENTRYPOINT=""
 24IMAGE=""
 25COMMAND=""
 26NAME=""
 27
 28# Read the config file line by line
 29while IFS='=' read -r key value; do
 30    # Skip comments and empty lines
 31    [[ "$key" =~ ^[[:space:]]*#.*$ || -z "$key" ]] && continue
 32
 33    # Trim whitespace
 34    key=$(echo "$key" | xargs)
 35    value=$(echo "$value" | xargs)
 36
 37    case "$key" in
 38        Volume|volume|volumes)
 39            VOLUMES+=("$value")
 40            ;;
 41        Port|port|ports)
 42            PORTS+=("$value")
 43            ;;
 44        Entrypoint|entrypoint)
 45            ENTRYPOINT="$value"
 46            ;;
 47        Image|image)
 48            IMAGE="$value"
 49            ;;
 50        Command|command)
 51            COMMAND="$value"
 52            ;;
 53        Name|name)
 54            NAME="$value"
 55            ;;
 56        *)
 57            echo "Warning: Unknown key '$key' in config file." >&2
 58            ;;
 59    esac
 60done < "$CONFIG_FILE"
 61
 62# Validate required fields
 63if [ -z "$IMAGE" ]; then
 64    echo "Error: 'image' is required in the config file." >&2
 65    exit 1
 66fi
 67
 68# Build the podman run command
 69PODMAN_CMD="podman run -d --stop-signal SIGKILL --userns=keep-id:uid=$(id -u),gid=$(id -g)"
 70
 71# Add container name if specified
 72if [ -n "$NAME" ]; then
 73    PODMAN_CMD+=" --name $NAME"
 74fi
 75
 76# Add volumes
 77for vol in "${VOLUMES[@]}"; do
 78    PODMAN_CMD+=" -v $vol"
 79done
 80
 81# Add ports
 82for port in "${PORTS[@]}"; do
 83    PODMAN_CMD+=" -p $port"
 84done
 85
 86# Add entrypoint
 87if [ -n "$ENTRYPOINT" ]; then
 88    PODMAN_CMD+=" --entrypoint $ENTRYPOINT"
 89fi
 90
 91# Add image
 92PODMAN_CMD+=" $IMAGE"
 93
 94# Add command
 95if [ -n "$COMMAND" ]; then
 96    PODMAN_CMD+=" $COMMAND"
 97fi
 98
 99# Print the command for debugging (optional)
100echo "Executing: $PODMAN_CMD"
101
102# Execute the command
103eval "$PODMAN_CMD"

Last but not least, the configuration file

 1# Example file, just save as .devenv in the root of your project
 2name=my-development-container
 3image=devcontainer
 4
 5# The root of the project goes to /workspace
 6volume=./:/workspace
 7
 8# Share .ssh configuration in read-only mode
 9volume=~/.ssh:/home/devuser/.ssh:ro
10
11# Neovim and fish doesn't really play nicely with read only, so create them as
12# an overlay share, container can modify but changes are not mirrored to the host
13volume=~/.config/fish:/home/devuser/.config/fish:O
14volume=~/.config/nvim:/home/devuser/.config/nvim:O
15
16# If you have multiple containers running mise, share the downloads umong them
17# be nice :)
18volume=~/.local/share/mise/installs:/home/devuser/.local/share/mise/installs
19
20# The git user information is likely shaared with the host
21volume=~/.gitconfig:/home/devuser/.gitconfig:ro
22
23# If you also want to share the SSH agent socket with the host
24volume=$SSH_AUTH_SOCK:/tmp/ssh-agent
25
26# Expose ports to the host
27port=1313:1313
28port=8080:80
29
30# This makes the container run indefinitely
31entrypoint=sleep
32command=infinity

Now all you need to do is:

  • Place .devenv in the root of your project.
  • Run devenv-buildimage to create/update the dev environment.
  • Run devenv-createcontainer to create the final development environment.
  • Run podman exec -it CONTAINER_NAME fish to enter the development shell.
  • Run mise use -g neovim lazygit to install Neovim and lazygit globally in the container. Other tools can also be used
  • Run mise trust . in /workspace to trust any mise settings inside that workspace (This is only needed once per container per path)

To get SSH working with the read-only share, you can add the following to the beginning of ~/.ssh/config

1Match exec "test \"$container\" = 'podman'"
2    UserKnownHostsFile /tmp/known_hosts_podman

For fish, the following customizations in ~/.config/fish/conf.d/mise.fish work well:

 1if test "$container" = podman; or test -f /run/.containerenv
 2    mise activate fish | source
 3
 4    alias vi nvim
 5    alias vim nvim
 6
 7    alias ll 'ls -l'
 8
 9    setenv EDITOR nvim
10    setenv SSH_AUTH_SOCK /tmp/ssh-agent
11end

About Me

Dev gone Ops gone DevOps. Any views expressed on this blog are mine alone and do not necessarily reflect the views of my employer.

Categories