Dockerfile with multi-layer dependencies using uv and pyproject.toml

Introduction

This post has been updated on 2025-10-23 with much more optimized solution for Dockerfile-uv-multi build. It also links to the existing Dockerfile examples instead of duplicating the content. The complete changelog can be found in this PR.

uv is a modern, fast Python package and project manager. It is arguably the best thing that has happened to the Python ecosystem in recent years.

Before its arrival, I had to use a combination of multiple tools to manage a project’s dependencies and environments, such as pip, venv, pyenv, poetry, etc. Setting up a development environment for a new project required multiple steps, which was quite error-prone. uv is a single tool that replaces all of them, and I don’t even need Python installed on my machine to use it.

There has been a lot of hype around uv since its release, and plenty of resources explaining its benefits and how to use it. So I won’t delve into the details here, since it is not the focus of this post.

Switching to uv is quite straightforward for most projects. However, there exist some edge cases that prevent teams from adopting it, such as adapting an existing Dockerfile.

For that purpose, the uv documentation already provides a very comprehensive guide to use uv in Docker, and a repository with multiple examples that covers most use cases, including multi-stage builds.

This post is dedicated to an edge case that is not detailed in the documentation: generating multi-layer dependencies for a Docker image to optimize the Docker workflow from development to production.

Docker lifecycle and Dockerfile

Before diving into the main content, I’ll first explain the importance of writing a good Dockerfile and how it affects each stage in the Docker lifecycle. If you are already familiar with Docker and its core concepts, you can skip straight to the main course.

Docker lifecycle

There are 3 primary stages in the Docker workflow:

1. Build: build the image from a Dockerfile. This is mostly used during development to test the application, where the image is built and run on the dev machine. If you develop using a Docker container, the build process is executed very frequently and should be as fast as possible.
2. Push: publish the image to a Docker registry (public or private), typically done in a CI/CD pipeline together with build. The image is then stored in the registry and can be pulled by remote machines (production environment, on-premise machines, etc.).
3. Pull: pull the image from the registry to run it. Mostly done in production environment.

flowchart LR
    A((Dev)) --> B[Build]
    B --> |Layer 1| C
    B --> |Layer 2| C
    B --> |Layer 3| C
    C((Dev)) --> D[Push]
    D --> |Layer 1| E
    D --> |Layer 2| E
    D --> |Layer 3| E
    E((Docker registry)) --> F[Pull]
    F --> |Layer 1| G
    F --> |Layer 2| G
    F --> |Layer 3| G
    G((Production))
    style A fill:#0f0,color:#333
    style C fill:#0f0,color:#333
    style E fill:#fff,color:#333
    style G fill:#ff0,color:#333

Simplified Docker lifecycle

What is a Dockerfile?

A Dockerfile is a text file that contains a series of instructions and commands to define the environment and dependencies of your application, which Docker follows to assemble the image layer by layer.

One of the most important concepts in Docker is the cache layers. It is a mechanism that allows Docker to reuse the same layer for subsequent builds, pushes, and pulls processes, which is crucial to speed up the whole workflow.

As we can see in the diagram above, the cache layers are propagated all the way from the build stage through the push and pull stages. If layer 2 is invalidated, for example, the subsequent layer 3 will be invalidated as well, and both layers will have to be rebuilt and propagated again through the network. Hence, the general rule of thumb is to order the layers from the least likely to change to the most likely to change.

Another important factor to consider is the size of the image. The smaller the image, the faster the push and pull processes, and the lower the storage cost.

This is particularly useful for remote machines with limited bandwidth or unstable Internet connection. It is a specific use case that I encountered at work, which inspired this post.

Now that the basics are covered, let’s move on to the main content.

Dockerfile with uv

In this section, we will cover 2 scenarios and how to adapt the Dockerfile with uv in each case.

1. Multi-layer dependencies within a single-stage build
2. Multi-layer dependencies with multi-stage builds

But first, let’s define an example project and go through some minimal steps to migrate it to uv.

The example project and all Dockerfiles can be found on my repository.

Migrate existing project to uv via pyproject.toml

Consider a simple FastAPI project with the following requirements files:

• requirements-heavy.txt with heavy packages that take a long time to install but rarely need to be updated:

torch==2.9.0

• requirements.txt with light packages that are frequently updated:

fastapi==0.119.1
uvicorn==0.38.0

To migrate the project to uv, we can simply create a pyproject.toml as below.

[project]
name = "test-docker"
version = "0.0.0"
requires-python = "==3.12.*"
dependencies = []

[dependency-groups]
heavy-rarely-updated = ["torch==2.9.0"]
light-frequently-updated = ["fastapi==0.119.1", "uvicorn==0.38.0"]

[tool.uv]
default-groups = ["heavy-rarely-updated", "light-frequently-updated"]

The pyproject.toml replaces all requirements files by defining all the dependencies in a single file, split into named groups. It is the new standard for Python projects, and is also adopted by uv as a centralized configuration file for project management.

To generate the virtual environment with all dependencies installed, simply run:

uv sync

This command will also create an auto-generated uv.lock file that locks the dependencies of the project. This is super useful to ensure that the dependencies and sub-dependencies are the same across different machines.

The requires-python field specifies the Python version that the project should run on, which will be picked up automatically by uv to generate the virtual environment. If the specific version does not exist on the machine, uv will automatically download it for immediate and future use without any input from the user. Very handy!

However, this structure raises 2 obvious questions that affect the project’s Dockerfile:

1. If we make unrelated changes to pyproject.toml, such as updating the project name or version, how can we avoid invalidating the whole layer?
2. How can we adapt a multi-layer structure with one single pyproject.toml file instead of multiple requirements files?

We will try to answer these questions in the following sections.

Multi-layer dependencies within a single-stage build

Let’s consider this legacy Dockerfile that uses pip to install the Python dependencies.

This is a very common structure where the dependencies are installed in 3 separate cache layers. It allows us to modify the light dependencies and the source code without invalidating the heavy dependencies layer, which takes a long time to build.

The adaptation for this case is covered in the uv documentation and example. Most of the comments are self-explanatory.

Even though quite efficient, this approach is NOT an equivalent adaptation of the legacy Dockerfile, since the layers will be invalidated by dependency changes no matter what.

As far as I know, it is impossible to achieve multi-layer dependencies in a single-stage build using a single pyproject.toml as with multiple requirements files.

Fortunately, we can use multi-stage builds technique to achieve this.

Multi-layer dependencies with multi-stage build

As you must have noticed, both approaches above are rather wasteful in terms of image size, as they must install gcc in system packages, and uv itself is not needed either during runtime.

To address this issue, we can use multi-stage build to install the dependencies in a separate stage, then copy the built packages to the final image.

The legacy Dockerfile can be found here.

And the uv-specific Dockerfile here.

The biggest difference is the additional generator stage in the uv approach. To achieve multi-layer dependencies with a single pyproject.toml file, we will have to fall back to regenerate the requirements. Fortunately, uv provides a convenient command to do so: uv pip compile. The separate stage is required to guard the builder stage by any unrelated changes to the pyproject.toml file.

The builder and final stages are quite similar. Both approaches make use of the pip –prefix option to install each requirements group into a separate directory, which is then copied to the final image, each copy being a separate layer. If the contents of the requirements files are not changed, the layers will remain intact.

This approach adds a new layer of complexity to the Dockerfile, but is it worth it? Let’s find out with some actual data.

Speed and image size comparison

Now it’s time to test the 4 approaches with actual data. Let’s name them: pip-single, uv-single, pip-multi, and uv-multi.

For each approach, I will execute the following steps and measure the execution time, as well as the image size.

1. Cold build: build the image from scratch

time docker build -f Dockerfile-uv-multi -t localhost:5000/test-repo:uv-multi-cold .

1. Hot build: build the image again with a small dependency update to the light-frequently-updated group. This will allow us to verify the heavy-rarely-updated cache layer invalidation mechanism.

time docker build -f Dockerfile-uv-multi -t localhost:5000/test-repo:uv-multi-hot .

1. Cold push: push the cold built image to the local registry

2. Hot push: push the hot built image to the local registry

docker run -d -p 5000:5000 --name registry registry:latest
time docker push localhost:5000/test-repo:uv-multi-cold
time docker push localhost:5000/test-repo:uv-multi-hot

It is not necessary to include the pull process, since the layers are identical to the push step and the results should be roughly the same.

In between each approach, I will reset the local registry and the docker caches to simulate a cold start.

docker stop registry
docker system prune -a --volumes

Here are the final results (lower is better).

Approach	pip-single	pip-multi	uv-single	uv-multi
Cold build	259.59s	278.44s	242.21s	257.29s
Hot build	4.86s	16.07s	156.04s	4.48s
Cold push	10.69s	10.40s	11.42s	10.28s
Hot push	0.94s	0.89s	9.84s	1.11s
Image size	11.71 GB	11.55 GB	11.53 GB	11.53 GB

As expected, the uv-multi approach is the fastest and the smallest in terms of image size, thanks to uv speed and optimized lock file.

The uv-single method is also fast during a cold build, but struggles with the hot build and hot push steps, which is also expected since the whole dependencies layer is invalidated.

Conclusion

The uv-multi approach is the clear winner in terms of speed and image size, but it requires a bit more effort to set up, which results in a more complex Dockerfile. I would recommend this approach for large projects with many heavy dependencies such as torch or tensorrt.

For projects with only light dependencies and where image size is not a big concern, the uv-single approach is the simplest to set up and maintain. Most of the time you won’t notice any difference due to the speed of uv.

I would not recommend using the legacy pip methods, since it would defeat the whole purpose of this post 😛. They are relics of the past and should stay that way.