Customizing the Cortex Runtime

​

Python Version

By default, Cerebrium configs are initialized to the Python version of your interpreter. You can edit the Python version you wish to use for your app in the [cerebrium.deployment] section of your config. Note, Cerebrium only supports Python 3.10 through 3.12. Support for 3.13 will follow shortly after its release.

[cerebrium.deployment]
name = "cerebrium-app"
...
python_version = "3.12"

​

Docker Base Image

Cerebrium supports the ability to define the base docker image you would like to use. At the moment, this feature is currently in beta and so we only have three options available:

• debian:bookworm-slim (default)
• nvidia/cuda:12.1.1-runtime-ubuntu22.04
• nvidia/cuda:11.8.0-runtime-ubuntu22.04

We will add support for bringing your own in the future.

​

Dependancies

Traditionally, when working with Python, you will need access to Apt packages, Pip packages and Conda packages. For a deployment to cerebrium, you can specify all of these in your cerebrium.toml file in the following locations:

• [cerebrium.dependencies.pip] - This is where you define your Pip packages in the form module = version_constraints.

• [cerebrium.dependencies.conda] - This is where you can define Conda packages you would like to install if you prefer using it for some libraries over pip. You can use both conda and pip in conjunction.

• [cerebrium.dependencies.apt] - This is where you can define Linux apt packages you would like to install.

Here is an example of the dependencies section of a cerebrium.toml file:

[cerebrium.dependencies.pip]
transformers = ">=4.35.0"
torch = "2.1.0"
accelerate = "latest"

[cerebrium.dependencies.conda]
cuda = ">=11.7"

[cerebrium.dependencies.apt]
ffmpeg = "latest"
"libopenblas-base" = "latest"
"libomp-dev" = "latest"

All the sections above are optional, however, have to contain these file names specifically.

Typically, specifying versions for packages leads to faster builds however, if you ever find you would like to change version numbers or find your library versions aren’t updating, please add the following flag to your deploy command: cerebrium deploy --name <<app-name>> --force-rebuild

​

Shell Commands

Cerebrium gives you the ability to run shell commands - this is for more complicated use cases of cloning repositories, running install scripts etc. You can also define environment variables, which will populate every time your application starts up.

In order to run shell commands you define a list of strings in the [cerebrium.build] section of your cerebrium.toml:

[cerebrium.deployment]
name = "cerebrium-app"
...
shell_commands = [
    "echo 'Hello, World!'",
    "curl -LsSf https://astral.sh/uv/install.sh | sh",
    "uv venv",
    "uv pip install transformers"
    "export SOME_ENV_VAR=value"
]