Initial Setup
You can quickly set up a Cortex project by running the following command:
cerebrium init <<project_name>>
This will create a Cortex project in the specified directory with the following structure:
project_name/
├── main.py
├── requirements.txt
├── pkglist.txt
├── conda_pkglist.txt
└── config.yaml
Cortex supports the use of config YAML files to configure various aspects of your project such as hardware requirements, memory and much more. Using config files makes it easier to keep track of your Cerebrium deployments, share them and use git versioning to show changes over time.
To deploy your model with a specific config file, you can use the cerebrium deploy command with the --config-file flag to specify the path to your config file. Otherwise cerebrium deploy will use the only yaml in the file directory.
cerebrium deploy
Your config file can be named anything you want and can be placed anywhere on your local machine. However, remember to use the cerebrium deploy command in the same directory as your main.py as you would normally.
Config File Format and Parameters
The parameters for your config file are the same as those which you would use as flags for a normal cerebrium deploy command. They’re tabulated below for your convenience:
| Parameter | Description | Type | Default |
|---|---|---|---|
name | Name of the deployment | string | |
api_key | API key for the deployment | string | not included for safety |
hardware | Hardware to use for the deployment | string | GPU |
gpu_count | The number of GPUs to specify | int | 2 |
cpu | The number of CPU cores to use | int | 2 |
memory | The amount of Memory to use in GB | int | 14.5 |
log_level | Log level for the deployment | string | INFO |
include | Local files to include in the deployment | string | ’[./*, main.py, requirements.txt, pkglist.txt, conda_pkglist.txt]’ |
exclude | Local Files to exclude from the deployment | string | ’[./., ./__]’ |
disable_animation | Whether to disable the animation in the logs. | boolean | false |
python_version | The Python version you would like to run | float | 3.9 |
min_replicas | The minimum number of replicas to run. | int | 0 |
max_replicas | The maximum number of replicas to scale to. | int | *plan limit |
cooldown | The number of seconds to keep your model warm after each request. It resets after every request | int | 60 |
Config File Example
%YAML 1.2
---
name: an-optional-name
api_key: an-optional-api-key
hardware: GPU
exclude: "[./.*, ./__*]"
include: "[./*, main.py, requirements.txt, pkglist.txt, conda_pkglist.txt]"
log_level: INFO
disable_animation: false
python_version: 3.9
min_replicas: 0
max_replicas: 30