The fastest way to get started developing a Cerebrium deployment is to set up a template project using the cerebrium init command below. This will create a folder with all the necessary files to get you started. You can then add your code and deploy it to Cerebrium.

cerebrium init first-project

Currently, our implementation has five components:

  • - This is where your Python code lives. This is mandatory to include.
  • requirements.txt - This is where you define your Python packages where each package should be on a new line. Deployment will be quicker if you specify specific versions. This is optional to include.
  • pkglist.txt - This is where you can define Linux packages where each package should be on a new line. We run the apt-install command for items here. This is optional to include.
  • conda_pkglist.txt - This is where you can define Conda packages where each package should be on a new line. if you prefer using it for some libraries over pip. You can use both conda and pip in conjunction. This is optional to include.
  • config.yaml - This is where you define all the configurations around your model such as the hardware you use, memory required, min replicas etc. Check here for a full list

Every you deploy needs the following mandatory layout:

from pydantic import BaseModel

class Item(BaseModel):
    parameter: value

def predict(item, run_id, logger):
    item = Item(**item)

    # Do something with parameters from item

    return {"key": "value}

The Item class is where you define the parameters your model receives as well as their type. Item needs to inherit from BaseModel which uses Pydantic to validate request schemas. You need to define a function with the name predict which receives 3 params: item, run_id and logger.

  • item: This is the expected request object containing the parameters you defined above.
  • run_id: This is a unique identifier for the user request if you want to use it to track predictions through another system
  • logger: Cerebrium supports logging via the logger (we also support “print()” statements) however, using the logger will format your logs nicer. It contains the 3 states across most loggers:
  • logger.debug
  • logger.error

As long as your contains the above you can write any other Python code. Import classes, add other functions etc.

Deploy model

Then navigate to where your model code (specifically your is located and run the following command:

cerebrium deploy my-first-model

Voila! Your app should start building and you should see logs of the deployment process. It shouldn’t take longer than a minute - easy peasy!

View model statistics and logs

Once you deploy a model, navigate back to the Cerebrium dashboard and click on the name of the model you just deployed. You will see the usual overview statistics of your model, but most importantly, you will see two tabs titled builds and runs.

  • Builds: This is where you can see the logs regarding the creation of your environment and the code specified in the Init function. You will see logs only on every deployment.
  • Runs: This is where you will see logs concerning every API call to your model endpoint. You can therefore debug every run based on input parameters and the model output.

Now that we have covered the basics of deploying a model, let’s dive into some of the more advanced functionality that Cortex provides.

Below are some links outlining some of the more advanced functionality that Cortex provides:

  • Custom Images: How to create your custom environments to run your ML Models.
  • Secrets: Use secrets to authenticate with third-party platforms.
  • Persistent Storage: Store model weights and files locally for faster access.
  • Long Running Tasks: Execute long running tasks in the background.
  • Streaming: Stream output live back to your endpoint