Persistent Storage for Caching
Cerebrium caches your models in a specific directory on the server. This cache
persists across your project, meaning that if you deploy a new model with the same
name or use the same HuggingFace model, it will be loaded from the cache rather than downloaded from the cloud. This allows
us to scale your model deployments to handle more requests quickly, as well as reducing the
size of your deployment container images. Currently, the cache can be accessed through
/persistent-storage in your container instance,
should you wish to access it directly and store other artefacts. While you have full access to this drive, we recommend that you only
store files in directories other than
/persistent-storage/cache, as this and its subdirectories are used by Cerebrium to store your models.
In the future we will charge per GB of persistent storage used, but for now it is free while we are in active development. We will also be implementing a way to clear the cache in the future, as well as pre-uploading models to the cache before deployment.
As a simple example, suppose you have an external SAM model that you want to use in your custom deployment. You can download it to the cache as such:
import os import torch file_path = "/persistent-storage/segment-anything/model.pt" # Check if the file already exists, if not download it if not os.path.exists("/persistent-storage/segment-anything/"): response = requests.get("https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth") with open(file_path, "wb") as f: f.write(response.content) # Load the model model = torch.jit.load(file_path) ... # Continue with your initialization
Now, in subsequent deployments the model will load from the cache rather than downloading it again.