Managing Files

Cerebrium offers file management through a 50GB persistent volume that’s available to all apps in a project. This storage mounts at /persistent-storage and helps store model weights and files efficiently across deployments.

​

Including Files in Deployments

The cerebrium.toml configuration file controls which files become part of the app:

[cerebrium.deployment]
include = [
    "src/*.py",         # Python files in src.
    "config/*.json",    # JSON files in config.
    "requirements.txt"  # Specific files.
]

exclude = [
    "tests/*",          # Skip test files.
    "*.log"            # Skip log files.
]

Files included in deployments must be under 2GB each, with deployments working best for files under 1GB. Larger files should use persistent storage instead.

​

Managing Persistent Storage

The CLI provides three commands for working with persistent storage:

1. Upload files with cerebrium cp:

   # Upload to root directory
   cerebrium cp src_file_name.txt
   
   # Upload to specific location
   cerebrium cp src_file_name.txt dest_file_name.txt
   
   # Upload to directory
   cerebrium cp dir_name sub_folder/
   

2. List files with cerebrium ls:

   # List root contents
   cerebrium ls
   
   # List specific folder
   cerebrium ls sub_folder/
   

3. Remove files with cerebrium rm:

   # Remove a file
   cerebrium rm file_name.txt
   
   # Remove a directory
   cerebrium rm sub_folder/
   

​

Using Stored Files

Here’s how to work with files in persistent storage:

import os
import torch

# Load a model from persistent storage.
file_path = "/persistent-storage/segment-anything/sam_vit_h_4b8939.pth"
model = torch.jit.load(file_path)