Custom Python Web Servers
Run ASGI/WSGI Python apps on Cerebrium
While Cerebrium’s default runtime works well for most app needs, teams sometimes need more control over their web server implementation. Using ASGI or WSGI servers through Cerebrium’s custom runtime feature enables capabilities like custom authentication, dynamic batching, frontend dashboards, public endpoints, and WebSocket connections.
Setting Up Custom Servers
Here’s a simple FastAPI server implementation that shows how custom servers work in Cerebrium:
Configure this server in cerebrium.toml
by adding a custom runtime section:
The configuration requires three key parameters:
entrypoint
: The command that starts your serverport
: The port your server listens onhealthcheck_endpoint
: The endpoint used to confirm instance health. If unspecified, defaults to a TCP ping on the configured port. If the health check registers a non-200 response, it will be considered unhealthy, and be restarted should it not recover timely.readycheck_endpoint
: The endpoint used to confirm if the instance is ready to receive. If unspecified, defaults to a TCP ping on the configured port. If the ready check registers a non-200 response, it will not be a viable target for request routing.
For ASGI applications like FastAPI, include the appropriate server package
(like uvicorn
) in your dependencies. After deployment, your endpoints become
available at https://api.cortex.cerebrium.ai/v4/{project - id}/{app - name} /your/endpoint
.
Our FastAPI Server Example provides a complete implementation.
Request Headers
When using custom web servers, you can access the Cerebrium run ID through the X-Request-Id
header, which is included in all requests to your endpoints. This header is particularly useful for tracking and debugging requests in your custom runtime implementation, as it corresponds to the run_id
that Cerebrium uses internally.