There are cases where the model pipelines you are running are longer than what your clients would be willing to wait for or are longer than the 3 minute limit Cerebrium allows for. Therefore you might want tasks to execute in the backend and be alerted of them when they are completed.

Cerebrium automatically adds the following name parameter to every request object you send in - named “webhook_endpoint”. This means you can send a endpoint for us to alert with your model results. If we detect the parameter in your request, we will give you a response immediately with the run_id. The webhook_endpoint we send you later will contain the same run_id so you can make the link on your side. Let us look at the example below:

We send the following request to our endpoint that runs a LLama 2 70B:

curl --location --request POST 'https://run.cerebrium.ai/v3/xxxxxx/predict' \
--header 'Authorization: <JWT_TOKEN>' \
--header 'Content-Type: application/json' \
--data '{"prompt": "Give me a detailed plan of how I can drill to the center of the Earth.", "webhook_endpoint": "https://webhook.site/0dc4773b-5e5b-4ef1-8a72-87065852a80e"}'

Response:

{
    "run_id": "52eda406-b81b-43f5-8deb-fcf80dfsb74b",
    "webhook_endpoint": "https://webhook.site/0dc4773b-5e5b-4ef1-8a72-87065852a80e",
    "message": "We will return the output to the endpoint provided when the task is complete."
}

Later response:

{
    {
  "run_id": "52eda406-b81b-43f5-8deb-fcf80dfsb74b",
  "message": "Finished inference request with run_id: `52eda406-b81b-43f5-8deb-fcf80d78b74b`",
  "result": {
    "text": "The concept of drilling to the center of Earth can be quite exciting but it faces a number of significant scientific and logistical challenges, most of which make the project currently impossible with current technology. But for an idea of what such a venture would entail, here is an imaginary plan:\n1. Planning and Research:\nYou'd need extensive research to understand the Earth's structure and composition. You'll need data on the Earth's core, how temperature and pressure increase with depth, current drilling technology, potential hazards and contingities.\n2. Site Selection:\nChoose a drilling site. Ideally, it would be a place with a relatively thin crust making the journey shorter. The oceanic crust is much thinner compared to the continental crust.\n3. Drill Design:\nYou might have to design a new drill to withstand extreme temperatures and pressures. The further down the Earth you go, the hotter and more pressurized it becomes. You would need a drill made from extremely heat and pressure-resistant materials, perhaps with in-built cooling mechanisms.\n4. Journey Begins:\nStart drilling. However, today's technology can only drill approximately 12 kilometers into the Earth's crust. It's a bit like the thickness of an eggshell compared to the radius of the egg. "
  },
  "status_code": 202,
  "run_time_ms": 3203.2923698425293
}
}
Cortex Config FilesScaling of Deployments