There are cases where the model pipelines you are running are longer than what your clients would be willing to wait for or are longer than the 3-minute limit Cerebrium allows for on endpoints. Therefore you might want tasks to execute in the background and be alerted of them when they are completed.

Cerebrium automatically adds the following name parameter to every request object you send in - named webhook_endpoint. This means you can provide an endpoint for us to send with your model results. If we detect the parameter in your request, we will give you a response immediately with the run_id and status code 200. The results we send to your webhook_endpoint later will contain the same run_id so that you can make the link on your side. We will always alert your endpoint regardless of whether the function executes successfully or not. If the function fails, we will send you the error message.

Let us look at the example below which demonstrates how to use this feature:

We send the following request to our endpoint on Cerebrium that runs a LLama 2 70B:

curl --location --request POST 'https://run.cerebrium.ai/v3/xxxxxx/predict' \
--header 'Authorization: <JWT_TOKEN>' \
--header 'Content-Type: application/json' \
--data '{"prompt": "Give me a detailed plan of how I can drill to the center of the Earth.", "webhook_endpoint": "https://webhook.site/0dc4773b-5e5b-4ef1-8a72-87065852a80e"}'

Response:

{
    "run_id": "52eda406-b81b-43f5-8deb-fcf80dfsb74b",
    "webhook_endpoint": "https://webhook.site/0dc4773b-5e5b-4ef1-8a72-87065852a80e",
    "message": "We will return the output to the endpoint provided when the task is complete."
}

Later response:

{
    {
  "run_id": "52eda406-b81b-43f5-8deb-fcf80dfsb74b",
  "message": "Finished inference request with run_id: `52eda406-b81b-43f5-8deb-fcf80d78b74b`",
  "result": {
    "text": "The concept of drilling to the center of Earth can be quite exciting but it faces a number of significant scientific and logistical challenges, most of which make the project currently impossible with current technology. But for an idea of what such a venture would entail, here is an imaginary plan:\n1. Planning and Research:\nYou'd need extensive research to understand the Earth's structure and composition. You'll need data on the Earth's core, how temperature and pressure increase with depth, current drilling technology, potential hazards and contingities.\n2. Site Selection:\nChoose a drilling site. Ideally, it would be a place with a relatively thin crust making the journey shorter. The oceanic crust is much thinner compared to the continental crust.\n3. Drill Design:\nYou might have to design a new drill to withstand extreme temperatures and pressures. The further down the Earth you go, the hotter and more pressurized it becomes. You would need a drill made from extremely heat and pressure-resistant materials, perhaps with in-built cooling mechanisms.\n4. Journey Begins:\nStart drilling. However, today's technology can only drill approximately 12 kilometers into the Earth's crust. It's a bit like the thickness of an eggshell compared to the radius of the egg. "
  },
  "status_code": 200,
  "run_time_ms": 3203.2923698425293
}
}
Persistent StorageAsync Functionality