External Monitoring Tools

Cerebrium allows you to log your custom model predictions to external ML monitoring tools. This ability allows you to monitor your model’s performance in real-time, and compare it to your actual results and baselines. Monitoring model performance is a crucial part of the ML lifecycle to ensure that your model is performing as expected. Currently, we support Censius, but we are planning to add support for Arize in Q1 2023. If there is a tool you would like integrated into the platform, please contact us!

Adding a monitoring logger to a Conduit object is as simple as calling the add_logger method on the object. The method has the following parameters:

• platform: The platform you would like to log to, specified using cerebrium.logging_platform.
• platform_authentication: A dictionary of authentication parameters for the platform.
• features: A list of strings corresponding to the names of each feature.
• targets: A list of targets corresponding to the names of each target.
• platform_args: A dictionary of parameters that the specific platform requires. You can find these in the documentation for each specific platform.
• log_ms: A boolean indicating whether to log the timestamp in seconds or milliseconds. If True, the timestamp will be logged in milliseconds.

Below is an example of how to add a Censius logger to a Conduit object.

from cerebrium import Conduit, model_type, logging_platform
conduit = Conduit('my-flow', "<API_KEY>", [(model_type.SKLEARN, "my-rf")])
conduit.add_logger(
    platform=logging_platform.CENSIUS,
    platform_authentication={"api_key": "<CENSIUS_API_KEY>", "tenant_id": "<CENSIUS_TENANT_ID>"},
    features=["feature_1", "feature_2", "feature_3"],
    targets=["target"],
    platform_args,
    log_ms=True
)

You can link back to individual predictions made by your deployed conduit using the prediction_ids field returned by your deployed endpoint after a request. This will allow you to see the predictions made by your model in the monitoring tool, or log actuals with the given prediction_id.

You can check out our available integrations below!

• Censius