Model Monitoring Inference Aggregator¶
Class: ModelMonitoringInferenceAggregatorBlockV1
This block 📊 transforms inference data reporting to a whole new level by periodically aggregating and sending a curated sample of predictions to Roboflow Model Monitoring.
✨ Key Features¶
-
Effortless Aggregation: Collects and organizes predictions in-memory, ensuring only the most relevant and confident predictions are reported.
-
Customizable Reporting Intervals: Choose how frequently (in seconds) data should be sent—ensuring optimal balance between granularity and resource efficiency.
-
Debug-Friendly Mode: Fine-tune operations by enabling or disabling asynchronous background execution.
🔍 Why Use This Block?¶
This block is a game-changer for projects relying on video processing in Workflows. With its aggregation process, it identifies the most confident predictions across classes and sends them at regular intervals in small messages to Roboflow backend - ensuring that video processing performance is impacted to the least extent.
Perfect for:
-
Monitoring production line performance in real-time 🏭.
-
Debugging and validating your model’s performance over time ⏱️.
-
Providing actionable insights from inference workflows with minimal overhead 🔧.
🚨 Limitations¶
-
The block is should not be relied on when running Workflow in
inferenceserver or via HTTP request to Roboflow hosted platform, as the internal state is not persisted in a memory that would be accessible for all requests to the server, causing aggregation to only have a scope of single request. We will solve that problem in future releases if proven to be serious limitation for clients. -
This block do not have ability to separate aggregations for multiple videos processed by
InferencePipeline- effectively aggregating data for all video feeds connected to single process runningInferencePipeline.
Type identifier¶
Use the following identifier in step "type" field: roboflow_core/model_monitoring_inference_aggregator@v1to add the block as
as step in your workflow.
Properties¶
| Name | Type | Description | Refs |
|---|---|---|---|
name |
str |
Enter a unique identifier for this step.. | ❌ |
frequency |
int |
Frequency of reporting (in seconds). For example, if 5 is provided, the block will report an aggregated sample of predictions every 5 seconds.. | ✅ |
unique_aggregator_key |
str |
Unique key used internally to track the session of inference results reporting. Must be unique for each step in your Workflow.. | ❌ |
fire_and_forget |
bool |
Boolean flag to run the block asynchronously (True) for faster workflows or synchronously (False) for debugging and error handling.. | ✅ |
The Refs column marks possibility to parametrise the property with dynamic values available
in workflow runtime. See Bindings for more info.
Available Connections¶
Compatible Blocks
Check what blocks you can connect to Model Monitoring Inference Aggregator in version v1.
- inputs:
Twilio SMS Notification,Slack Notification,VLM as Detector,VLM as Classifier,LMM,Path Deviation,Detections Merge,Google Gemini,Detection Offset,Roboflow Dataset Upload,Single-Label Classification Model,Line Counter,OpenAI,Detections Consensus,Gaze Detection,Webhook Sink,Detections Classes Replacement,VLM as Classifier,Dynamic Zone,Instance Segmentation Model,CogVLM,Email Notification,Object Detection Model,Single-Label Classification Model,Llama 3.2 Vision,Google Vision OCR,Roboflow Dataset Upload,Byte Tracker,JSON Parser,Object Detection Model,Keypoint Detection Model,Local File Sink,Detections Transformation,Anthropic Claude,Identify Outliers,Detections Stitch,OCR Model,YOLO-World Model,Stitch OCR Detections,Byte Tracker,Perspective Correction,OpenAI,Path Deviation,Time in Zone,Detections Filter,Clip Comparison,Time in Zone,Dynamic Crop,Template Matching,CSV Formatter,Byte Tracker,Florence-2 Model,Instance Segmentation Model,Keypoint Detection Model,SIFT Comparison,Multi-Label Classification Model,Florence-2 Model,Model Monitoring Inference Aggregator,Bounding Rectangle,Velocity,SIFT Comparison,VLM as Detector,Identify Changes,Multi-Label Classification Model,LMM For Classification,Roboflow Custom Metadata,Segment Anything 2 Model,Detections Stabilizer - outputs:
Circle Visualization,Background Color Visualization,Corner Visualization,Twilio SMS Notification,Slack Notification,LMM,Polygon Zone Visualization,Image Blur,Cache Set,Dot Visualization,Path Deviation,Google Gemini,Roboflow Dataset Upload,Single-Label Classification Model,Stability AI Inpainting,Pixelate Visualization,Line Counter,OpenAI,Detections Consensus,Gaze Detection,Distance Measurement,Stability AI Image Generation,Webhook Sink,Color Visualization,Image Threshold,Halo Visualization,Polygon Visualization,Instance Segmentation Model,CogVLM,Email Notification,Object Detection Model,Classification Label Visualization,Single-Label Classification Model,Llama 3.2 Vision,Google Vision OCR,Roboflow Dataset Upload,Ellipse Visualization,Size Measurement,Pixel Color Count,Cache Get,Bounding Box Visualization,Object Detection Model,Line Counter Visualization,Image Preprocessing,Keypoint Detection Model,Trace Visualization,Label Visualization,Local File Sink,Anthropic Claude,Crop Visualization,Detections Stitch,YOLO-World Model,Model Comparison Visualization,Perspective Correction,OpenAI,Path Deviation,Mask Visualization,Time in Zone,Clip Comparison,Time in Zone,Dynamic Crop,Template Matching,Florence-2 Model,Instance Segmentation Model,Keypoint Detection Model,Reference Path Visualization,Multi-Label Classification Model,Florence-2 Model,Triangle Visualization,Model Monitoring Inference Aggregator,CLIP Embedding Model,SIFT Comparison,Keypoint Visualization,Multi-Label Classification Model,LMM For Classification,Roboflow Custom Metadata,Line Counter,Segment Anything 2 Model,Blur Visualization
Input and Output Bindings¶
The available connections depend on its binding kinds. Check what binding kinds
Model Monitoring Inference Aggregator in version v1 has.
Bindings
-
input
predictions(Union[keypoint_detection_prediction,classification_prediction,instance_segmentation_prediction,object_detection_prediction]): Model predictions to report to Roboflow Model Monitoring..model_id(roboflow_model_id): Model ID to report to Roboflow Model Monitoring..frequency(string): Frequency of reporting (in seconds). For example, if 5 is provided, the block will report an aggregated sample of predictions every 5 seconds..fire_and_forget(boolean): Boolean flag to run the block asynchronously (True) for faster workflows or synchronously (False) for debugging and error handling..
-
output
Example JSON definition of step Model Monitoring Inference Aggregator in version v1
{
"name": "<your_step_name_here>",
"type": "roboflow_core/model_monitoring_inference_aggregator@v1",
"predictions": "$steps.my_step.predictions",
"model_id": "my_project/3",
"frequency": "3",
"unique_aggregator_key": "session-1v73kdhfse",
"fire_and_forget": true
}