Model Monitoring Inference Aggregator¶
Class: ModelMonitoringInferenceAggregatorBlockV1
This block 📊 transforms inference data reporting to a whole new level by periodically aggregating and sending a curated sample of predictions to Roboflow Model Monitoring.
✨ Key Features¶
-
Effortless Aggregation: Collects and organizes predictions in-memory, ensuring only the most relevant and confident predictions are reported.
-
Customizable Reporting Intervals: Choose how frequently (in seconds) data should be sent—ensuring optimal balance between granularity and resource efficiency.
-
Debug-Friendly Mode: Fine-tune operations by enabling or disabling asynchronous background execution.
🔍 Why Use This Block?¶
This block is a game-changer for projects relying on video processing in Workflows. With its aggregation process, it identifies the most confident predictions across classes and sends them at regular intervals in small messages to Roboflow backend - ensuring that video processing performance is impacted to the least extent.
Perfect for:
-
Monitoring production line performance in real-time 🏭.
-
Debugging and validating your model’s performance over time ⏱️.
-
Providing actionable insights from inference workflows with minimal overhead 🔧.
🚨 Limitations¶
-
The block is should not be relied on when running Workflow in
inferenceserver or via HTTP request to Roboflow hosted platform, as the internal state is not persisted in a memory that would be accessible for all requests to the server, causing aggregation to only have a scope of single request. We will solve that problem in future releases if proven to be serious limitation for clients. -
This block do not have ability to separate aggregations for multiple videos processed by
InferencePipeline- effectively aggregating data for all video feeds connected to single process runningInferencePipeline.
Type identifier¶
Use the following identifier in step "type" field: roboflow_core/model_monitoring_inference_aggregator@v1to add the block as
as step in your workflow.
Properties¶
| Name | Type | Description | Refs |
|---|---|---|---|
name |
str |
Enter a unique identifier for this step.. | ❌ |
frequency |
int |
Frequency of reporting (in seconds). For example, if 5 is provided, the block will report an aggregated sample of predictions every 5 seconds.. | ✅ |
unique_aggregator_key |
str |
Unique key used internally to track the session of inference results reporting. Must be unique for each step in your Workflow.. | ❌ |
fire_and_forget |
bool |
Boolean flag dictating if sink is supposed to be executed in the background, not waiting on status of registration before end of workflow run. Use True if best-effort registration is needed, use False while debugging and if error handling is needed. |
✅ |
The Refs column marks possibility to parametrise the property with dynamic values available
in workflow runtime. See Bindings for more info.
Available Connections¶
Compatible Blocks
Check what blocks you can connect to Model Monitoring Inference Aggregator in version v1.
- inputs:
Path Deviation,Multi-Label Classification Model,LMM For Classification,Keypoint Detection Model,Gaze Detection,Instance Segmentation Model,Single-Label Classification Model,OCR Model,Object Detection Model,Line Counter,Detections Filter,YOLO-World Model,Model Monitoring Inference Aggregator,VLM as Detector,Google Vision OCR,Email Notification,CogVLM,Byte Tracker,Detections Classes Replacement,Template Matching,Detection Offset,Roboflow Dataset Upload,Slack Notification,Stitch OCR Detections,Identify Changes,Byte Tracker,LMM,VLM as Detector,Byte Tracker,Bounding Rectangle,Object Detection Model,Time in Zone,Detections Stitch,Florence-2 Model,SIFT Comparison,Keypoint Detection Model,Perspective Correction,Local File Sink,VLM as Classifier,Twilio SMS Notification,Detections Consensus,Webhook Sink,OpenAI,Roboflow Custom Metadata,Instance Segmentation Model,Roboflow Dataset Upload,Clip Comparison,VLM as Classifier,Anthropic Claude,SIFT Comparison,Google Gemini,Segment Anything 2 Model,JSON Parser,Single-Label Classification Model,Identify Outliers,Time in Zone,Florence-2 Model,Detections Stabilizer,Path Deviation,OpenAI,Multi-Label Classification Model,CSV Formatter,Llama 3.2 Vision,Detections Transformation - outputs:
Multi-Label Classification Model,Pixelate Visualization,Path Deviation,LMM For Classification,Keypoint Detection Model,Gaze Detection,Line Counter,Instance Segmentation Model,CLIP Embedding Model,Single-Label Classification Model,Blur Visualization,Mask Visualization,Object Detection Model,Line Counter,YOLO-World Model,Model Monitoring Inference Aggregator,Cache Get,Polygon Visualization,Halo Visualization,Google Vision OCR,Email Notification,Model Comparison Visualization,CogVLM,Image Threshold,Keypoint Visualization,Template Matching,Image Preprocessing,Slack Notification,Roboflow Dataset Upload,Background Color Visualization,Bounding Box Visualization,Label Visualization,Classification Label Visualization,Ellipse Visualization,Line Counter Visualization,LMM,Reference Path Visualization,Stability AI Inpainting,Dynamic Crop,Triangle Visualization,Object Detection Model,Distance Measurement,Time in Zone,Detections Stitch,Florence-2 Model,SIFT Comparison,Keypoint Detection Model,Corner Visualization,Perspective Correction,Local File Sink,Polygon Zone Visualization,Twilio SMS Notification,Trace Visualization,Webhook Sink,Detections Consensus,Size Measurement,Roboflow Custom Metadata,OpenAI,Cache Set,Instance Segmentation Model,Crop Visualization,Roboflow Dataset Upload,Clip Comparison,Anthropic Claude,Image Blur,Dot Visualization,Circle Visualization,Google Gemini,Segment Anything 2 Model,Single-Label Classification Model,Time in Zone,Florence-2 Model,Path Deviation,OpenAI,Color Visualization,Multi-Label Classification Model,Pixel Color Count,Llama 3.2 Vision
Input and Output Bindings¶
The available connections depend on its binding kinds. Check what binding kinds
Model Monitoring Inference Aggregator in version v1 has.
Bindings
-
input
predictions(Union[instance_segmentation_prediction,keypoint_detection_prediction,classification_prediction,object_detection_prediction]): Reference data to extract property from.model_id(roboflow_model_id): Model ID to report to Roboflow Model Monitoring.frequency(string): Frequency of reporting (in seconds). For example, if 5 is provided, the block will report an aggregated sample of predictions every 5 seconds..fire_and_forget(boolean): Boolean flag dictating if sink is supposed to be executed in the background, not waiting on status of registration before end of workflow run. UseTrueif best-effort registration is needed, useFalsewhile debugging and if error handling is needed.
-
output
Example JSON definition of step Model Monitoring Inference Aggregator in version v1
{
"name": "<your_step_name_here>",
"type": "roboflow_core/model_monitoring_inference_aggregator@v1",
"predictions": "$steps.my_step.predictions",
"model_id": "my_project/3",
"frequency": "3",
"unique_aggregator_key": "session-1v73kdhfse",
"fire_and_forget": true
}