Model Monitoring Inference Aggregator¶
Class: ModelMonitoringInferenceAggregatorBlockV1
This block 📊 transforms inference data reporting to a whole new level by periodically aggregating and sending a curated sample of predictions to Roboflow Model Monitoring.
✨ Key Features¶
-
Effortless Aggregation: Collects and organizes predictions in-memory, ensuring only the most relevant and confident predictions are reported.
-
Customizable Reporting Intervals: Choose how frequently (in seconds) data should be sent—ensuring optimal balance between granularity and resource efficiency.
-
Debug-Friendly Mode: Fine-tune operations by enabling or disabling asynchronous background execution.
🔍 Why Use This Block?¶
This block is a game-changer for projects relying on video processing in Workflows. With its aggregation process, it identifies the most confident predictions across classes and sends them at regular intervals in small messages to Roboflow backend - ensuring that video processing performance is impacted to the least extent.
Perfect for:
-
Monitoring production line performance in real-time 🏭.
-
Debugging and validating your model’s performance over time ⏱️.
-
Providing actionable insights from inference workflows with minimal overhead 🔧.
🚨 Limitations¶
-
The block is should not be relied on when running Workflow in
inferenceserver or via HTTP request to Roboflow hosted platform, as the internal state is not persisted in a memory that would be accessible for all requests to the server, causing aggregation to only have a scope of single request. We will solve that problem in future releases if proven to be serious limitation for clients. -
This block do not have ability to separate aggregations for multiple videos processed by
InferencePipeline- effectively aggregating data for all video feeds connected to single process runningInferencePipeline.
Type identifier¶
Use the following identifier in step "type" field: roboflow_core/model_monitoring_inference_aggregator@v1to add the block as
as step in your workflow.
Properties¶
| Name | Type | Description | Refs |
|---|---|---|---|
name |
str |
Enter a unique identifier for this step.. | ❌ |
frequency |
int |
Frequency of reporting (in seconds). For example, if 5 is provided, the block will report an aggregated sample of predictions every 5 seconds.. | ✅ |
unique_aggregator_key |
str |
Unique key used internally to track the session of inference results reporting. Must be unique for each step in your Workflow.. | ❌ |
fire_and_forget |
bool |
Boolean flag to run the block asynchronously (True) for faster workflows or synchronously (False) for debugging and error handling.. | ✅ |
The Refs column marks possibility to parametrise the property with dynamic values available
in workflow runtime. See Bindings for more info.
Available Connections¶
Compatible Blocks
Check what blocks you can connect to Model Monitoring Inference Aggregator in version v1.
- inputs:
Segment Anything 2 Model,Clip Comparison,Perspective Correction,Object Detection Model,Roboflow Custom Metadata,Object Detection Model,SIFT Comparison,Detection Offset,CogVLM,VLM as Detector,Multi-Label Classification Model,OpenAI,Multi-Label Classification Model,VLM as Detector,Google Vision OCR,Identify Outliers,Roboflow Dataset Upload,Gaze Detection,Identify Changes,VLM as Classifier,Byte Tracker,Llama 3.2 Vision,Detections Stabilizer,Template Matching,Dynamic Zone,Detections Transformation,Time in Zone,Model Monitoring Inference Aggregator,Line Counter,Anthropic Claude,Webhook Sink,Instance Segmentation Model,SIFT Comparison,Time in Zone,Instance Segmentation Model,Slack Notification,Detections Filter,Stitch OCR Detections,Path Deviation,Detections Consensus,Twilio SMS Notification,VLM as Classifier,Keypoint Detection Model,Roboflow Dataset Upload,Google Gemini,JSON Parser,Byte Tracker,Velocity,Keypoint Detection Model,Bounding Rectangle,Florence-2 Model,Single-Label Classification Model,Byte Tracker,Florence-2 Model,Local File Sink,LMM For Classification,Detections Stitch,OCR Model,LMM,Email Notification,Path Deviation,Single-Label Classification Model,YOLO-World Model,CSV Formatter,Detections Classes Replacement,OpenAI - outputs:
Segment Anything 2 Model,Cache Get,Stability AI Inpainting,Clip Comparison,Perspective Correction,Cache Set,Object Detection Model,Roboflow Custom Metadata,Object Detection Model,SIFT Comparison,CogVLM,Ellipse Visualization,Multi-Label Classification Model,OpenAI,Trace Visualization,CLIP Embedding Model,Multi-Label Classification Model,Dot Visualization,Google Vision OCR,Roboflow Dataset Upload,Polygon Zone Visualization,Gaze Detection,Classification Label Visualization,Corner Visualization,Llama 3.2 Vision,Line Counter,Reference Path Visualization,Dynamic Crop,Label Visualization,Mask Visualization,Triangle Visualization,Template Matching,Line Counter Visualization,Model Monitoring Inference Aggregator,Time in Zone,Blur Visualization,Line Counter,Anthropic Claude,Instance Segmentation Model,Webhook Sink,Time in Zone,Instance Segmentation Model,Slack Notification,Pixelate Visualization,Path Deviation,OpenAI,Detections Consensus,Twilio SMS Notification,Keypoint Detection Model,Roboflow Dataset Upload,Google Gemini,Model Comparison Visualization,Halo Visualization,Crop Visualization,Image Blur,Distance Measurement,Circle Visualization,Keypoint Detection Model,Image Preprocessing,Background Color Visualization,Pixel Color Count,Size Measurement,Florence-2 Model,Single-Label Classification Model,Bounding Box Visualization,Florence-2 Model,Local File Sink,LMM For Classification,Stability AI Image Generation,Image Threshold,Detections Stitch,LMM,Keypoint Visualization,Email Notification,Color Visualization,Path Deviation,Single-Label Classification Model,YOLO-World Model,Polygon Visualization
Input and Output Bindings¶
The available connections depend on its binding kinds. Check what binding kinds
Model Monitoring Inference Aggregator in version v1 has.
Bindings
-
input
predictions(Union[instance_segmentation_prediction,keypoint_detection_prediction,object_detection_prediction,classification_prediction]): Model predictions to report to Roboflow Model Monitoring..model_id(roboflow_model_id): Model ID to report to Roboflow Model Monitoring..frequency(string): Frequency of reporting (in seconds). For example, if 5 is provided, the block will report an aggregated sample of predictions every 5 seconds..fire_and_forget(boolean): Boolean flag to run the block asynchronously (True) for faster workflows or synchronously (False) for debugging and error handling..
-
output
Example JSON definition of step Model Monitoring Inference Aggregator in version v1
{
"name": "<your_step_name_here>",
"type": "roboflow_core/model_monitoring_inference_aggregator@v1",
"predictions": "$steps.my_step.predictions",
"model_id": "my_project/3",
"frequency": "3",
"unique_aggregator_key": "session-1v73kdhfse",
"fire_and_forget": true
}