Model Monitoring Inference Aggregator¶
Class: ModelMonitoringInferenceAggregatorBlockV1
This block 📊 transforms inference data reporting to a whole new level by periodically aggregating and sending a curated sample of predictions to Roboflow Model Monitoring.
✨ Key Features¶
-
Effortless Aggregation: Collects and organizes predictions in-memory, ensuring only the most relevant and confident predictions are reported.
-
Customizable Reporting Intervals: Choose how frequently (in seconds) data should be sent—ensuring optimal balance between granularity and resource efficiency.
-
Debug-Friendly Mode: Fine-tune operations by enabling or disabling asynchronous background execution.
🔍 Why Use This Block?¶
This block is a game-changer for projects relying on video processing in Workflows. With its aggregation process, it identifies the most confident predictions across classes and sends them at regular intervals in small messages to Roboflow backend - ensuring that video processing performance is impacted to the least extent.
Perfect for:
-
Monitoring production line performance in real-time 🏭.
-
Debugging and validating your model’s performance over time ⏱️.
-
Providing actionable insights from inference workflows with minimal overhead 🔧.
🚨 Limitations¶
-
The block is should not be relied on when running Workflow in
inference
server or via HTTP request to Roboflow hosted platform, as the internal state is not persisted in a memory that would be accessible for all requests to the server, causing aggregation to only have a scope of single request. We will solve that problem in future releases if proven to be serious limitation for clients. -
This block do not have ability to separate aggregations for multiple videos processed by
InferencePipeline
- effectively aggregating data for all video feeds connected to single process runningInferencePipeline
.
Type identifier¶
Use the following identifier in step "type"
field: roboflow_core/model_monitoring_inference_aggregator@v1
to add the block as
as step in your workflow.
Properties¶
Name | Type | Description | Refs |
---|---|---|---|
name |
str |
Enter a unique identifier for this step.. | ❌ |
frequency |
int |
Frequency of reporting (in seconds). For example, if 5 is provided, the block will report an aggregated sample of predictions every 5 seconds.. | ✅ |
unique_aggregator_key |
str |
Unique key used internally to track the session of inference results reporting. Must be unique for each step in your Workflow.. | ❌ |
fire_and_forget |
bool |
Boolean flag to run the block asynchronously (True) for faster workflows or synchronously (False) for debugging and error handling.. | ✅ |
The Refs column marks possibility to parametrise the property with dynamic values available
in workflow
runtime. See Bindings for more info.
Available Connections¶
Compatible Blocks
Check what blocks you can connect to Model Monitoring Inference Aggregator
in version v1
.
- inputs:
Path Deviation
,VLM as Classifier
,Byte Tracker
,Detections Filter
,Time in Zone
,Google Vision OCR
,OpenAI
,Model Monitoring Inference Aggregator
,SIFT Comparison
,Anthropic Claude
,Email Notification
,Time in Zone
,LMM
,Multi-Label Classification Model
,Line Counter
,OCR Model
,Object Detection Model
,Dynamic Crop
,CSV Formatter
,LMM For Classification
,JSON Parser
,Gaze Detection
,CogVLM
,Object Detection Model
,Detection Offset
,Florence-2 Model
,VLM as Detector
,Keypoint Detection Model
,Keypoint Detection Model
,PTZ Tracking (ONVIF)
.md),YOLO-World Model
,Moondream2
,Slack Notification
,Byte Tracker
,Roboflow Dataset Upload
,Dynamic Zone
,Roboflow Custom Metadata
,Local File Sink
,Detections Transformation
,Florence-2 Model
,Byte Tracker
,Path Deviation
,Identify Outliers
,VLM as Classifier
,Instance Segmentation Model
,Detections Consensus
,Single-Label Classification Model
,Bounding Rectangle
,Detections Classes Replacement
,Template Matching
,Stitch OCR Detections
,Detections Stabilizer
,Twilio SMS Notification
,Clip Comparison
,Webhook Sink
,Overlap Filter
,OpenAI
,OpenAI
,Detections Stitch
,Multi-Label Classification Model
,Time in Zone
,Single-Label Classification Model
,Llama 3.2 Vision
,SIFT Comparison
,Roboflow Dataset Upload
,Segment Anything 2 Model
,Velocity
,VLM as Detector
,Identify Changes
,Instance Segmentation Model
,Google Gemini
,Perspective Correction
,Detections Merge
- outputs:
Time in Zone
,OpenAI
,Line Counter
,Line Counter Visualization
,SIFT Comparison
,LMM
,Multi-Label Classification Model
,Model Comparison Visualization
,Line Counter
,Stability AI Image Generation
,Distance Measurement
,Keypoint Detection Model
,Keypoint Detection Model
,YOLO-World Model
,Slack Notification
,Label Visualization
,Image Threshold
,Local File Sink
,Cache Set
,Polygon Visualization
,Pixelate Visualization
,Instance Segmentation Model
,Detections Consensus
,Keypoint Visualization
,Polygon Zone Visualization
,Template Matching
,Stability AI Inpainting
,QR Code Generator
,OpenAI
,Stability AI Outpainting
,Detections Stitch
,Multi-Label Classification Model
,Time in Zone
,Roboflow Dataset Upload
,Blur Visualization
,Color Visualization
,Instance Segmentation Model
,Halo Visualization
,Mask Visualization
,Time in Zone
,Path Deviation
,Background Color Visualization
,Perception Encoder Embedding Model
,Trace Visualization
,Google Vision OCR
,Dot Visualization
,Model Monitoring Inference Aggregator
,Classification Label Visualization
,Email Notification
,Anthropic Claude
,Pixel Color Count
,Object Detection Model
,Dynamic Crop
,LMM For Classification
,Gaze Detection
,CogVLM
,Object Detection Model
,Image Preprocessing
,Circle Visualization
,CLIP Embedding Model
,Florence-2 Model
,PTZ Tracking (ONVIF)
.md),Moondream2
,Roboflow Dataset Upload
,Reference Path Visualization
,Cache Get
,Bounding Box Visualization
,Dynamic Zone
,Roboflow Custom Metadata
,Icon Visualization
,Florence-2 Model
,Path Deviation
,Ellipse Visualization
,Single-Label Classification Model
,Image Blur
,Detections Classes Replacement
,Size Measurement
,Crop Visualization
,Corner Visualization
,Triangle Visualization
,Twilio SMS Notification
,Clip Comparison
,Webhook Sink
,OpenAI
,Single-Label Classification Model
,Llama 3.2 Vision
,Segment Anything 2 Model
,Google Gemini
,Perspective Correction
Input and Output Bindings¶
The available connections depend on its binding kinds. Check what binding kinds
Model Monitoring Inference Aggregator
in version v1
has.
Bindings
-
input
predictions
(Union[classification_prediction
,object_detection_prediction
,instance_segmentation_prediction
,keypoint_detection_prediction
]): Model predictions to report to Roboflow Model Monitoring..model_id
(roboflow_model_id
): Model ID to report to Roboflow Model Monitoring..frequency
(string
): Frequency of reporting (in seconds). For example, if 5 is provided, the block will report an aggregated sample of predictions every 5 seconds..fire_and_forget
(boolean
): Boolean flag to run the block asynchronously (True) for faster workflows or synchronously (False) for debugging and error handling..
-
output
Example JSON definition of step Model Monitoring Inference Aggregator
in version v1
{
"name": "<your_step_name_here>",
"type": "roboflow_core/model_monitoring_inference_aggregator@v1",
"predictions": "$steps.my_step.predictions",
"model_id": "my_project/3",
"frequency": "3",
"unique_aggregator_key": "session-1v73kdhfse",
"fire_and_forget": true
}