Model Monitoring Inference Aggregator¶
Class: ModelMonitoringInferenceAggregatorBlockV1
This block 📊 transforms inference data reporting to a whole new level by periodically aggregating and sending a curated sample of predictions to Roboflow Model Monitoring.
✨ Key Features¶
-
Effortless Aggregation: Collects and organizes predictions in-memory, ensuring only the most relevant and confident predictions are reported.
-
Customizable Reporting Intervals: Choose how frequently (in seconds) data should be sent—ensuring optimal balance between granularity and resource efficiency.
-
Debug-Friendly Mode: Fine-tune operations by enabling or disabling asynchronous background execution.
🔍 Why Use This Block?¶
This block is a game-changer for projects relying on video processing in Workflows. With its aggregation process, it identifies the most confident predictions across classes and sends them at regular intervals in small messages to Roboflow backend - ensuring that video processing performance is impacted to the least extent.
Perfect for:
-
Monitoring production line performance in real-time 🏭.
-
Debugging and validating your model’s performance over time ⏱️.
-
Providing actionable insights from inference workflows with minimal overhead 🔧.
🚨 Limitations¶
-
The block is should not be relied on when running Workflow in
inference
server or via HTTP request to Roboflow hosted platform, as the internal state is not persisted in a memory that would be accessible for all requests to the server, causing aggregation to only have a scope of single request. We will solve that problem in future releases if proven to be serious limitation for clients. -
This block do not have ability to separate aggregations for multiple videos processed by
InferencePipeline
- effectively aggregating data for all video feeds connected to single process runningInferencePipeline
.
Type identifier¶
Use the following identifier in step "type"
field: roboflow_core/model_monitoring_inference_aggregator@v1
to add the block as
as step in your workflow.
Properties¶
Name | Type | Description | Refs |
---|---|---|---|
name |
str |
Enter a unique identifier for this step.. | ❌ |
frequency |
int |
Frequency of reporting (in seconds). For example, if 5 is provided, the block will report an aggregated sample of predictions every 5 seconds.. | ✅ |
unique_aggregator_key |
str |
Unique key used internally to track the session of inference results reporting. Must be unique for each step in your Workflow.. | ❌ |
fire_and_forget |
bool |
Boolean flag to run the block asynchronously (True) for faster workflows or synchronously (False) for debugging and error handling.. | ✅ |
The Refs column marks possibility to parametrise the property with dynamic values available
in workflow
runtime. See Bindings for more info.
Available Connections¶
Compatible Blocks
Check what blocks you can connect to Model Monitoring Inference Aggregator
in version v1
.
- inputs:
Byte Tracker
,VLM as Classifier
,Time in Zone
,LMM For Classification
,Identify Outliers
,Perspective Correction
,CSV Formatter
,LMM
,PTZ Tracking (ONVIF)
.md),Florence-2 Model
,Florence-2 Model
,OpenAI
,Keypoint Detection Model
,Multi-Label Classification Model
,Detections Combine
,Detection Offset
,CogVLM
,EasyOCR
,Byte Tracker
,Stitch OCR Detections
,VLM as Detector
,VLM as Classifier
,Twilio SMS Notification
,Keypoint Detection Model
,Google Vision OCR
,Identify Changes
,Roboflow Dataset Upload
,Email Notification
,Instance Segmentation Model
,Clip Comparison
,Template Matching
,OCR Model
,Detections Stabilizer
,Llama 3.2 Vision
,Instance Segmentation Model
,Dynamic Crop
,Roboflow Dataset Upload
,Local File Sink
,Webhook Sink
,OpenAI
,Detections Stitch
,SIFT Comparison
,Time in Zone
,Velocity
,Object Detection Model
,Detections Transformation
,Byte Tracker
,Overlap Filter
,SIFT Comparison
,Roboflow Custom Metadata
,Object Detection Model
,Dynamic Zone
,Model Monitoring Inference Aggregator
,Gaze Detection
,Line Counter
,Anthropic Claude
,Time in Zone
,Path Deviation
,OpenAI
,Slack Notification
,JSON Parser
,Detections Filter
,YOLO-World Model
,Single-Label Classification Model
,Detections Classes Replacement
,Bounding Rectangle
,VLM as Detector
,Google Gemini
,Multi-Label Classification Model
,Detections Merge
,Path Deviation
,Single-Label Classification Model
,Detections Consensus
,Moondream2
,Segment Anything 2 Model
- outputs:
Distance Measurement
,Time in Zone
,Dot Visualization
,Perspective Correction
,Blur Visualization
,OpenAI
,Keypoint Detection Model
,Stability AI Outpainting
,Twilio SMS Notification
,Google Vision OCR
,Roboflow Dataset Upload
,Email Notification
,Instance Segmentation Model
,Llama 3.2 Vision
,Bounding Box Visualization
,Reference Path Visualization
,Image Preprocessing
,Local File Sink
,OpenAI
,Detections Stitch
,Object Detection Model
,Stability AI Image Generation
,SIFT Comparison
,Roboflow Custom Metadata
,Cache Get
,Model Comparison Visualization
,Dynamic Zone
,Line Counter
,Time in Zone
,Polygon Visualization
,Slack Notification
,Triangle Visualization
,Single-Label Classification Model
,YOLO-World Model
,Detections Classes Replacement
,Circle Visualization
,Label Visualization
,Google Gemini
,Multi-Label Classification Model
,Path Deviation
,Detections Consensus
,Moondream2
,Segment Anything 2 Model
,Polygon Zone Visualization
,LMM For Classification
,Morphological Transformation
,Size Measurement
,Corner Visualization
,LMM
,Pixel Color Count
,PTZ Tracking (ONVIF)
.md),Florence-2 Model
,Image Threshold
,Florence-2 Model
,Halo Visualization
,Multi-Label Classification Model
,CogVLM
,Line Counter Visualization
,Perception Encoder Embedding Model
,Stitch OCR Detections
,Keypoint Detection Model
,CLIP Embedding Model
,Clip Comparison
,Keypoint Visualization
,Template Matching
,Line Counter
,Instance Segmentation Model
,Dynamic Crop
,Roboflow Dataset Upload
,Mask Visualization
,Background Color Visualization
,Webhook Sink
,QR Code Generator
,Trace Visualization
,Time in Zone
,Contrast Equalization
,Cache Set
,Crop Visualization
,Object Detection Model
,Pixelate Visualization
,Model Monitoring Inference Aggregator
,Gaze Detection
,Anthropic Claude
,OpenAI
,Path Deviation
,Classification Label Visualization
,Image Blur
,Stability AI Inpainting
,Icon Visualization
,Ellipse Visualization
,Color Visualization
,Single-Label Classification Model
Input and Output Bindings¶
The available connections depend on its binding kinds. Check what binding kinds
Model Monitoring Inference Aggregator
in version v1
has.
Bindings
-
input
predictions
(Union[classification_prediction
,object_detection_prediction
,instance_segmentation_prediction
,keypoint_detection_prediction
]): Model predictions to report to Roboflow Model Monitoring..model_id
(roboflow_model_id
): Model ID to report to Roboflow Model Monitoring..frequency
(string
): Frequency of reporting (in seconds). For example, if 5 is provided, the block will report an aggregated sample of predictions every 5 seconds..fire_and_forget
(boolean
): Boolean flag to run the block asynchronously (True) for faster workflows or synchronously (False) for debugging and error handling..
-
output
Example JSON definition of step Model Monitoring Inference Aggregator
in version v1
{
"name": "<your_step_name_here>",
"type": "roboflow_core/model_monitoring_inference_aggregator@v1",
"predictions": "$steps.my_step.predictions",
"model_id": "my_project/3",
"frequency": "3",
"unique_aggregator_key": "session-1v73kdhfse",
"fire_and_forget": true
}