Model Monitoring Inference Aggregator¶
Class: ModelMonitoringInferenceAggregatorBlockV1
This block 📊 transforms inference data reporting to a whole new level by periodically aggregating and sending a curated sample of predictions to Roboflow Model Monitoring.
✨ Key Features¶
-
Effortless Aggregation: Collects and organizes predictions in-memory, ensuring only the most relevant and confident predictions are reported.
-
Customizable Reporting Intervals: Choose how frequently (in seconds) data should be sent—ensuring optimal balance between granularity and resource efficiency.
-
Debug-Friendly Mode: Fine-tune operations by enabling or disabling asynchronous background execution.
🔍 Why Use This Block?¶
This block is a game-changer for projects relying on video processing in Workflows. With its aggregation process, it identifies the most confident predictions across classes and sends them at regular intervals in small messages to Roboflow backend - ensuring that video processing performance is impacted to the least extent.
Perfect for:
-
Monitoring production line performance in real-time 🏭.
-
Debugging and validating your model’s performance over time ⏱️.
-
Providing actionable insights from inference workflows with minimal overhead 🔧.
🚨 Limitations¶
-
The block is should not be relied on when running Workflow in
inference
server or via HTTP request to Roboflow hosted platform, as the internal state is not persisted in a memory that would be accessible for all requests to the server, causing aggregation to only have a scope of single request. We will solve that problem in future releases if proven to be serious limitation for clients. -
This block do not have ability to separate aggregations for multiple videos processed by
InferencePipeline
- effectively aggregating data for all video feeds connected to single process runningInferencePipeline
.
Type identifier¶
Use the following identifier in step "type"
field: roboflow_core/model_monitoring_inference_aggregator@v1
to add the block as
as step in your workflow.
Properties¶
Name | Type | Description | Refs |
---|---|---|---|
name |
str |
Enter a unique identifier for this step.. | ❌ |
frequency |
int |
Frequency of reporting (in seconds). For example, if 5 is provided, the block will report an aggregated sample of predictions every 5 seconds.. | ✅ |
unique_aggregator_key |
str |
Unique key used internally to track the session of inference results reporting. Must be unique for each step in your Workflow.. | ❌ |
fire_and_forget |
bool |
Boolean flag to run the block asynchronously (True) for faster workflows or synchronously (False) for debugging and error handling.. | ✅ |
The Refs column marks possibility to parametrise the property with dynamic values available
in workflow
runtime. See Bindings for more info.
Available Connections¶
Compatible Blocks
Check what blocks you can connect to Model Monitoring Inference Aggregator
in version v1
.
- inputs:
LMM
,Stitch OCR Detections
,LMM For Classification
,Clip Comparison
,CSV Formatter
,Instance Segmentation Model
,CogVLM
,Byte Tracker
,Roboflow Dataset Upload
,Multi-Label Classification Model
,Twilio SMS Notification
,Object Detection Model
,SIFT Comparison
,Florence-2 Model
,Byte Tracker
,OpenAI
,Path Deviation
,Single-Label Classification Model
,Roboflow Dataset Upload
,Detections Stitch
,Llama 3.2 Vision
,Slack Notification
,Object Detection Model
,VLM as Classifier
,Moondream2
,Bounding Rectangle
,Detections Consensus
,Detections Merge
,Dynamic Zone
,Detection Offset
,Overlap Filter
,Time in Zone
,Keypoint Detection Model
,Model Monitoring Inference Aggregator
,Perspective Correction
,Dynamic Crop
,Detections Transformation
,Detections Classes Replacement
,Time in Zone
,VLM as Detector
,Identify Changes
,Florence-2 Model
,Local File Sink
,Path Deviation
,VLM as Detector
,Google Vision OCR
,OpenAI
,Gaze Detection
,Multi-Label Classification Model
,Single-Label Classification Model
,Byte Tracker
,Detections Filter
,Identify Outliers
,Google Gemini
,Keypoint Detection Model
,VLM as Classifier
,Detections Stabilizer
,OCR Model
,Template Matching
,Velocity
,Roboflow Custom Metadata
,Anthropic Claude
,Webhook Sink
,SIFT Comparison
,YOLO-World Model
,JSON Parser
,Instance Segmentation Model
,Segment Anything 2 Model
,Email Notification
,Line Counter
- outputs:
LMM
,Classification Label Visualization
,LMM For Classification
,Clip Comparison
,Line Counter
,Instance Segmentation Model
,CogVLM
,Roboflow Dataset Upload
,Distance Measurement
,Multi-Label Classification Model
,Twilio SMS Notification
,Object Detection Model
,Florence-2 Model
,OpenAI
,Label Visualization
,Path Deviation
,Single-Label Classification Model
,Roboflow Dataset Upload
,Detections Stitch
,Bounding Box Visualization
,Llama 3.2 Vision
,Model Comparison Visualization
,Slack Notification
,Object Detection Model
,Pixel Color Count
,Detections Consensus
,Dynamic Zone
,Halo Visualization
,Triangle Visualization
,Time in Zone
,Keypoint Detection Model
,Model Monitoring Inference Aggregator
,Reference Path Visualization
,Perspective Correction
,Time in Zone
,Dynamic Crop
,Cache Get
,Florence-2 Model
,Cache Set
,Local File Sink
,Path Deviation
,OpenAI
,Google Vision OCR
,Stability AI Image Generation
,Ellipse Visualization
,Size Measurement
,Blur Visualization
,Dot Visualization
,Circle Visualization
,Image Blur
,CLIP Embedding Model
,Background Color Visualization
,Gaze Detection
,Multi-Label Classification Model
,Single-Label Classification Model
,Color Visualization
,Pixelate Visualization
,Google Gemini
,Keypoint Detection Model
,Stability AI Inpainting
,Polygon Zone Visualization
,Keypoint Visualization
,Template Matching
,Mask Visualization
,Image Preprocessing
,Line Counter Visualization
,Roboflow Custom Metadata
,Anthropic Claude
,Webhook Sink
,SIFT Comparison
,YOLO-World Model
,Trace Visualization
,Instance Segmentation Model
,Corner Visualization
,Polygon Visualization
,Segment Anything 2 Model
,Crop Visualization
,Email Notification
,Line Counter
,Image Threshold
Input and Output Bindings¶
The available connections depend on its binding kinds. Check what binding kinds
Model Monitoring Inference Aggregator
in version v1
has.
Bindings
-
input
predictions
(Union[object_detection_prediction
,keypoint_detection_prediction
,classification_prediction
,instance_segmentation_prediction
]): Model predictions to report to Roboflow Model Monitoring..model_id
(roboflow_model_id
): Model ID to report to Roboflow Model Monitoring..frequency
(string
): Frequency of reporting (in seconds). For example, if 5 is provided, the block will report an aggregated sample of predictions every 5 seconds..fire_and_forget
(boolean
): Boolean flag to run the block asynchronously (True) for faster workflows or synchronously (False) for debugging and error handling..
-
output
Example JSON definition of step Model Monitoring Inference Aggregator
in version v1
{
"name": "<your_step_name_here>",
"type": "roboflow_core/model_monitoring_inference_aggregator@v1",
"predictions": "$steps.my_step.predictions",
"model_id": "my_project/3",
"frequency": "3",
"unique_aggregator_key": "session-1v73kdhfse",
"fire_and_forget": true
}