Detections Stabilizer¶

Apply smoothing algorithms to reduce noise and flickering in tracked detections across video frames by using Kalman filtering to predict object velocities, exponential moving average to smooth bounding box positions, and gap filling to restore temporarily missing detections for improved tracking stability and smoother visualization workflows.

How This Block Works¶

This block stabilizes tracked detections by reducing jitter, smoothing positions, and filling gaps when objects temporarily disappear from detection. The block:

1. Receives tracked detection predictions with unique tracker IDs and an image with embedded video metadata
2. Extracts video metadata from the image:
3. Accesses video_metadata to get video_identifier
4. Uses video_identifier to maintain separate stabilization state for different videos
5. Validates that detections have tracker IDs (required for tracking object movement across frames)
6. Initializes or retrieves stabilization state for the video:
7. Maintains a cache of last known detections for each tracker_id per video
8. Creates or retrieves a Kalman filter for velocity prediction per video
9. Stores separate state for each video using video_identifier
10. Measures object velocities for existing tracks:
11. Calculates velocity by comparing current frame bounding box centers to previous frame centers
12. Computes displacement (change in position) for objects present in both current and previous frames
13. Velocity measurements are used to update the Kalman filter
14. Updates Kalman filter with velocity measurements:
15. Uses Kalman filtering to predict smoothed velocities based on historical measurements
16. Maintains a sliding window of velocity measurements (controlled by smoothing_window_size)
17. Applies exponential moving average within the Kalman filter to smooth velocity estimates
18. Filters out noise from detection inaccuracies and frame-to-frame variations
19. Smooths bounding boxes for objects present in current frame:
20. Applies exponential moving average smoothing to bounding box coordinates
21. Combines previous frame position with current frame position using bbox_smoothing_coefficient
22. Formula: smoothed_bbox = alpha * current_bbox + (1 - alpha) * previous_bbox
23. Reduces jitter and flickering from detection variations
24. Predicts positions for missing detections:
25. Uses Kalman filter predicted velocities to estimate positions of objects that disappeared
26. Applies predicted velocity to last known bounding box position
27. Fills gaps by restoring detections that were temporarily missing from current frame
28. Smooths predicted positions using exponential moving average
29. Manages tracking state:
30. Updates cache with current frame detections for next frame calculations
31. Removes tracking entries for objects that have been missing longer than smoothing_window_size frames
32. Maintains separate state per video_identifier
33. Merges and returns stabilized detections:
    • Combines smoothed detections (from current frame) and predicted detections (for missing objects)
    • Outputs stabilized detection objects with reduced noise and filled gaps
    • All detections maintain their tracker IDs for consistent tracking

The block uses two complementary smoothing techniques: Kalman filtering for velocity prediction (estimating how fast objects are moving) and exponential moving average for position smoothing (reducing bounding box jitter). The Kalman filter maintains a history of velocity measurements and uses statistical estimation to predict future velocities while filtering out noise. The exponential moving average smooths bounding box coordinates by blending current and previous positions. Gap filling uses predicted velocities to restore detections that temporarily disappear, helping maintain track continuity. Note: This block may produce short-lived bounding boxes for unstable trackers, as it attempts to fill gaps even when objects are inconsistently detected.

Common Use Cases¶

• Video Visualization: Reduce flickering and jitter in video annotations for smoother visualizations (e.g., smooth bounding box movements, reduce annotation noise, improve video visualization quality), enabling stable video visualization workflows
• Tracking Stability: Improve tracking stability when detections are noisy or inconsistent (e.g., stabilize noisy detections, reduce tracking jitter, improve tracking continuity), enabling stable tracking workflows
• Temporary Occlusion Handling: Fill gaps when objects are temporarily occluded or missing from detections (e.g., maintain tracks during brief occlusions, fill detection gaps, preserve tracking continuity), enabling occlusion handling workflows
• Real-Time Monitoring: Improve visual quality in real-time monitoring applications (e.g., smooth live video annotations, reduce flickering in monitoring displays, improve real-time visualization), enabling stable real-time monitoring workflows
• Analytics Accuracy: Reduce noise in analytics calculations that depend on stable detection positions (e.g., improve position-based analytics, reduce noise in measurements, stabilize movement calculations), enabling accurate analytics workflows
• Quality Control: Improve detection quality for downstream processing (e.g., smooth detections before analysis, reduce noise for better processing, stabilize inputs for other blocks), enabling quality improvement workflows

Connecting to Other Blocks¶

This block receives tracked detections and an image, and produces stabilized tracked_detections:

• After Byte Tracker blocks to stabilize tracked detections (e.g., smooth tracked object positions, reduce tracking jitter, fill tracking gaps), enabling tracking-stabilization workflows
• After object detection or instance segmentation blocks with tracking enabled to stabilize detections (e.g., smooth detection positions, reduce detection noise, improve tracking stability), enabling detection-stabilization workflows
• Before visualization blocks to display stabilized detections (e.g., visualize smooth bounding boxes, display stable annotations, show gap-filled detections), enabling stable visualization workflows
• Before analytics blocks to provide stable inputs for analysis (e.g., analyze stabilized positions, process smooth movement data, work with gap-filled detections), enabling stable analytics workflows
• Before velocity or path analysis blocks to improve measurement accuracy (e.g., calculate velocities from stable positions, analyze paths from smooth trajectories, measure from gap-filled detections), enabling accurate measurement workflows
• In video processing pipelines where detection stability is required for downstream processing (e.g., stabilize detections in processing chains, improve quality for analysis, reduce noise in pipelines), enabling stable video processing workflows

Requirements¶

This block requires tracked detections with tracker_id information (detections must come from a tracking block like Byte Tracker). The image's video_metadata should include video_identifier to maintain separate stabilization state for different videos. The block maintains persistent stabilization state across frames for each video, so it should be used in video workflows where frames are processed sequentially. For optimal stabilization, detections should be provided consistently across frames with valid tracker IDs. The smoothing_window_size controls how many historical velocity measurements are used for Kalman filtering and how long missing detections are retained. The bbox_smoothing_coefficient (0-1) controls the balance between current and previous positions - lower values provide more smoothing but slower response to changes, higher values provide less smoothing but faster response. Note: This block may produce short-lived bounding boxes for unstable trackers as it attempts to fill gaps even when objects are inconsistently detected.

Type identifier¶

Use the following identifier in step "type" field: roboflow_core/stabilize_detections@v1to add the block as as step in your workflow.

Properties¶

The Refs column marks possibility to parametrise the property with dynamic values available in workflow runtime. See Bindings for more info.

Available Connections¶

Input and Output Bindings¶

The available connections depend on its binding kinds. Check what binding kinds Detections Stabilizer in version v1 has.

{
    "name": "<your_step_name_here>",
    "type": "roboflow_core/stabilize_detections@v1",
    "image": "<block_does_not_provide_example>",
    "detections": "$steps.object_detection_model.predictions",
    "smoothing_window_size": 3,
    "bbox_smoothing_coefficient": 0.2
}