Detections Stabilizer¶

Class: StabilizeTrackedDetectionsBlockV1

Source: inference.core.workflows.core_steps.transformations.stabilize_detections.v1.StabilizeTrackedDetectionsBlockV1

Apply smoothing algorithms to reduce noise and flickering in tracked detections across video frames by using Kalman filtering to predict object velocities, exponential moving average to smooth bounding box positions, and gap filling to restore temporarily missing detections for improved tracking stability and smoother visualization workflows.

How This Block Works¶

This block stabilizes tracked detections by reducing jitter, smoothing positions, and filling gaps when objects temporarily disappear from detection. The block:

Receives tracked detection predictions with unique tracker IDs and an image with embedded video metadata
Extracts video metadata from the image:
Accesses video_metadata to get video_identifier
Uses video_identifier to maintain separate stabilization state for different videos
Validates that detections have tracker IDs (required for tracking object movement across frames)
Initializes or retrieves stabilization state for the video:
Maintains a cache of last known detections for each tracker_id per video
Creates or retrieves a Kalman filter for velocity prediction per video
Stores separate state for each video using video_identifier
Measures object velocities for existing tracks:
Calculates velocity by comparing current frame bounding box centers to previous frame centers
Computes displacement (change in position) for objects present in both current and previous frames
Velocity measurements are used to update the Kalman filter
Updates Kalman filter with velocity measurements:
Uses Kalman filtering to predict smoothed velocities based on historical measurements
Maintains a sliding window of velocity measurements (controlled by smoothing_window_size)
Applies exponential moving average within the Kalman filter to smooth velocity estimates
Filters out noise from detection inaccuracies and frame-to-frame variations
Smooths bounding boxes for objects present in current frame:
Applies exponential moving average smoothing to bounding box coordinates
Combines previous frame position with current frame position using bbox_smoothing_coefficient
Formula: smoothed_bbox = alpha * current_bbox + (1 - alpha) * previous_bbox
Reduces jitter and flickering from detection variations
Predicts positions for missing detections:
Uses Kalman filter predicted velocities to estimate positions of objects that disappeared
Applies predicted velocity to last known bounding box position
Fills gaps by restoring detections that were temporarily missing from current frame
Smooths predicted positions using exponential moving average
Manages tracking state:
Updates cache with current frame detections for next frame calculations
Removes tracking entries for objects that have been missing longer than smoothing_window_size frames
Maintains separate state per video_identifier
Merges and returns stabilized detections:
- Combines smoothed detections (from current frame) and predicted detections (for missing objects)
- Outputs stabilized detection objects with reduced noise and filled gaps
- All detections maintain their tracker IDs for consistent tracking

The block uses two complementary smoothing techniques: Kalman filtering for velocity prediction (estimating how fast objects are moving) and exponential moving average for position smoothing (reducing bounding box jitter). The Kalman filter maintains a history of velocity measurements and uses statistical estimation to predict future velocities while filtering out noise. The exponential moving average smooths bounding box coordinates by blending current and previous positions. Gap filling uses predicted velocities to restore detections that temporarily disappear, helping maintain track continuity. Note: This block may produce short-lived bounding boxes for unstable trackers, as it attempts to fill gaps even when objects are inconsistently detected.

Common Use Cases¶

Video Visualization: Reduce flickering and jitter in video annotations for smoother visualizations (e.g., smooth bounding box movements, reduce annotation noise, improve video visualization quality), enabling stable video visualization workflows
Tracking Stability: Improve tracking stability when detections are noisy or inconsistent (e.g., stabilize noisy detections, reduce tracking jitter, improve tracking continuity), enabling stable tracking workflows
Temporary Occlusion Handling: Fill gaps when objects are temporarily occluded or missing from detections (e.g., maintain tracks during brief occlusions, fill detection gaps, preserve tracking continuity), enabling occlusion handling workflows
Real-Time Monitoring: Improve visual quality in real-time monitoring applications (e.g., smooth live video annotations, reduce flickering in monitoring displays, improve real-time visualization), enabling stable real-time monitoring workflows
Analytics Accuracy: Reduce noise in analytics calculations that depend on stable detection positions (e.g., improve position-based analytics, reduce noise in measurements, stabilize movement calculations), enabling accurate analytics workflows
Quality Control: Improve detection quality for downstream processing (e.g., smooth detections before analysis, reduce noise for better processing, stabilize inputs for other blocks), enabling quality improvement workflows

Connecting to Other Blocks¶

This block receives tracked detections and an image, and produces stabilized tracked_detections:

After Byte Tracker blocks to stabilize tracked detections (e.g., smooth tracked object positions, reduce tracking jitter, fill tracking gaps), enabling tracking-stabilization workflows
After object detection or instance segmentation blocks with tracking enabled to stabilize detections (e.g., smooth detection positions, reduce detection noise, improve tracking stability), enabling detection-stabilization workflows
Before visualization blocks to display stabilized detections (e.g., visualize smooth bounding boxes, display stable annotations, show gap-filled detections), enabling stable visualization workflows
Before analytics blocks to provide stable inputs for analysis (e.g., analyze stabilized positions, process smooth movement data, work with gap-filled detections), enabling stable analytics workflows
Before velocity or path analysis blocks to improve measurement accuracy (e.g., calculate velocities from stable positions, analyze paths from smooth trajectories, measure from gap-filled detections), enabling accurate measurement workflows
In video processing pipelines where detection stability is required for downstream processing (e.g., stabilize detections in processing chains, improve quality for analysis, reduce noise in pipelines), enabling stable video processing workflows

Requirements¶

This block requires tracked detections with tracker_id information (detections must come from a tracking block like Byte Tracker). The image's video_metadata should include video_identifier to maintain separate stabilization state for different videos. The block maintains persistent stabilization state across frames for each video, so it should be used in video workflows where frames are processed sequentially. For optimal stabilization, detections should be provided consistently across frames with valid tracker IDs. The smoothing_window_size controls how many historical velocity measurements are used for Kalman filtering and how long missing detections are retained. The bbox_smoothing_coefficient (0-1) controls the balance between current and previous positions - lower values provide more smoothing but slower response to changes, higher values provide less smoothing but faster response. Note: This block may produce short-lived bounding boxes for unstable trackers as it attempts to fill gaps even when objects are inconsistently detected.

Type identifier¶

Use the following identifier in step "type" field: roboflow_core/stabilize_detections@v1to add the block as as step in your workflow.

Properties¶

Name	Type	Description	Refs
`name`	`str`	Enter a unique identifier for this step..	❌
`smoothing_window_size`	`int`	Size of the sliding window for velocity smoothing in Kalman filter, controlling how many historical velocity measurements are used. Also determines how long missing detections are retained before removal. Larger values provide more smoothing but slower adaptation to changes. Smaller values provide less smoothing but faster adaptation. Detections missing for longer than this number of frames are removed from tracking state. Typical range: 3-10 frames..	✅
`bbox_smoothing_coefficient`	`float`	Exponential moving average coefficient (alpha) for bounding box position smoothing, range 0.0-1.0. Controls the blend between current and previous bounding box positions: smoothed_bbox = alpha * current + (1-alpha) * previous. Lower values (closer to 0) provide more smoothing - slower response to changes, less jitter. Higher values (closer to 1) provide less smoothing - faster response to changes, more jitter. Default 0.2 balances smoothness and responsiveness. Typical range: 0.1-0.5..	✅

The Refs column marks possibility to parametrise the property with dynamic values available in workflow runtime. See Bindings for more info.

Available Connections¶

Compatible Blocks

Check what blocks you can connect to Detections Stabilizer in version v1.

Input and Output Bindings¶

The available connections depend on its binding kinds. Check what binding kinds Detections Stabilizer in version v1 has.

Bindings

input
- image (image): Image with embedded video metadata. The video_metadata contains video_identifier to maintain separate stabilization state for different videos. Required for persistent state management across frames..
- detections (Union[object_detection_prediction, instance_segmentation_prediction]): Tracked object detection or instance segmentation predictions. Must include tracker_id information from a tracking block. The block applies Kalman filtering for velocity prediction, exponential moving average for position smoothing, and gap filling for missing detections. Output detections are stabilized with reduced noise and jitter..
- smoothing_window_size (integer): Size of the sliding window for velocity smoothing in Kalman filter, controlling how many historical velocity measurements are used. Also determines how long missing detections are retained before removal. Larger values provide more smoothing but slower adaptation to changes. Smaller values provide less smoothing but faster adaptation. Detections missing for longer than this number of frames are removed from tracking state. Typical range: 3-10 frames..
- bbox_smoothing_coefficient (float_zero_to_one): Exponential moving average coefficient (alpha) for bounding box position smoothing, range 0.0-1.0. Controls the blend between current and previous bounding box positions: smoothed_bbox = alpha * current + (1-alpha) * previous. Lower values (closer to 0) provide more smoothing - slower response to changes, less jitter. Higher values (closer to 1) provide less smoothing - faster response to changes, more jitter. Default 0.2 balances smoothness and responsiveness. Typical range: 0.1-0.5..
output
- tracked_detections (Union[object_detection_prediction, instance_segmentation_prediction]): Prediction with detected bounding boxes in form of sv.Detections(...) object if object_detection_prediction or Prediction with detected bounding boxes and segmentation masks in form of sv.Detections(...) object if instance_segmentation_prediction.

Example JSON definition of step Detections Stabilizer in version v1

{
    "name": "<your_step_name_here>",
    "type": "roboflow_core/stabilize_detections@v1",
    "image": "<block_does_not_provide_example>",
    "detections": "$steps.object_detection_model.predictions",
    "smoothing_window_size": 3,
    "bbox_smoothing_coefficient": 0.2
}