PTZ Tracking (ONVIF)

Class: ONVIFSinkBlockV1

Source: inference.core.workflows.core_steps.sinks.onvif_movement.v1.ONVIFSinkBlockV1

Control an ONVIF-compatible PTZ (Pan-Tilt-Zoom) camera to automatically follow detected objects, move to preset positions, and maintain smooth tracking using PID control for surveillance, security monitoring, and automated camera control workflows.

How This Block Works

This block controls PTZ cameras through the ONVIF protocol to automatically track and follow objects in real-time. The block:

1. Receives object detection or instance segmentation predictions from upstream blocks
2. Connects to an ONVIF-compatible PTZ camera using IP address, port, username, and password credentials
3. Determines the movement mode (Follow or Go To Preset):

For Follow Mode: - Selects the target object to track: either the highest confidence detection or a specific tracked object (if tracker IDs are present) - Calculates the position error by comparing the object's bounding box center to the frame center - Uses PID (Proportional-Integral-Derivative) control to calculate smooth movement commands: - Proportional (Kp): Responds to current position error - Integral (Ki): Corrects steady-state errors over time - Derivative (Kd): Predicts future error and dampens oscillations - Normalizes movement commands to the camera's velocity limits and applies rate limiting to prevent command flooding - Sends continuous movement commands to the camera via ONVIF ContinuousMove service - Monitors the dead zone (region around center where camera stops moving) to prevent hunting behavior - Optionally zooms into the object when it's centered, adjusting zoom speed to fill the frame - Maintains tracking of a specific object using tracker IDs until it disappears or tracking is reset - Automatically moves to a preset position after a configurable idle period when no objects are detected

For Go To Preset Mode: - Moves the camera to a predefined preset position using the ONVIF GotoPreset service - Requires the camera to support preset functionality and have presets configured - Uses the preset name specified in the configuration

1. Handles camera communication asynchronously using a separate event loop to prevent blocking workflow execution
2. Manages camera state including seeking status, tracked object IDs, zoom state, and movement history
3. Returns two outputs:
4. predictions: The detection being tracked (empty if no object is being followed)
5. seeking: Boolean indicating whether the camera is currently moving/seeking an object

The block uses PID control to calculate smooth, proportional movement commands based on the distance between the object center and frame center. Movement is normalized as a percentage of the camera's maximum speed, and commands are rate-limited to prevent overwhelming the camera with updates. The dead zone prevents small movements when the object is near the center, reducing hunting behavior. When zoom is enabled, the camera first centers the object with pan/tilt, then zooms in to fill the frame while maintaining the object in view.

Common Use Cases

• Surveillance and Security: Automatically track individuals or vehicles in surveillance scenarios (e.g., follow suspicious activity, track intruders, monitor security perimeters), enabling automated surveillance workflows
• Sports and Event Coverage: Track athletes or objects during sports events or performances (e.g., follow players on field, track ball movement, cover event action), enabling automated sports coverage workflows
• Wildlife Monitoring: Follow animals or wildlife in natural habitats (e.g., track bird movements, follow animals in reserves, monitor wildlife behavior), enabling wildlife observation workflows
• Industrial Monitoring: Automatically follow objects or personnel in industrial settings (e.g., track equipment movement, monitor worker activities, follow vehicles in facilities), enabling industrial automation workflows
• Traffic Monitoring: Track vehicles or objects in traffic scenarios (e.g., follow vehicles through intersections, track traffic violations, monitor road activity), enabling automated traffic monitoring workflows
• Retail Analytics: Track customers or products in retail environments (e.g., follow customer paths, track product interactions, monitor shopping behavior), enabling retail analytics workflows

Connecting to Other Blocks

This block receives predictions and produces camera control commands and tracking status:

• After object detection or instance segmentation blocks to track detected objects with the camera (e.g., follow detected people, track detected vehicles, monitor detected objects), enabling detection-to-camera-tracking workflows
• After Byte Tracker blocks to follow specific tracked objects with consistent IDs (e.g., follow tracked person across frames, maintain tracking of specific vehicle, monitor tracked object persistently), enabling tracking-to-camera workflows
• After detection filter blocks to track specific object classes or filtered detections (e.g., track only specific classes, follow filtered detections, monitor selected objects), enabling filtered-tracking workflows
• Before visualization blocks to display camera movement status and tracked objects (e.g., visualize tracking status, display seeking indicator, show camera control feedback), enabling camera control visualization workflows
• Before notification blocks to alert when camera starts or stops tracking (e.g., notify when tracking begins, alert on tracking loss, report camera status), enabling camera status notification workflows
• In surveillance and monitoring pipelines where automated camera control is part of a larger security or monitoring system (e.g., automated security systems, monitoring pipelines, camera control chains), enabling comprehensive surveillance workflows

Requirements

This block requires an ONVIF-compatible PTZ camera with network access. The camera must support the ONVIF ContinuousMove service for Follow mode and GotoPreset service for preset movement. For optimal performance, use a camera with variable speed movement capability - cameras without variable speed can use the simulate_variable_speed option but may experience jerky movement. The block must run in local execution mode (not suitable for remote/cloud execution). PID tuning is recommended to achieve smooth tracking without overshooting or hunting - adjust pid_kp, pid_ki, and pid_kd parameters based on camera responsiveness and video latency. For accurate tracking, use an eager buffer consumption strategy to minimize lag between camera movement and video feedback. The camera must have presets configured if using preset movement or auto-reset functionality.

Type identifier

Use the following identifier in step "type" field: roboflow_core/onvif_sink@v1to add the block as as step in your workflow.

Properties

Name	Type	Description	Refs
name	str	Enter a unique identifier for this step..	❌
camera_ip	str	Camera IP address or hostname for ONVIF connection. Must be reachable from the workflow execution environment..	✅
camera_port	int	Camera ONVIF service port (typically 80, 8080, or camera-specific port). Must match the camera's ONVIF configuration..	✅
camera_username	str	Camera username for ONVIF authentication. Must have PTZ control permissions on the camera..	✅
camera_password	str	Camera password for ONVIF authentication. Should be stored as a secret for security..	✅
movement_type	str	Movement mode for the camera. 'Follow' mode tracks detected objects using PID control. 'Go To Preset' mode moves the camera to a predefined preset position (requires default_position_preset to be configured)..	❌
simulate_variable_speed	bool	Enable variable speed simulation for cameras without native variable speed support. When enabled, sends 100% speed commands followed by stop commands to approximate percentage speeds. May result in jerky movement - only use if camera lacks variable speed capability..	✅
zoom_if_able	bool	Enable automatic zoom to fill frame with the tracked object. When enabled, camera first centers the object with pan/tilt, then zooms in until the object fills the frame. Requires camera to support zoom functionality..	✅
follow_tracker	bool	Enable persistent tracking using tracker IDs. When enabled, camera locks onto the tracker ID of the highest confidence detection and continues following that specific object until it disappears or tracking resets. Requires a Byte Tracker block in the workflow to assign tracker IDs..	✅
dead_zone	int	Dead zone size in pixels around frame center where camera movement stops. Prevents hunting behavior when object is near center. Larger values reduce pan/tilt hunting but may cause zoom hunting. Smaller values improve zoom stability but may cause pan/tilt oscillations. Typical range: 30-100 pixels..	✅
default_position_preset	str	Preset name for default/home position. Camera will return to this preset after idle period (if move_to_position_after_idle_seconds is set) or when using Go To Preset mode. Must match exactly a preset name configured on the camera. Required for preset movement functionality..	✅
move_to_position_after_idle_seconds	int	Auto-reset time in seconds. After camera stops seeking/moving for this duration, automatically moves to default_position_preset. Set to 0 to disable auto-reset. Requires default_position_preset to be configured..	✅
camera_update_rate_limit	int	Minimum time in milliseconds between camera movement commands. Rate limits ONVIF updates to prevent overwhelming the camera. Lower values provide more responsive tracking but may overload slower cameras. Higher values reduce camera load but may cause less smooth movement. Typical range: 100-500ms..	✅
flip_x_movement	bool	Invert horizontal (pan) movement direction. Enable if the camera image is mirrored horizontally and movement appears reversed. Use to correct camera movement when image is flipped..	✅
flip_y_movement	bool	Invert vertical (tilt) movement direction. Enabled by default as many cameras have inverted Y-axis. Disable if vertical movement appears reversed..	✅
minimum_camera_speed	float	Minimum movement speed as percentage (0.0-1.0). Movement commands below this threshold are boosted to this minimum. Some cameras ignore very low speeds - increase if camera doesn't respond to small movements. Typical range: 0.02-0.1 (2%-10%)..	✅
pid_kp	float	PID proportional gain (Kp). Controls response to position error - higher values make camera respond faster but may cause overshooting and hunting. Lower values reduce hunting but make tracking slower. Start with default and adjust based on camera responsiveness. Typical range: 0.1-0.5..	✅
pid_ki	float	PID integral gain (Ki). Eliminates steady-state error by accumulating error over time. Usually kept at 0 as it can cause oscillations. Increase slightly (0.01-0.1) if camera consistently stops slightly off-center despite small errors..	✅
pid_kd	float	PID derivative gain (Kd). Predicts future error and dampens oscillations. Higher values improve stability with video lag but excessive values can cause hunting. Increase (1-5) if there's significant delay between camera movement and video feedback. Decrease if tracking appears jerky..	✅

The Refs column marks possibility to parametrise the property with dynamic values available in workflow runtime. See Bindings for more info.

Available Connections

Compatible Blocks

Check what blocks you can connect to PTZ Tracking (ONVIF) in version v1.

• inputs: Detections Stitch, Roboflow Dataset Upload, Detections Classes Replacement, LMM, Florence-2 Model, Anthropic Claude, Google Gemini, Camera Focus, Llama 3.2 Vision, Motion Detection, VLM As Detector, SIFT Comparison, Line Counter, Roboflow Dataset Upload, Stitch OCR Detections, SIFT Comparison, Dynamic Zone, Distance Measurement, Email Notification, SAM 3, Detections Stabilizer, Slack Notification, Object Detection Model, Path Deviation, EasyOCR, Florence-2 Model, SAM 3, CSV Formatter, Object Detection Model, Anthropic Claude, Detections Combine, OpenAI, Google Gemini, Keypoint Detection Model, Anthropic Claude, Pixel Color Count, VLM As Classifier, Time in Zone, Clip Comparison, Detections Merge, CogVLM, Byte Tracker, Twilio SMS Notification, VLM As Detector, PTZ Tracking (ONVIF).md), Instance Segmentation Model, OpenAI, Detections Transformation, OCR Model, Moondream2, Stitch OCR Detections, Dynamic Crop, Model Monitoring Inference Aggregator, Detection Offset, Byte Tracker, OpenAI, YOLO-World Model, Cosine Similarity, Time in Zone, Velocity, Email Notification, Instance Segmentation Model, Path Deviation, Camera Focus, Twilio SMS/MMS Notification, LMM For Classification, Segment Anything 2 Model, Bounding Rectangle, Detections Filter, Roboflow Custom Metadata, Perspective Correction, Multi-Label Classification Model, Image Contours, Detection Event Log, Local File Sink, Identify Changes, Byte Tracker, JSON Parser, Google Vision OCR, Time in Zone, SAM 3, Identify Outliers, Detections Consensus, Template Matching, Detections List Roll-Up, Google Gemini, Overlap Filter, Webhook Sink, VLM As Classifier, Seg Preview, OpenAI, Single-Label Classification Model, Line Counter, Gaze Detection
• outputs: Detections Stitch, Roboflow Dataset Upload, Triangle Visualization, Detections Classes Replacement, Ellipse Visualization, Florence-2 Model, Blur Visualization, Halo Visualization, Motion Detection, Model Comparison Visualization, Keypoint Visualization, Pixelate Visualization, Size Measurement, Line Counter, Stitch OCR Detections, Roboflow Dataset Upload, Line Counter Visualization, Label Visualization, Keypoint Detection Model, SIFT Comparison, Dynamic Zone, Distance Measurement, Email Notification, SAM 3, Detections Stabilizer, Slack Notification, Object Detection Model, Path Deviation, Corner Visualization, Florence-2 Model, SAM 3, Object Detection Model, Detections Combine, Google Gemini, Bounding Box Visualization, Keypoint Detection Model, Background Color Visualization, Camera Calibration, Polygon Visualization, Time in Zone, Detections Merge, Heatmap Visualization, Mask Visualization, Byte Tracker, Twilio SMS Notification, PTZ Tracking (ONVIF).md), Instance Segmentation Model, Detections Transformation, Single-Label Classification Model, Stitch OCR Detections, Dynamic Crop, Model Monitoring Inference Aggregator, Circle Visualization, Detection Offset, Byte Tracker, Trace Visualization, Color Visualization, Icon Visualization, Dot Visualization, Time in Zone, Velocity, Email Notification, Instance Segmentation Model, Path Deviation, Camera Focus, Twilio SMS/MMS Notification, Roboflow Custom Metadata, Segment Anything 2 Model, Detections Filter, Text Display, Reference Path Visualization, Perspective Correction, Polygon Zone Visualization, Multi-Label Classification Model, Detection Event Log, Polygon Visualization, Byte Tracker, Halo Visualization, Time in Zone, Stability AI Inpainting, Crop Visualization, Detections Consensus, Template Matching, Detections List Roll-Up, Overlap Filter, Webhook Sink, Classification Label Visualization, Multi-Label Classification Model, Line Counter, Single-Label Classification Model, Gaze Detection

Input and Output Bindings

The available connections depend on its binding kinds. Check what binding kinds PTZ Tracking (ONVIF) in version v1 has.

Bindings

• input

  • predictions (Union[object_detection_prediction, instance_segmentation_prediction]): Object detection or instance segmentation predictions to track. In Follow mode, the block will follow the highest confidence prediction or a tracked object if tracker IDs are present. Predictions should include bounding box coordinates and optionally tracker IDs for persistent tracking..
  • camera_ip (string): Camera IP address or hostname for ONVIF connection. Must be reachable from the workflow execution environment..
  • camera_port (integer): Camera ONVIF service port (typically 80, 8080, or camera-specific port). Must match the camera's ONVIF configuration..
  • camera_username (string): Camera username for ONVIF authentication. Must have PTZ control permissions on the camera..
  • camera_password (secret): Camera password for ONVIF authentication. Should be stored as a secret for security..
  • simulate_variable_speed (boolean): Enable variable speed simulation for cameras without native variable speed support. When enabled, sends 100% speed commands followed by stop commands to approximate percentage speeds. May result in jerky movement - only use if camera lacks variable speed capability..
  • zoom_if_able (boolean): Enable automatic zoom to fill frame with the tracked object. When enabled, camera first centers the object with pan/tilt, then zooms in until the object fills the frame. Requires camera to support zoom functionality..
  • follow_tracker (boolean): Enable persistent tracking using tracker IDs. When enabled, camera locks onto the tracker ID of the highest confidence detection and continues following that specific object until it disappears or tracking resets. Requires a Byte Tracker block in the workflow to assign tracker IDs..
  • dead_zone (integer): Dead zone size in pixels around frame center where camera movement stops. Prevents hunting behavior when object is near center. Larger values reduce pan/tilt hunting but may cause zoom hunting. Smaller values improve zoom stability but may cause pan/tilt oscillations. Typical range: 30-100 pixels..
  • default_position_preset (string): Preset name for default/home position. Camera will return to this preset after idle period (if move_to_position_after_idle_seconds is set) or when using Go To Preset mode. Must match exactly a preset name configured on the camera. Required for preset movement functionality..
  • move_to_position_after_idle_seconds (integer): Auto-reset time in seconds. After camera stops seeking/moving for this duration, automatically moves to default_position_preset. Set to 0 to disable auto-reset. Requires default_position_preset to be configured..
  • camera_update_rate_limit (integer): Minimum time in milliseconds between camera movement commands. Rate limits ONVIF updates to prevent overwhelming the camera. Lower values provide more responsive tracking but may overload slower cameras. Higher values reduce camera load but may cause less smooth movement. Typical range: 100-500ms..
  • flip_x_movement (boolean): Invert horizontal (pan) movement direction. Enable if the camera image is mirrored horizontally and movement appears reversed. Use to correct camera movement when image is flipped..
  • flip_y_movement (boolean): Invert vertical (tilt) movement direction. Enabled by default as many cameras have inverted Y-axis. Disable if vertical movement appears reversed..
  • minimum_camera_speed (float_zero_to_one): Minimum movement speed as percentage (0.0-1.0). Movement commands below this threshold are boosted to this minimum. Some cameras ignore very low speeds - increase if camera doesn't respond to small movements. Typical range: 0.02-0.1 (2%-10%)..
  • pid_kp (float): PID proportional gain (Kp). Controls response to position error - higher values make camera respond faster but may cause overshooting and hunting. Lower values reduce hunting but make tracking slower. Start with default and adjust based on camera responsiveness. Typical range: 0.1-0.5..
  • pid_ki (float): PID integral gain (Ki). Eliminates steady-state error by accumulating error over time. Usually kept at 0 as it can cause oscillations. Increase slightly (0.01-0.1) if camera consistently stops slightly off-center despite small errors..
  • pid_kd (float): PID derivative gain (Kd). Predicts future error and dampens oscillations. Higher values improve stability with video lag but excessive values can cause hunting. Increase (1-5) if there's significant delay between camera movement and video feedback. Decrease if tracking appears jerky..

• output

  • predictions (object_detection_prediction): Prediction with detected bounding boxes in form of sv.Detections(...) object.
  • seeking (boolean): Boolean flag.

Example JSON definition of step PTZ Tracking (ONVIF) in version v1

{
    "name": "<your_step_name_here>",
    "type": "roboflow_core/onvif_sink@v1",
    "predictions": "$steps.object_detection_model.predictions",
    "camera_ip": "<block_does_not_provide_example>",
    "camera_port": "<block_does_not_provide_example>",
    "camera_username": "<block_does_not_provide_example>",
    "camera_password": "<block_does_not_provide_example>",
    "movement_type": "Follow",
    "simulate_variable_speed": true,
    "zoom_if_able": true,
    "follow_tracker": true,
    "dead_zone": 50,
    "default_position_preset": "",
    "move_to_position_after_idle_seconds": "<block_does_not_provide_example>",
    "camera_update_rate_limit": "<block_does_not_provide_example>",
    "flip_x_movement": true,
    "flip_y_movement": true,
    "minimum_camera_speed": "<block_does_not_provide_example>",
    "pid_kp": "<block_does_not_provide_example>",
    "pid_ki": "<block_does_not_provide_example>",
    "pid_kd": "<block_does_not_provide_example>"
}