PTZ Tracking (ONVIF)

Class: ONVIFSinkBlockV1

Source: inference.core.workflows.core_steps.sinks.onvif_movement.v1.ONVIFSinkBlockV1

Control an ONVIF-compatible PTZ (Pan-Tilt-Zoom) camera to automatically follow detected objects, move to preset positions, and maintain smooth tracking using PID control for surveillance, security monitoring, and automated camera control workflows.

How This Block Works

This block controls PTZ cameras through the ONVIF protocol to automatically track and follow objects in real-time. The block:

1. Receives object detection or instance segmentation predictions from upstream blocks
2. Connects to an ONVIF-compatible PTZ camera using IP address, port, username, and password credentials
3. Determines the movement mode (Follow or Go To Preset):

For Follow Mode: - Selects the target object to track: either the highest confidence detection or a specific tracked object (if tracker IDs are present) - Calculates the position error by comparing the object's bounding box center to the frame center - Uses PID (Proportional-Integral-Derivative) control to calculate smooth movement commands: - Proportional (Kp): Responds to current position error - Integral (Ki): Corrects steady-state errors over time - Derivative (Kd): Predicts future error and dampens oscillations - Normalizes movement commands to the camera's velocity limits and applies rate limiting to prevent command flooding - Sends continuous movement commands to the camera via ONVIF ContinuousMove service - Monitors the dead zone (region around center where camera stops moving) to prevent hunting behavior - Optionally zooms into the object when it's centered, adjusting zoom speed to fill the frame - Maintains tracking of a specific object using tracker IDs until it disappears or tracking is reset - Automatically moves to a preset position after a configurable idle period when no objects are detected

For Go To Preset Mode: - Moves the camera to a predefined preset position using the ONVIF GotoPreset service - Requires the camera to support preset functionality and have presets configured - Uses the preset name specified in the configuration

1. Handles camera communication asynchronously using a separate event loop to prevent blocking workflow execution
2. Manages camera state including seeking status, tracked object IDs, zoom state, and movement history
3. Returns two outputs:
4. predictions: The detection being tracked (empty if no object is being followed)
5. seeking: Boolean indicating whether the camera is currently moving/seeking an object

The block uses PID control to calculate smooth, proportional movement commands based on the distance between the object center and frame center. Movement is normalized as a percentage of the camera's maximum speed, and commands are rate-limited to prevent overwhelming the camera with updates. The dead zone prevents small movements when the object is near the center, reducing hunting behavior. When zoom is enabled, the camera first centers the object with pan/tilt, then zooms in to fill the frame while maintaining the object in view.

Common Use Cases

• Surveillance and Security: Automatically track individuals or vehicles in surveillance scenarios (e.g., follow suspicious activity, track intruders, monitor security perimeters), enabling automated surveillance workflows
• Sports and Event Coverage: Track athletes or objects during sports events or performances (e.g., follow players on field, track ball movement, cover event action), enabling automated sports coverage workflows
• Wildlife Monitoring: Follow animals or wildlife in natural habitats (e.g., track bird movements, follow animals in reserves, monitor wildlife behavior), enabling wildlife observation workflows
• Industrial Monitoring: Automatically follow objects or personnel in industrial settings (e.g., track equipment movement, monitor worker activities, follow vehicles in facilities), enabling industrial automation workflows
• Traffic Monitoring: Track vehicles or objects in traffic scenarios (e.g., follow vehicles through intersections, track traffic violations, monitor road activity), enabling automated traffic monitoring workflows
• Retail Analytics: Track customers or products in retail environments (e.g., follow customer paths, track product interactions, monitor shopping behavior), enabling retail analytics workflows

Connecting to Other Blocks

This block receives predictions and produces camera control commands and tracking status:

• After object detection or instance segmentation blocks to track detected objects with the camera (e.g., follow detected people, track detected vehicles, monitor detected objects), enabling detection-to-camera-tracking workflows
• After Byte Tracker blocks to follow specific tracked objects with consistent IDs (e.g., follow tracked person across frames, maintain tracking of specific vehicle, monitor tracked object persistently), enabling tracking-to-camera workflows
• After detection filter blocks to track specific object classes or filtered detections (e.g., track only specific classes, follow filtered detections, monitor selected objects), enabling filtered-tracking workflows
• Before visualization blocks to display camera movement status and tracked objects (e.g., visualize tracking status, display seeking indicator, show camera control feedback), enabling camera control visualization workflows
• Before notification blocks to alert when camera starts or stops tracking (e.g., notify when tracking begins, alert on tracking loss, report camera status), enabling camera status notification workflows
• In surveillance and monitoring pipelines where automated camera control is part of a larger security or monitoring system (e.g., automated security systems, monitoring pipelines, camera control chains), enabling comprehensive surveillance workflows

Requirements

This block requires an ONVIF-compatible PTZ camera with network access. The camera must support the ONVIF ContinuousMove service for Follow mode and GotoPreset service for preset movement. For optimal performance, use a camera with variable speed movement capability - cameras without variable speed can use the simulate_variable_speed option but may experience jerky movement. The block must run in local execution mode (not suitable for remote/cloud execution). PID tuning is recommended to achieve smooth tracking without overshooting or hunting - adjust pid_kp, pid_ki, and pid_kd parameters based on camera responsiveness and video latency. For accurate tracking, use an eager buffer consumption strategy to minimize lag between camera movement and video feedback. The camera must have presets configured if using preset movement or auto-reset functionality.

Type identifier

Use the following identifier in step "type" field: roboflow_core/onvif_sink@v1to add the block as as step in your workflow.

Properties

Name	Type	Description	Refs
name	str	Enter a unique identifier for this step..	❌
camera_ip	str	Camera IP address or hostname for ONVIF connection. Must be reachable from the workflow execution environment..	✅
camera_port	int	Camera ONVIF service port (typically 80, 8080, or camera-specific port). Must match the camera's ONVIF configuration..	✅
camera_username	str	Camera username for ONVIF authentication. Must have PTZ control permissions on the camera..	✅
camera_password	str	Camera password for ONVIF authentication. Should be stored as a secret for security..	✅
movement_type	str	Movement mode for the camera. 'Follow' mode tracks detected objects using PID control. 'Go To Preset' mode moves the camera to a predefined preset position (requires default_position_preset to be configured)..	❌
simulate_variable_speed	bool	Enable variable speed simulation for cameras without native variable speed support. When enabled, sends 100% speed commands followed by stop commands to approximate percentage speeds. May result in jerky movement - only use if camera lacks variable speed capability..	✅
zoom_if_able	bool	Enable automatic zoom to fill frame with the tracked object. When enabled, camera first centers the object with pan/tilt, then zooms in until the object fills the frame. Requires camera to support zoom functionality..	✅
follow_tracker	bool	Enable persistent tracking using tracker IDs. When enabled, camera locks onto the tracker ID of the highest confidence detection and continues following that specific object until it disappears or tracking resets. Requires a Byte Tracker block in the workflow to assign tracker IDs..	✅
dead_zone	int	Dead zone size in pixels around frame center where camera movement stops. Prevents hunting behavior when object is near center. Larger values reduce pan/tilt hunting but may cause zoom hunting. Smaller values improve zoom stability but may cause pan/tilt oscillations. Typical range: 30-100 pixels..	✅
default_position_preset	str	Preset name for default/home position. Camera will return to this preset after idle period (if move_to_position_after_idle_seconds is set) or when using Go To Preset mode. Must match exactly a preset name configured on the camera. Required for preset movement functionality..	✅
move_to_position_after_idle_seconds	int	Auto-reset time in seconds. After camera stops seeking/moving for this duration, automatically moves to default_position_preset. Set to 0 to disable auto-reset. Requires default_position_preset to be configured..	✅
camera_update_rate_limit	int	Minimum time in milliseconds between camera movement commands. Rate limits ONVIF updates to prevent overwhelming the camera. Lower values provide more responsive tracking but may overload slower cameras. Higher values reduce camera load but may cause less smooth movement. Typical range: 100-500ms..	✅
flip_x_movement	bool	Invert horizontal (pan) movement direction. Enable if the camera image is mirrored horizontally and movement appears reversed. Use to correct camera movement when image is flipped..	✅
flip_y_movement	bool	Invert vertical (tilt) movement direction. Enabled by default as many cameras have inverted Y-axis. Disable if vertical movement appears reversed..	✅
minimum_camera_speed	float	Minimum movement speed as percentage (0.0-1.0). Movement commands below this threshold are boosted to this minimum. Some cameras ignore very low speeds - increase if camera doesn't respond to small movements. Typical range: 0.02-0.1 (2%-10%)..	✅
pid_kp	float	PID proportional gain (Kp). Controls response to position error - higher values make camera respond faster but may cause overshooting and hunting. Lower values reduce hunting but make tracking slower. Start with default and adjust based on camera responsiveness. Typical range: 0.1-0.5..	✅
pid_ki	float	PID integral gain (Ki). Eliminates steady-state error by accumulating error over time. Usually kept at 0 as it can cause oscillations. Increase slightly (0.01-0.1) if camera consistently stops slightly off-center despite small errors..	✅
pid_kd	float	PID derivative gain (Kd). Predicts future error and dampens oscillations. Higher values improve stability with video lag but excessive values can cause hunting. Increase (1-5) if there's significant delay between camera movement and video feedback. Decrease if tracking appears jerky..	✅

The Refs column marks possibility to parametrise the property with dynamic values available in workflow runtime. See Bindings for more info.

Available Connections

Compatible Blocks

Check what blocks you can connect to PTZ Tracking (ONVIF) in version v1.

• inputs: Clip Comparison, Anthropic Claude, Email Notification, Camera Focus, CSV Formatter, Object Detection Model, Segment Anything 2 Model, Detections Stabilizer, VLM As Detector, Roboflow Dataset Upload, Dynamic Crop, Multi-Label Classification Model, SAM 3, Camera Focus, Stitch OCR Detections, VLM As Classifier, Path Deviation, YOLO-World Model, Detections Combine, Bounding Rectangle, PTZ Tracking (ONVIF).md), Moondream2, Local File Sink, Florence-2 Model, Stitch OCR Detections, Line Counter, SIFT Comparison, Roboflow Custom Metadata, Byte Tracker, Identify Changes, LMM, SAM 3, Detections List Roll-Up, OpenAI, Twilio SMS Notification, Identify Outliers, Byte Tracker, SIFT Comparison, Velocity, Template Matching, Overlap Filter, Single-Label Classification Model, Dynamic Zone, Google Gemini, Google Vision OCR, Llama 3.2 Vision, Gaze Detection, OpenAI, Image Contours, VLM As Classifier, LMM For Classification, Email Notification, Cosine Similarity, VLM As Detector, Anthropic Claude, Distance Measurement, OpenAI, Detections Classes Replacement, EasyOCR, OCR Model, Instance Segmentation Model, Detections Consensus, Slack Notification, Time in Zone, Line Counter, Detection Event Log, Google Gemini, Florence-2 Model, JSON Parser, Mask Area Measurement, Pixel Color Count, Seg Preview, Detections Filter, Google Gemini, Time in Zone, Detections Stitch, OpenAI, Time in Zone, Detections Merge, Object Detection Model, CogVLM, Webhook Sink, Instance Segmentation Model, Byte Tracker, Anthropic Claude, Path Deviation, Perspective Correction, SAM 3, Detections Transformation, Motion Detection, Keypoint Detection Model, Detection Offset, Model Monitoring Inference Aggregator, Twilio SMS/MMS Notification, Roboflow Dataset Upload
• outputs: Halo Visualization, Email Notification, Text Display, Object Detection Model, Segment Anything 2 Model, Blur Visualization, Detections Stabilizer, Roboflow Dataset Upload, Corner Visualization, Dynamic Crop, Classification Label Visualization, Trace Visualization, Keypoint Detection Model, Multi-Label Classification Model, Camera Focus, Mask Visualization, Stitch OCR Detections, Path Deviation, Detections Combine, Background Color Visualization, PTZ Tracking (ONVIF).md), Polygon Visualization, Florence-2 Model, Stitch OCR Detections, Size Measurement, Line Counter, Ellipse Visualization, Pixelate Visualization, SIFT Comparison, Roboflow Custom Metadata, Byte Tracker, Single-Label Classification Model, SAM 3, Detections List Roll-Up, Dot Visualization, Circle Visualization, Twilio SMS Notification, Byte Tracker, Velocity, Template Matching, Overlap Filter, Crop Visualization, Single-Label Classification Model, Google Gemini, Dynamic Zone, Keypoint Visualization, Polygon Visualization, Icon Visualization, Multi-Label Classification Model, Gaze Detection, Color Visualization, Email Notification, Distance Measurement, Triangle Visualization, Detections Classes Replacement, Bounding Box Visualization, Model Comparison Visualization, Reference Path Visualization, Instance Segmentation Model, Line Counter Visualization, Detections Consensus, Halo Visualization, Slack Notification, Time in Zone, Stability AI Inpainting, Line Counter, Detection Event Log, Florence-2 Model, Mask Area Measurement, Heatmap Visualization, Polygon Zone Visualization, Detections Filter, Label Visualization, Time in Zone, Detections Stitch, Time in Zone, Detections Merge, Object Detection Model, Webhook Sink, Instance Segmentation Model, Byte Tracker, Path Deviation, Perspective Correction, Camera Calibration, SAM 3, Detections Transformation, Motion Detection, Keypoint Detection Model, Model Monitoring Inference Aggregator, Detection Offset, Twilio SMS/MMS Notification, Roboflow Dataset Upload

Input and Output Bindings

The available connections depend on its binding kinds. Check what binding kinds PTZ Tracking (ONVIF) in version v1 has.

Bindings

• input

  • predictions (Union[object_detection_prediction, instance_segmentation_prediction]): Object detection or instance segmentation predictions to track. In Follow mode, the block will follow the highest confidence prediction or a tracked object if tracker IDs are present. Predictions should include bounding box coordinates and optionally tracker IDs for persistent tracking..
  • camera_ip (string): Camera IP address or hostname for ONVIF connection. Must be reachable from the workflow execution environment..
  • camera_port (integer): Camera ONVIF service port (typically 80, 8080, or camera-specific port). Must match the camera's ONVIF configuration..
  • camera_username (string): Camera username for ONVIF authentication. Must have PTZ control permissions on the camera..
  • camera_password (secret): Camera password for ONVIF authentication. Should be stored as a secret for security..
  • simulate_variable_speed (boolean): Enable variable speed simulation for cameras without native variable speed support. When enabled, sends 100% speed commands followed by stop commands to approximate percentage speeds. May result in jerky movement - only use if camera lacks variable speed capability..
  • zoom_if_able (boolean): Enable automatic zoom to fill frame with the tracked object. When enabled, camera first centers the object with pan/tilt, then zooms in until the object fills the frame. Requires camera to support zoom functionality..
  • follow_tracker (boolean): Enable persistent tracking using tracker IDs. When enabled, camera locks onto the tracker ID of the highest confidence detection and continues following that specific object until it disappears or tracking resets. Requires a Byte Tracker block in the workflow to assign tracker IDs..
  • dead_zone (integer): Dead zone size in pixels around frame center where camera movement stops. Prevents hunting behavior when object is near center. Larger values reduce pan/tilt hunting but may cause zoom hunting. Smaller values improve zoom stability but may cause pan/tilt oscillations. Typical range: 30-100 pixels..
  • default_position_preset (string): Preset name for default/home position. Camera will return to this preset after idle period (if move_to_position_after_idle_seconds is set) or when using Go To Preset mode. Must match exactly a preset name configured on the camera. Required for preset movement functionality..
  • move_to_position_after_idle_seconds (integer): Auto-reset time in seconds. After camera stops seeking/moving for this duration, automatically moves to default_position_preset. Set to 0 to disable auto-reset. Requires default_position_preset to be configured..
  • camera_update_rate_limit (integer): Minimum time in milliseconds between camera movement commands. Rate limits ONVIF updates to prevent overwhelming the camera. Lower values provide more responsive tracking but may overload slower cameras. Higher values reduce camera load but may cause less smooth movement. Typical range: 100-500ms..
  • flip_x_movement (boolean): Invert horizontal (pan) movement direction. Enable if the camera image is mirrored horizontally and movement appears reversed. Use to correct camera movement when image is flipped..
  • flip_y_movement (boolean): Invert vertical (tilt) movement direction. Enabled by default as many cameras have inverted Y-axis. Disable if vertical movement appears reversed..
  • minimum_camera_speed (float_zero_to_one): Minimum movement speed as percentage (0.0-1.0). Movement commands below this threshold are boosted to this minimum. Some cameras ignore very low speeds - increase if camera doesn't respond to small movements. Typical range: 0.02-0.1 (2%-10%)..
  • pid_kp (float): PID proportional gain (Kp). Controls response to position error - higher values make camera respond faster but may cause overshooting and hunting. Lower values reduce hunting but make tracking slower. Start with default and adjust based on camera responsiveness. Typical range: 0.1-0.5..
  • pid_ki (float): PID integral gain (Ki). Eliminates steady-state error by accumulating error over time. Usually kept at 0 as it can cause oscillations. Increase slightly (0.01-0.1) if camera consistently stops slightly off-center despite small errors..
  • pid_kd (float): PID derivative gain (Kd). Predicts future error and dampens oscillations. Higher values improve stability with video lag but excessive values can cause hunting. Increase (1-5) if there's significant delay between camera movement and video feedback. Decrease if tracking appears jerky..

• output

  • predictions (object_detection_prediction): Prediction with detected bounding boxes in form of sv.Detections(...) object.
  • seeking (boolean): Boolean flag.

Example JSON definition of step PTZ Tracking (ONVIF) in version v1

{
    "name": "<your_step_name_here>",
    "type": "roboflow_core/onvif_sink@v1",
    "predictions": "$steps.object_detection_model.predictions",
    "camera_ip": "<block_does_not_provide_example>",
    "camera_port": "<block_does_not_provide_example>",
    "camera_username": "<block_does_not_provide_example>",
    "camera_password": "<block_does_not_provide_example>",
    "movement_type": "Follow",
    "simulate_variable_speed": true,
    "zoom_if_able": true,
    "follow_tracker": true,
    "dead_zone": 50,
    "default_position_preset": "",
    "move_to_position_after_idle_seconds": "<block_does_not_provide_example>",
    "camera_update_rate_limit": "<block_does_not_provide_example>",
    "flip_x_movement": true,
    "flip_y_movement": true,
    "minimum_camera_speed": "<block_does_not_provide_example>",
    "pid_kp": "<block_does_not_provide_example>",
    "pid_ki": "<block_does_not_provide_example>",
    "pid_kd": "<block_does_not_provide_example>"
}