PTZ Tracking (ONVIF)

Class: ONVIFSinkBlockV1

Source: inference.core.workflows.core_steps.sinks.onvif_movement.v1.ONVIFSinkBlockV1

Control an ONVIF-compatible PTZ (Pan-Tilt-Zoom) camera to automatically follow detected objects, move to preset positions, and maintain smooth tracking using PID control for surveillance, security monitoring, and automated camera control workflows.

How This Block Works

This block controls PTZ cameras through the ONVIF protocol to automatically track and follow objects in real-time. The block:

1. Receives object detection or instance segmentation predictions from upstream blocks
2. Connects to an ONVIF-compatible PTZ camera using IP address, port, username, and password credentials
3. Determines the movement mode (Follow or Go To Preset):

For Follow Mode: - Selects the target object to track: either the highest confidence detection or a specific tracked object (if tracker IDs are present) - Calculates the position error by comparing the object's bounding box center to the frame center - Uses PID (Proportional-Integral-Derivative) control to calculate smooth movement commands: - Proportional (Kp): Responds to current position error - Integral (Ki): Corrects steady-state errors over time - Derivative (Kd): Predicts future error and dampens oscillations - Normalizes movement commands to the camera's velocity limits and applies rate limiting to prevent command flooding - Sends continuous movement commands to the camera via ONVIF ContinuousMove service - Monitors the dead zone (region around center where camera stops moving) to prevent hunting behavior - Optionally zooms into the object when it's centered, adjusting zoom speed to fill the frame - Maintains tracking of a specific object using tracker IDs until it disappears or tracking is reset - Automatically moves to a preset position after a configurable idle period when no objects are detected

For Go To Preset Mode: - Moves the camera to a predefined preset position using the ONVIF GotoPreset service - Requires the camera to support preset functionality and have presets configured - Uses the preset name specified in the configuration

1. Handles camera communication asynchronously using a separate event loop to prevent blocking workflow execution
2. Manages camera state including seeking status, tracked object IDs, zoom state, and movement history
3. Returns two outputs:
4. predictions: The detection being tracked (empty if no object is being followed)
5. seeking: Boolean indicating whether the camera is currently moving/seeking an object

The block uses PID control to calculate smooth, proportional movement commands based on the distance between the object center and frame center. Movement is normalized as a percentage of the camera's maximum speed, and commands are rate-limited to prevent overwhelming the camera with updates. The dead zone prevents small movements when the object is near the center, reducing hunting behavior. When zoom is enabled, the camera first centers the object with pan/tilt, then zooms in to fill the frame while maintaining the object in view.

Common Use Cases

• Surveillance and Security: Automatically track individuals or vehicles in surveillance scenarios (e.g., follow suspicious activity, track intruders, monitor security perimeters), enabling automated surveillance workflows
• Sports and Event Coverage: Track athletes or objects during sports events or performances (e.g., follow players on field, track ball movement, cover event action), enabling automated sports coverage workflows
• Wildlife Monitoring: Follow animals or wildlife in natural habitats (e.g., track bird movements, follow animals in reserves, monitor wildlife behavior), enabling wildlife observation workflows
• Industrial Monitoring: Automatically follow objects or personnel in industrial settings (e.g., track equipment movement, monitor worker activities, follow vehicles in facilities), enabling industrial automation workflows
• Traffic Monitoring: Track vehicles or objects in traffic scenarios (e.g., follow vehicles through intersections, track traffic violations, monitor road activity), enabling automated traffic monitoring workflows
• Retail Analytics: Track customers or products in retail environments (e.g., follow customer paths, track product interactions, monitor shopping behavior), enabling retail analytics workflows

Connecting to Other Blocks

This block receives predictions and produces camera control commands and tracking status:

• After object detection or instance segmentation blocks to track detected objects with the camera (e.g., follow detected people, track detected vehicles, monitor detected objects), enabling detection-to-camera-tracking workflows
• After Byte Tracker blocks to follow specific tracked objects with consistent IDs (e.g., follow tracked person across frames, maintain tracking of specific vehicle, monitor tracked object persistently), enabling tracking-to-camera workflows
• After detection filter blocks to track specific object classes or filtered detections (e.g., track only specific classes, follow filtered detections, monitor selected objects), enabling filtered-tracking workflows
• Before visualization blocks to display camera movement status and tracked objects (e.g., visualize tracking status, display seeking indicator, show camera control feedback), enabling camera control visualization workflows
• Before notification blocks to alert when camera starts or stops tracking (e.g., notify when tracking begins, alert on tracking loss, report camera status), enabling camera status notification workflows
• In surveillance and monitoring pipelines where automated camera control is part of a larger security or monitoring system (e.g., automated security systems, monitoring pipelines, camera control chains), enabling comprehensive surveillance workflows

Requirements

This block requires an ONVIF-compatible PTZ camera with network access. The camera must support the ONVIF ContinuousMove service for Follow mode and GotoPreset service for preset movement. For optimal performance, use a camera with variable speed movement capability - cameras without variable speed can use the simulate_variable_speed option but may experience jerky movement. The block must run in local execution mode (not suitable for remote/cloud execution). PID tuning is recommended to achieve smooth tracking without overshooting or hunting - adjust pid_kp, pid_ki, and pid_kd parameters based on camera responsiveness and video latency. For accurate tracking, use an eager buffer consumption strategy to minimize lag between camera movement and video feedback. The camera must have presets configured if using preset movement or auto-reset functionality.

Type identifier

Use the following identifier in step "type" field: roboflow_core/onvif_sink@v1to add the block as as step in your workflow.

Properties

Name	Type	Description	Refs
name	str	Enter a unique identifier for this step..	❌
camera_ip	str	Camera IP address or hostname for ONVIF connection. Must be reachable from the workflow execution environment..	✅
camera_port	int	Camera ONVIF service port (typically 80, 8080, or camera-specific port). Must match the camera's ONVIF configuration..	✅
camera_username	str	Camera username for ONVIF authentication. Must have PTZ control permissions on the camera..	✅
camera_password	str	Camera password for ONVIF authentication. Should be stored as a secret for security..	✅
movement_type	str	Movement mode for the camera. 'Follow' mode tracks detected objects using PID control. 'Go To Preset' mode moves the camera to a predefined preset position (requires default_position_preset to be configured)..	❌
simulate_variable_speed	bool	Enable variable speed simulation for cameras without native variable speed support. When enabled, sends 100% speed commands followed by stop commands to approximate percentage speeds. May result in jerky movement - only use if camera lacks variable speed capability..	✅
zoom_if_able	bool	Enable automatic zoom to fill frame with the tracked object. When enabled, camera first centers the object with pan/tilt, then zooms in until the object fills the frame. Requires camera to support zoom functionality..	✅
follow_tracker	bool	Enable persistent tracking using tracker IDs. When enabled, camera locks onto the tracker ID of the highest confidence detection and continues following that specific object until it disappears or tracking resets. Requires a Byte Tracker block in the workflow to assign tracker IDs..	✅
dead_zone	int	Dead zone size in pixels around frame center where camera movement stops. Prevents hunting behavior when object is near center. Larger values reduce pan/tilt hunting but may cause zoom hunting. Smaller values improve zoom stability but may cause pan/tilt oscillations. Typical range: 30-100 pixels..	✅
default_position_preset	str	Preset name for default/home position. Camera will return to this preset after idle period (if move_to_position_after_idle_seconds is set) or when using Go To Preset mode. Must match exactly a preset name configured on the camera. Required for preset movement functionality..	✅
move_to_position_after_idle_seconds	int	Auto-reset time in seconds. After camera stops seeking/moving for this duration, automatically moves to default_position_preset. Set to 0 to disable auto-reset. Requires default_position_preset to be configured..	✅
camera_update_rate_limit	int	Minimum time in milliseconds between camera movement commands. Rate limits ONVIF updates to prevent overwhelming the camera. Lower values provide more responsive tracking but may overload slower cameras. Higher values reduce camera load but may cause less smooth movement. Typical range: 100-500ms..	✅
flip_x_movement	bool	Invert horizontal (pan) movement direction. Enable if the camera image is mirrored horizontally and movement appears reversed. Use to correct camera movement when image is flipped..	✅
flip_y_movement	bool	Invert vertical (tilt) movement direction. Enabled by default as many cameras have inverted Y-axis. Disable if vertical movement appears reversed..	✅
minimum_camera_speed	float	Minimum movement speed as percentage (0.0-1.0). Movement commands below this threshold are boosted to this minimum. Some cameras ignore very low speeds - increase if camera doesn't respond to small movements. Typical range: 0.02-0.1 (2%-10%)..	✅
pid_kp	float	PID proportional gain (Kp). Controls response to position error - higher values make camera respond faster but may cause overshooting and hunting. Lower values reduce hunting but make tracking slower. Start with default and adjust based on camera responsiveness. Typical range: 0.1-0.5..	✅
pid_ki	float	PID integral gain (Ki). Eliminates steady-state error by accumulating error over time. Usually kept at 0 as it can cause oscillations. Increase slightly (0.01-0.1) if camera consistently stops slightly off-center despite small errors..	✅
pid_kd	float	PID derivative gain (Kd). Predicts future error and dampens oscillations. Higher values improve stability with video lag but excessive values can cause hunting. Increase (1-5) if there's significant delay between camera movement and video feedback. Decrease if tracking appears jerky..	✅

The Refs column marks possibility to parametrise the property with dynamic values available in workflow runtime. See Bindings for more info.

Available Connections

Compatible Blocks

Check what blocks you can connect to PTZ Tracking (ONVIF) in version v1.

• inputs: Llama 3.2 Vision, Clip Comparison, SIFT Comparison, Anthropic Claude, Detections Transformation, Time in Zone, VLM as Detector, Local File Sink, SIFT Comparison, Email Notification, Roboflow Dataset Upload, Motion Detection, Camera Focus, PTZ Tracking (ONVIF).md), Moondream2, LMM, Byte Tracker, Google Vision OCR, SAM 3, Anthropic Claude, Cosine Similarity, Object Detection Model, Detections Merge, Seg Preview, EasyOCR, Time in Zone, VLM as Classifier, Detection Offset, Time in Zone, Detections Filter, Instance Segmentation Model, Detections Combine, Perspective Correction, Path Deviation, Overlap Filter, Keypoint Detection Model, CSV Formatter, Florence-2 Model, Detections Stabilizer, Twilio SMS Notification, Line Counter, Dynamic Zone, Detections List Roll-Up, Identify Changes, SAM 3, Segment Anything 2 Model, Detections Consensus, Image Contours, Multi-Label Classification Model, Pixel Color Count, Detections Stitch, Dynamic Crop, VLM as Classifier, Model Monitoring Inference Aggregator, YOLO-World Model, Detection Event Log, Instance Segmentation Model, Detections Classes Replacement, Single-Label Classification Model, Line Counter, Email Notification, OCR Model, Distance Measurement, Path Deviation, Roboflow Custom Metadata, Google Gemini, OpenAI, OpenAI, CogVLM, Camera Focus, Byte Tracker, Identify Outliers, Slack Notification, Florence-2 Model, Google Gemini, JSON Parser, Google Gemini, Object Detection Model, LMM For Classification, Template Matching, OpenAI, Stitch OCR Detections, Gaze Detection, Bounding Rectangle, OpenAI, Stitch OCR Detections, SAM 3, Byte Tracker, Roboflow Dataset Upload, Velocity, Twilio SMS/MMS Notification, Anthropic Claude, VLM as Detector, Webhook Sink
• outputs: Detections Transformation, Time in Zone, Polygon Visualization, SIFT Comparison, Email Notification, Roboflow Dataset Upload, Motion Detection, Text Display, Model Comparison Visualization, Camera Focus, PTZ Tracking (ONVIF).md), Byte Tracker, Single-Label Classification Model, Mask Visualization, Object Detection Model, Detections Merge, Keypoint Detection Model, Circle Visualization, Pixelate Visualization, Stability AI Inpainting, Multi-Label Classification Model, Time in Zone, Reference Path Visualization, Detection Offset, Time in Zone, Detections Filter, Instance Segmentation Model, Detections Combine, Perspective Correction, Crop Visualization, Ellipse Visualization, Halo Visualization, Path Deviation, Overlap Filter, Keypoint Detection Model, Florence-2 Model, Detections Stabilizer, Twilio SMS Notification, Corner Visualization, Line Counter, Dynamic Zone, Detections List Roll-Up, Icon Visualization, SAM 3, Segment Anything 2 Model, Detections Consensus, Multi-Label Classification Model, Detections Stitch, Dynamic Crop, Bounding Box Visualization, Model Monitoring Inference Aggregator, Detection Event Log, Instance Segmentation Model, Detections Classes Replacement, Line Counter Visualization, Blur Visualization, Single-Label Classification Model, Polygon Zone Visualization, Line Counter, Email Notification, Keypoint Visualization, Distance Measurement, Path Deviation, Roboflow Custom Metadata, Trace Visualization, Color Visualization, Size Measurement, Byte Tracker, Dot Visualization, Label Visualization, Slack Notification, Florence-2 Model, Object Detection Model, Template Matching, Stitch OCR Detections, Gaze Detection, Stitch OCR Detections, Background Color Visualization, Roboflow Dataset Upload, Byte Tracker, Classification Label Visualization, Velocity, SAM 3, Twilio SMS/MMS Notification, Triangle Visualization, Webhook Sink

Input and Output Bindings

The available connections depend on its binding kinds. Check what binding kinds PTZ Tracking (ONVIF) in version v1 has.

Bindings

• input

  • predictions (Union[instance_segmentation_prediction, object_detection_prediction]): Object detection or instance segmentation predictions to track. In Follow mode, the block will follow the highest confidence prediction or a tracked object if tracker IDs are present. Predictions should include bounding box coordinates and optionally tracker IDs for persistent tracking..
  • camera_ip (string): Camera IP address or hostname for ONVIF connection. Must be reachable from the workflow execution environment..
  • camera_port (integer): Camera ONVIF service port (typically 80, 8080, or camera-specific port). Must match the camera's ONVIF configuration..
  • camera_username (string): Camera username for ONVIF authentication. Must have PTZ control permissions on the camera..
  • camera_password (secret): Camera password for ONVIF authentication. Should be stored as a secret for security..
  • simulate_variable_speed (boolean): Enable variable speed simulation for cameras without native variable speed support. When enabled, sends 100% speed commands followed by stop commands to approximate percentage speeds. May result in jerky movement - only use if camera lacks variable speed capability..
  • zoom_if_able (boolean): Enable automatic zoom to fill frame with the tracked object. When enabled, camera first centers the object with pan/tilt, then zooms in until the object fills the frame. Requires camera to support zoom functionality..
  • follow_tracker (boolean): Enable persistent tracking using tracker IDs. When enabled, camera locks onto the tracker ID of the highest confidence detection and continues following that specific object until it disappears or tracking resets. Requires a Byte Tracker block in the workflow to assign tracker IDs..
  • dead_zone (integer): Dead zone size in pixels around frame center where camera movement stops. Prevents hunting behavior when object is near center. Larger values reduce pan/tilt hunting but may cause zoom hunting. Smaller values improve zoom stability but may cause pan/tilt oscillations. Typical range: 30-100 pixels..
  • default_position_preset (string): Preset name for default/home position. Camera will return to this preset after idle period (if move_to_position_after_idle_seconds is set) or when using Go To Preset mode. Must match exactly a preset name configured on the camera. Required for preset movement functionality..
  • move_to_position_after_idle_seconds (integer): Auto-reset time in seconds. After camera stops seeking/moving for this duration, automatically moves to default_position_preset. Set to 0 to disable auto-reset. Requires default_position_preset to be configured..
  • camera_update_rate_limit (integer): Minimum time in milliseconds between camera movement commands. Rate limits ONVIF updates to prevent overwhelming the camera. Lower values provide more responsive tracking but may overload slower cameras. Higher values reduce camera load but may cause less smooth movement. Typical range: 100-500ms..
  • flip_x_movement (boolean): Invert horizontal (pan) movement direction. Enable if the camera image is mirrored horizontally and movement appears reversed. Use to correct camera movement when image is flipped..
  • flip_y_movement (boolean): Invert vertical (tilt) movement direction. Enabled by default as many cameras have inverted Y-axis. Disable if vertical movement appears reversed..
  • minimum_camera_speed (float_zero_to_one): Minimum movement speed as percentage (0.0-1.0). Movement commands below this threshold are boosted to this minimum. Some cameras ignore very low speeds - increase if camera doesn't respond to small movements. Typical range: 0.02-0.1 (2%-10%)..
  • pid_kp (float): PID proportional gain (Kp). Controls response to position error - higher values make camera respond faster but may cause overshooting and hunting. Lower values reduce hunting but make tracking slower. Start with default and adjust based on camera responsiveness. Typical range: 0.1-0.5..
  • pid_ki (float): PID integral gain (Ki). Eliminates steady-state error by accumulating error over time. Usually kept at 0 as it can cause oscillations. Increase slightly (0.01-0.1) if camera consistently stops slightly off-center despite small errors..
  • pid_kd (float): PID derivative gain (Kd). Predicts future error and dampens oscillations. Higher values improve stability with video lag but excessive values can cause hunting. Increase (1-5) if there's significant delay between camera movement and video feedback. Decrease if tracking appears jerky..

• output

  • predictions (object_detection_prediction): Prediction with detected bounding boxes in form of sv.Detections(...) object.
  • seeking (boolean): Boolean flag.

Example JSON definition of step PTZ Tracking (ONVIF) in version v1

{
    "name": "<your_step_name_here>",
    "type": "roboflow_core/onvif_sink@v1",
    "predictions": "$steps.object_detection_model.predictions",
    "camera_ip": "<block_does_not_provide_example>",
    "camera_port": "<block_does_not_provide_example>",
    "camera_username": "<block_does_not_provide_example>",
    "camera_password": "<block_does_not_provide_example>",
    "movement_type": "Follow",
    "simulate_variable_speed": true,
    "zoom_if_able": true,
    "follow_tracker": true,
    "dead_zone": 50,
    "default_position_preset": "",
    "move_to_position_after_idle_seconds": "<block_does_not_provide_example>",
    "camera_update_rate_limit": "<block_does_not_provide_example>",
    "flip_x_movement": true,
    "flip_y_movement": true,
    "minimum_camera_speed": "<block_does_not_provide_example>",
    "pid_kp": "<block_does_not_provide_example>",
    "pid_ki": "<block_does_not_provide_example>",
    "pid_kd": "<block_does_not_provide_example>"
}