SAM2 Video Tracker¶
Class: SegmentAnything2VideoBlockV1
Run Segment Anything 2 on a live video stream frame by frame, keeping per-video temporal memory so object identities are preserved across frames.
Feed box detections from an upstream detector (e.g. a YOLO block) as
prompts. The block multiplexes a single SAM2 camera predictor across
many video streams by keying state on video_metadata.video_identifier;
depending on prompt_mode, it either re-seeds the prompts periodically
or simply propagates existing tracks.
Intended for use with InferencePipeline, which delivers one frame at
a time and tags each frame with video metadata.
Type identifier¶
Use the following identifier in step "type" field: roboflow_core/segment_anything_2_video@v1to add the block as
as step in your workflow.
Properties¶
| Name | Type | Description | Refs |
|---|---|---|---|
name |
str |
Enter a unique identifier for this step.. | ❌ |
model_id |
str |
Streaming video tracker model id resolved by inference_models. The sam2video family ships four Hiera backbone sizes; small is the default trade-off between speed and quality. sam3trackervideo is SAM3's visually prompted tracker — the same prompt contract with a larger backbone, markedly better at identity retention on long videos and crowded scenes, at higher compute cost.. |
✅ |
prompt_mode |
str |
When to consume boxes as SAM2 prompts. first_frame prompts once per session and then tracks; every_n_frames re-seeds every prompt_interval frames; every_frame re-seeds every frame. On frames where re-seeding does not happen, boxes is ignored and the block simply propagates.. |
❌ |
prompt_interval |
int |
For prompt_mode=every_n_frames: re-prompt every N frames.. |
✅ |
threshold |
float |
Minimum confidence for emitted masks.. | ✅ |
The Refs column marks possibility to parametrise the property with dynamic values available
in workflow runtime. See Bindings for more info.
Runtime compatibility¶
-
soft— runtimehosted_serverless,dedicated_deployment; executionremote; inputvideo - Block keeps per-video state in process memory (keyed by video_metadata.video_identifier). With remote step execution on stateless or multi-replica HTTP runtimes, successive requests may be served by different worker processes, so the state resets between calls and the output is meaningless for tracking / counting / aggregation. Use local step execution in an InferencePipeline for stable cross-frame results.
-
hard— runtimeself_hosted_cpu; executionlocal - Requires a GPU; the streaming SAM2 video model needs CUDA.
-
soft— inputimage - Block depends on temporal context from video or repeated-frame workflows. With a still image/photo, there is no meaningful history to track, compare, aggregate, or visualize, so the block provides little or no benefit.
Available Connections¶
Compatible Blocks
Check what blocks you can connect to SAM2 Video Tracker in version v1.
- inputs:
Keypoint Visualization,Keypoint Detection Model,PP-OCR,Blur Visualization,EasyOCR,SIFT Comparison,Grid Visualization,Segment Anything 2 Model,SAM 3,Object Detection Model,Color Visualization,Per-Class Confidence Filter,Dynamic Zone,Image Convert Grayscale,Motion Detection,Absolute Static Crop,YOLO-World Model,Distance Measurement,Roboflow Visual Search Classifier,Velocity,Image Threshold,Icon Visualization,Polygon Zone Visualization,Trace Visualization,Time in Zone,Model Comparison Visualization,Camera Focus,QR Code Generator,BoT-SORT Tracker,Detections Stitch,Image Stack,Contrast Equalization,Detections List Roll-Up,Instance Segmentation Model,OCR Model,Background Subtraction,Mask Visualization,Reference Path Visualization,Contrast Enhancement,Mask Edge Snap,Instance Segmentation Model,Detections Filter,Detections Classes Replacement,Detections Merge,PTZ Tracking (ONVIF),Text Display,VLM As Detector,Gaze Detection,Dynamic Crop,Halo Visualization,Image Blur,Stability AI Image Generation,SAM 3 Interactive,Google Vision OCR,Seg Preview,SAM 3,VLM As Detector,Ellipse Visualization,Cosine Similarity,Pixelate Visualization,Stitch Images,Byte Tracker,Detection Offset,Line Counter,Classification Label Visualization,Time in Zone,Overlap Filter,Polygon Visualization,Instance Segmentation Model,Path Deviation,Instance Segmentation Model,Semantic Segmentation Model,Time in Zone,Single-Label Classification Model,Keypoint Detection Model,Morphological Transformation,Detection Event Log,Byte Tracker,Bounding Box Visualization,Template Matching,Image Contours,Image Slicer,Heatmap Visualization,Stability AI Outpainting,SORT Tracker,Semantic Segmentation Model,Circle Visualization,Multi-Label Classification Model,Perspective Correction,SAM2 Video Tracker,Camera Focus,Byte Tracker,Camera Calibration,Crop Visualization,Morphological Transformation,Polygon Visualization,Detections Consensus,Keypoint Detection Model,Detections Stabilizer,Line Counter,Path Deviation,ByteTrack Tracker,Detections Combine,Relative Static Crop,Mask Area Measurement,Background Color Visualization,Stability AI Inpainting,Corner Visualization,SAM 3,Object Detection Model,SIFT,Track Class Lock,Image Slicer,Moondream2,Roboflow Visual Search,Detections Transformation,Multi-Label Classification Model,Bounding Rectangle,Identify Changes,Object Detection Model,Single-Label Classification Model,SAM3 Video Tracker,Triangle Visualization,Pixel Color Count,OC-SORT Tracker,Dot Visualization,Image Preprocessing,SIFT Comparison,Line Counter Visualization,Halo Visualization,Depth Estimation,Label Visualization - outputs:
Blur Visualization,Segment Anything 2 Model,Color Visualization,Per-Class Confidence Filter,Dynamic Zone,Distance Measurement,Velocity,Trace Visualization,Icon Visualization,Time in Zone,Model Comparison Visualization,BoT-SORT Tracker,Detections Stitch,Detections List Roll-Up,Mask Visualization,Mask Edge Snap,Detections Filter,Detections Merge,Detections Classes Replacement,PTZ Tracking (ONVIF),Florence-2 Model,Dynamic Crop,SAM 3 Interactive,Halo Visualization,Ellipse Visualization,Pixelate Visualization,Byte Tracker,Florence-2 Model,Detection Offset,Line Counter,Time in Zone,Overlap Filter,Polygon Visualization,Path Deviation,Roboflow Dataset Upload,Event Writer,Time in Zone,Byte Tracker,Detection Event Log,Bounding Box Visualization,Heatmap Visualization,SORT Tracker,Circle Visualization,Roboflow Custom Metadata,SAM2 Video Tracker,Perspective Correction,Camera Focus,Byte Tracker,Crop Visualization,Polygon Visualization,Detections Consensus,Detections Stabilizer,Line Counter,ByteTrack Tracker,Path Deviation,Detections Combine,Mask Area Measurement,Background Color Visualization,Model Monitoring Inference Aggregator,Stability AI Inpainting,GeoTag Detection,Corner Visualization,Track Class Lock,Roboflow Vision Events,Detections Transformation,Bounding Rectangle,Size Measurement,Overlap Analysis,Triangle Visualization,Roboflow Dataset Upload,OC-SORT Tracker,Dot Visualization,Halo Visualization,Label Visualization
Input and Output Bindings¶
The available connections depend on its binding kinds. Check what binding kinds
SAM2 Video Tracker in version v1 has.
Bindings
-
input
images(image): The image to infer on..boxes(Union[object_detection_prediction,keypoint_detection_prediction,instance_segmentation_prediction]): Bounding boxes to use as SAM2 prompts. Only read on frames where the block re-prompts (seeprompt_mode)..model_id(roboflow_model_id): Streaming video tracker model id resolved byinference_models. Thesam2videofamily ships four Hiera backbone sizes;smallis the default trade-off between speed and quality.sam3trackervideois SAM3's visually prompted tracker — the same prompt contract with a larger backbone, markedly better at identity retention on long videos and crowded scenes, at higher compute cost..prompt_interval(integer): Forprompt_mode=every_n_frames: re-prompt every N frames..threshold(float): Minimum confidence for emitted masks..
-
output
predictions(instance_segmentation_prediction): Prediction with detected bounding boxes and segmentation masks in form of sv.Detections(...) object.
Example JSON definition of step SAM2 Video Tracker in version v1
{
"name": "<your_step_name_here>",
"type": "roboflow_core/segment_anything_2_video@v1",
"images": "$inputs.image",
"boxes": "$steps.object_detection_model.predictions",
"model_id": "sam2video/tiny",
"prompt_mode": "<block_does_not_provide_example>",
"prompt_interval": 30,
"threshold": 0.0
}