SAM3 Video Tracker¶
Class: SegmentAnything3VideoBlockV1
Run Segment Anything 3 on a live video stream frame by frame, keeping per-video temporal memory so object identities are preserved across frames.
Provide the concepts to track as text in class_names (e.g.
["person", "forklift"]) — no upstream detector is needed. SAM3 runs
fused detection and tracking on every frame, so objects matching a
concept that enter the scene mid-stream are picked up automatically and
assigned fresh tracker_ids. Each emitted mask carries the prompt it
matched as its class name and the model's detection score as its
confidence.
The block multiplexes a single SAM3 streaming model across many video
streams by keying state on video_metadata.video_identifier; a session
is re-seeded only when the source stream restarts or class_names
changes. For detector-driven (box-prompted) video tracking, use the
SAM2 Video Tracker block instead.
Intended for use with InferencePipeline, which delivers one frame at
a time and tags each frame with video metadata.
Type identifier¶
Use the following identifier in step "type" field: roboflow_core/sam3_video@v1to add the block as
as step in your workflow.
Properties¶
| Name | Type | Description | Refs |
|---|---|---|---|
name |
str |
Enter a unique identifier for this step.. | ❌ |
class_names |
Union[List[str], str] |
Concepts to segment and track, as a list of phrases (or a single comma-separated string). Each emitted mask carries the concept it matched as its class name.. | ✅ |
model_id |
str |
Streaming SAM3 model id resolved by inference_models.. |
✅ |
threshold |
float |
Minimum detection score for emitted masks. Scores come from SAM3's per-object concept detection head.. | ✅ |
The Refs column marks possibility to parametrise the property with dynamic values available
in workflow runtime. See Bindings for more info.
Runtime compatibility¶
-
soft— runtimehosted_serverless,dedicated_deployment; executionremote; inputvideo - Block keeps per-video state in process memory (keyed by video_metadata.video_identifier). With remote step execution on stateless or multi-replica HTTP runtimes, successive requests may be served by different worker processes, so the state resets between calls and the output is meaningless for tracking / counting / aggregation. Use local step execution in an InferencePipeline for stable cross-frame results.
-
hard— runtimeself_hosted_cpu; executionlocal - Requires a GPU; the streaming SAM3 video model needs CUDA.
-
soft— inputimage - Block depends on temporal context from video or repeated-frame workflows. With a still image/photo, there is no meaningful history to track, compare, aggregate, or visualize, so the block provides little or no benefit.
Available Connections¶
Compatible Blocks
Check what blocks you can connect to SAM3 Video Tracker in version v1.
- inputs:
Image Preprocessing,Single-Label Classification Model,Anthropic Claude,Image Slicer,Dynamic Crop,Bounding Box Visualization,Object Detection Model,Absolute Static Crop,SIFT Comparison,Stitch Images,Stitch OCR Detections,OpenAI,Instance Segmentation Model,Email Notification,Stability AI Inpainting,Cosine Similarity,EasyOCR,Llama 3.2 Vision,Florence-2 Model,Gaze Detection,Roboflow Custom Metadata,Dynamic Zone,Auto Rotate on Edges,Keypoint Detection Model,Stability AI Outpainting,Model Comparison Visualization,Slack Notification,Line Counter Visualization,Camera Calibration,Single-Label Classification Model,Clip Comparison,VLM As Detector,CogVLM,Camera Focus,Corner Visualization,Ellipse Visualization,PP-OCR,Morphological Transformation,Detections List Roll-Up,Anthropic Claude,Roboflow Visual Search,Color Visualization,Instance Segmentation Model,OpenAI,Triangle Visualization,Object Detection Model,Image Contours,Image Threshold,Current Time,Roboflow Visual Search Classifier,QR Code Generator,OpenAI-Compatible LLM,Florence-2 Model,Semantic Segmentation Model,Polygon Zone Visualization,Stitch OCR Detections,Roboflow Asset Library Attributes,GeoTag Detection,Microsoft SQL Server Sink,VLM As Classifier,Camera Focus,Image Convert Grayscale,Label Visualization,Llama 3.2 Vision,Stability AI Image Generation,Instance Segmentation Model,MQTT Writer,Roboflow Dataset Upload,Local File Sink,Google Gemini,Event Writer,Semantic Segmentation Model,Depth Estimation,Google Gemini,OpenAI,Trace Visualization,Twilio SMS Notification,PLC EthernetIP,LMM For Classification,Object Detection Model,Webhook Sink,Halo Visualization,Buffer,Mask Visualization,Pixelate Visualization,Twilio SMS/MMS Notification,MoonshotAI Kimi,Dot Visualization,Multi-Label Classification Model,Image Stack,OPC UA Writer Sink,Google Gemini,Keypoint Visualization,Dimension Collapse,LMM,Image Slicer,OCR Model,Circle Visualization,Contrast Enhancement,Relative Static Crop,Morphological Transformation,Email Notification,Halo Visualization,Clip Comparison,Cosmos 3,Polygon Visualization,Qwen-VL,PLC Writer,Google Gemma,Crop Visualization,Qwen 3.5 API,Model Monitoring Inference Aggregator,Keypoint Detection Model,Size Measurement,Icon Visualization,Heatmap Visualization,Single-Label Classification Model,Motion Detection,Multi-Label Classification Model,Google Gemma API,Instance Segmentation Model,CSV Formatter,Image Blur,Background Color Visualization,Grid Visualization,Blur Visualization,GLM-OCR,Anthropic Claude,Reference Path Visualization,Classification Label Visualization,Google Vision OCR,Perspective Correction,Background Subtraction,Polygon Visualization,Contrast Equalization,SIFT,Qwen 3.6 API,Text Display,MoonshotAI Kimi,OpenRouter,Qwen3.5-VL,Roboflow Vision Events,Identify Changes,OpenAI,Keypoint Detection Model,PLC ModbusTCP,Multi-Label Classification Model,S3 Sink,Roboflow Dataset Upload - outputs:
Roboflow Dataset Upload,Trace Visualization,SAM 3 Interactive,Time in Zone,Dynamic Crop,Detections Filter,Mask Area Measurement,BoT-SORT Tracker,Bounding Box Visualization,Mask Edge Snap,Velocity,Halo Visualization,Mask Visualization,Path Deviation,Pixelate Visualization,Dot Visualization,Stability AI Inpainting,Frame Delay,Distance Measurement,OC-SORT Tracker,Track Class Lock,Florence-2 Model,Roboflow Custom Metadata,Dynamic Zone,Detections Combine,Detections Transformation,PTZ Tracking (ONVIF),Time in Zone,Byte Tracker,Circle Visualization,Model Comparison Visualization,Detections Classes Replacement,Byte Tracker,SAM2 Video Tracker,ByteTrack Tracker,Per-Class Confidence Filter,Halo Visualization,Polygon Visualization,SORT Tracker,Ellipse Visualization,Corner Visualization,Camera Focus,Crop Visualization,Model Monitoring Inference Aggregator,Detections List Roll-Up,Size Measurement,Icon Visualization,Heatmap Visualization,Color Visualization,Detections Consensus,Overlap Analysis,Time in Zone,Detection Event Log,Detections Stitch,Background Color Visualization,Segment Anything 2 Model,Triangle Visualization,Detections Stabilizer,Blur Visualization,Detections Merge,Florence-2 Model,Path Deviation,GeoTag Detection,Perspective Correction,Polygon Visualization,Label Visualization,Detection Offset,Line Counter,Roboflow Dataset Upload,Roboflow Vision Events,Bounding Rectangle,Overlap Filter,Event Writer,Line Counter,Byte Tracker
Input and Output Bindings¶
The available connections depend on its binding kinds. Check what binding kinds
SAM3 Video Tracker in version v1 has.
Bindings
-
input
images(image): The image to infer on..class_names(Union[string,list_of_values]): Concepts to segment and track, as a list of phrases (or a single comma-separated string). Each emitted mask carries the concept it matched as its class name..model_id(roboflow_model_id): Streaming SAM3 model id resolved byinference_models..threshold(float): Minimum detection score for emitted masks. Scores come from SAM3's per-object concept detection head..
-
output
predictions(instance_segmentation_prediction): Prediction with detected bounding boxes and segmentation masks in form of sv.Detections(...) object.
Example JSON definition of step SAM3 Video Tracker in version v1
{
"name": "<your_step_name_here>",
"type": "roboflow_core/sam3_video@v1",
"images": "$inputs.image",
"class_names": [
"person",
"forklift"
],
"model_id": "sam3video",
"threshold": 0.5
}