VLM as Classifier¶
v2¶
Class: VLMAsClassifierBlockV2
(there are multiple versions of this block)
Source: inference.core.workflows.core_steps.formatters.vlm_as_classifier.v2.VLMAsClassifierBlockV2
Warning: This block has multiple versions. Please refer to the specific version for details. You can learn more about how versions work here: Versioning
The block expects string input that would be produced by blocks exposing Large Language Models (LLMs) and Visual Language Models (VLMs). Input is parsed to classification prediction and returned as block output.
Accepted formats:
-
valid JSON strings
-
JSON documents wrapped with Markdown tags (very common for GPT responses)
Example:
{"my": "json"}
Details regarding block behavior:
-
error_status
is setTrue
whenever parsing cannot be completed -
in case of multiple markdown blocks with raw JSON content - only first will be parsed
Type identifier¶
Use the following identifier in step "type"
field: roboflow_core/vlm_as_classifier@v2
to add the block as
as step in your workflow.
Properties¶
Name | Type | Description | Refs |
---|---|---|---|
name |
str |
Enter a unique identifier for this step.. | ❌ |
classes |
List[str] |
List of all classes used by the model, required to generate mapping between class name and class id.. | ✅ |
The Refs column marks possibility to parametrise the property with dynamic values available
in workflow
runtime. See Bindings for more info.
Available Connections¶
Compatible Blocks
Check what blocks you can connect to VLM as Classifier
in version v2
.
- inputs:
Image Convert Grayscale
,Camera Focus
,Depth Estimation
,Perspective Correction
,Image Threshold
,Polygon Visualization
,Camera Calibration
,Line Counter Visualization
,Dynamic Zone
,Pixelate Visualization
,Color Visualization
,SIFT
,Circle Visualization
,Dynamic Crop
,Dot Visualization
,Model Comparison Visualization
,Image Slicer
,Background Color Visualization
,Absolute Static Crop
,Mask Visualization
,Keypoint Visualization
,Crop Visualization
,Grid Visualization
,Clip Comparison
,Florence-2 Model
,Relative Static Crop
,Bounding Box Visualization
,Stitch Images
,Google Gemini
,Clip Comparison
,Buffer
,Ellipse Visualization
,Image Blur
,Reference Path Visualization
,Label Visualization
,SIFT Comparison
,Image Preprocessing
,Image Slicer
,Triangle Visualization
,Anthropic Claude
,Corner Visualization
,Classification Label Visualization
,Stability AI Inpainting
,Florence-2 Model
,Image Contours
,OpenAI
,Dimension Collapse
,Blur Visualization
,Llama 3.2 Vision
,Polygon Zone Visualization
,Size Measurement
,Trace Visualization
,Halo Visualization
,Stability AI Image Generation
- outputs:
Model Monitoring Inference Aggregator
,Instance Segmentation Model
,Segment Anything 2 Model
,Keypoint Detection Model
,Perspective Correction
,Single-Label Classification Model
,Polygon Visualization
,Line Counter Visualization
,Multi-Label Classification Model
,Object Detection Model
,Email Notification
,Pixelate Visualization
,Color Visualization
,Time in Zone
,Circle Visualization
,Dot Visualization
,Slack Notification
,Model Comparison Visualization
,Detections Classes Replacement
,Background Color Visualization
,Mask Visualization
,Twilio SMS Notification
,Roboflow Custom Metadata
,Crop Visualization
,Single-Label Classification Model
,Template Matching
,Time in Zone
,Keypoint Visualization
,Detections Consensus
,Roboflow Dataset Upload
,Instance Segmentation Model
,Webhook Sink
,Ellipse Visualization
,Halo Visualization
,Reference Path Visualization
,Label Visualization
,SIFT Comparison
,Multi-Label Classification Model
,Gaze Detection
,Roboflow Dataset Upload
,Triangle Visualization
,Keypoint Detection Model
,Corner Visualization
,Classification Label Visualization
,Blur Visualization
,Polygon Zone Visualization
,Object Detection Model
,Trace Visualization
,Bounding Box Visualization
Input and Output Bindings¶
The available connections depend on its binding kinds. Check what binding kinds
VLM as Classifier
in version v2
has.
Bindings
-
input
image
(image
): The image which was the base to generate VLM prediction.vlm_output
(language_model_output
): The string with raw classification prediction to parse..classes
(list_of_values
): List of all classes used by the model, required to generate mapping between class name and class id..
-
output
error_status
(boolean
): Boolean flag.predictions
(classification_prediction
): Predictions from classifier.inference_id
(inference_id
): Inference identifier.
Example JSON definition of step VLM as Classifier
in version v2
{
"name": "<your_step_name_here>",
"type": "roboflow_core/vlm_as_classifier@v2",
"image": "$inputs.image",
"vlm_output": [
"$steps.lmm.output"
],
"classes": [
"$steps.lmm.classes",
"$inputs.classes",
[
"class_a",
"class_b"
]
]
}
v1¶
Class: VLMAsClassifierBlockV1
(there are multiple versions of this block)
Source: inference.core.workflows.core_steps.formatters.vlm_as_classifier.v1.VLMAsClassifierBlockV1
Warning: This block has multiple versions. Please refer to the specific version for details. You can learn more about how versions work here: Versioning
The block expects string input that would be produced by blocks exposing Large Language Models (LLMs) and Visual Language Models (VLMs). Input is parsed to classification prediction and returned as block output.
Accepted formats:
-
valid JSON strings
-
JSON documents wrapped with Markdown tags (very common for GPT responses)
Example:
{"my": "json"}
Details regarding block behavior:
-
error_status
is setTrue
whenever parsing cannot be completed -
in case of multiple markdown blocks with raw JSON content - only first will be parsed
Type identifier¶
Use the following identifier in step "type"
field: roboflow_core/vlm_as_classifier@v1
to add the block as
as step in your workflow.
Properties¶
Name | Type | Description | Refs |
---|---|---|---|
name |
str |
Enter a unique identifier for this step.. | ❌ |
classes |
List[str] |
List of all classes used by the model, required to generate mapping between class name and class id.. | ✅ |
The Refs column marks possibility to parametrise the property with dynamic values available
in workflow
runtime. See Bindings for more info.
Available Connections¶
Compatible Blocks
Check what blocks you can connect to VLM as Classifier
in version v1
.
- inputs:
Image Convert Grayscale
,Camera Focus
,Depth Estimation
,Perspective Correction
,Image Threshold
,Polygon Visualization
,Camera Calibration
,Line Counter Visualization
,Dynamic Zone
,Pixelate Visualization
,Color Visualization
,SIFT
,Circle Visualization
,Dynamic Crop
,Dot Visualization
,Model Comparison Visualization
,Image Slicer
,Background Color Visualization
,Absolute Static Crop
,Mask Visualization
,Keypoint Visualization
,Crop Visualization
,Grid Visualization
,Clip Comparison
,Florence-2 Model
,Relative Static Crop
,Bounding Box Visualization
,Stitch Images
,Google Gemini
,Clip Comparison
,Buffer
,Ellipse Visualization
,Image Blur
,Reference Path Visualization
,Label Visualization
,SIFT Comparison
,Image Preprocessing
,Image Slicer
,Triangle Visualization
,Anthropic Claude
,Corner Visualization
,Classification Label Visualization
,Stability AI Inpainting
,Florence-2 Model
,Image Contours
,OpenAI
,Dimension Collapse
,Blur Visualization
,Llama 3.2 Vision
,Polygon Zone Visualization
,Size Measurement
,Trace Visualization
,Halo Visualization
,Stability AI Image Generation
- outputs:
Segment Anything 2 Model
,Keypoint Detection Model
,Image Threshold
,Detections Stitch
,Polygon Visualization
,YOLO-World Model
,CogVLM
,Pixelate Visualization
,Time in Zone
,Circle Visualization
,Dynamic Crop
,Dot Visualization
,Slack Notification
,Model Comparison Visualization
,Background Color Visualization
,Cache Set
,Keypoint Visualization
,Roboflow Custom Metadata
,Single-Label Classification Model
,Cache Get
,Template Matching
,Time in Zone
,LMM
,Detections Consensus
,Roboflow Dataset Upload
,LMM For Classification
,Florence-2 Model
,Google Gemini
,Label Visualization
,Local File Sink
,Gaze Detection
,Keypoint Detection Model
,Triangle Visualization
,Anthropic Claude
,Corner Visualization
,Stability AI Inpainting
,Florence-2 Model
,Path Deviation
,Path Deviation
,Llama 3.2 Vision
,Size Measurement
,Line Counter
,Model Monitoring Inference Aggregator
,Instance Segmentation Model
,Distance Measurement
,Perspective Correction
,Google Vision OCR
,Single-Label Classification Model
,Line Counter Visualization
,Multi-Label Classification Model
,Object Detection Model
,Email Notification
,OpenAI
,Color Visualization
,Detections Classes Replacement
,Mask Visualization
,Twilio SMS Notification
,Crop Visualization
,Pixel Color Count
,Clip Comparison
,Bounding Box Visualization
,Instance Segmentation Model
,Webhook Sink
,Ellipse Visualization
,Image Blur
,Reference Path Visualization
,SIFT Comparison
,Multi-Label Classification Model
,Line Counter
,Image Preprocessing
,Roboflow Dataset Upload
,OpenAI
,Classification Label Visualization
,CLIP Embedding Model
,Blur Visualization
,Polygon Zone Visualization
,Object Detection Model
,Trace Visualization
,Halo Visualization
,Stability AI Image Generation
Input and Output Bindings¶
The available connections depend on its binding kinds. Check what binding kinds
VLM as Classifier
in version v1
has.
Bindings
-
input
image
(image
): The image which was the base to generate VLM prediction.vlm_output
(language_model_output
): The string with raw classification prediction to parse..classes
(list_of_values
): List of all classes used by the model, required to generate mapping between class name and class id..
-
output
error_status
(boolean
): Boolean flag.predictions
(classification_prediction
): Predictions from classifier.inference_id
(string
): String value.
Example JSON definition of step VLM as Classifier
in version v1
{
"name": "<your_step_name_here>",
"type": "roboflow_core/vlm_as_classifier@v1",
"image": "$inputs.image",
"vlm_output": [
"$steps.lmm.output"
],
"classes": [
"$steps.lmm.classes",
"$inputs.classes",
[
"class_a",
"class_b"
]
]
}