LMM For Classification¶
Deprecated
This block is deprecated and may be removed in a future release.
Class: LMMForClassificationBlockV1
Source: inference.core.workflows.core_steps.models.foundation.lmm_classifier.v1.LMMForClassificationBlockV1
Classify an image into one or more categories using a Large Multimodal Model (LMM).
You can specify arbitrary classes to an LMMBlock.
The LLMBlock supports two LMMs:
- OpenAI's GPT-4 with Vision.
You need to provide your OpenAI API key to use the GPT-4 with Vision model.
Type identifier¶
Use the following identifier in step "type" field: roboflow_core/lmm_for_classification@v1to add the block as
as step in your workflow.
Properties¶
| Name | Type | Description | Refs |
|---|---|---|---|
name |
str |
Enter a unique identifier for this step.. | ❌ |
lmm_type |
str |
Type of LMM to be used. | ✅ |
classes |
List[str] |
List of classes that LMM shall classify against. | ✅ |
lmm_config |
LMMConfig |
Configuration of LMM. | ❌ |
remote_api_key |
str |
Holds API key required to call LMM model - in current state of development, we require OpenAI key when lmm_type=gpt_4v.. |
✅ |
The Refs column marks possibility to parametrise the property with dynamic values available
in workflow runtime. See Bindings for more info.
Available Connections¶
Compatible Blocks
Check what blocks you can connect to LMM For Classification in version v1.
- inputs:
Roboflow Dataset Upload,Line Counter Visualization,OCR Model,Image Slicer,Instance Segmentation Model,Color Visualization,Ellipse Visualization,Polygon Visualization,Relative Static Crop,Webhook Sink,Trace Visualization,Stitch OCR Detections,Camera Focus,Qwen 3.5 API,OpenAI,Buffer,Size Measurement,Image Threshold,Heatmap Visualization,Florence-2 Model,Halo Visualization,GLM-OCR,Dot Visualization,S3 Sink,Twilio SMS Notification,Model Monitoring Inference Aggregator,Google Gemini,Roboflow Dataset Upload,Dynamic Zone,Clip Comparison,Pixelate Visualization,Twilio SMS/MMS Notification,Polygon Zone Visualization,Motion Detection,Blur Visualization,Background Subtraction,Text Display,CSV Formatter,Stability AI Image Generation,Perspective Correction,Anthropic Claude,Bounding Box Visualization,Depth Estimation,Stability AI Inpainting,Polygon Visualization,SIFT,Roboflow Vision Events,Google Gemini,Label Visualization,Grid Visualization,Qwen3.5-VL,Contrast Equalization,Triangle Visualization,Halo Visualization,Circle Visualization,Mask Visualization,OpenAI,MoonshotAI Kimi,Llama 3.2 Vision,Email Notification,Slack Notification,Object Detection Model,Stability AI Outpainting,Email Notification,Google Gemma API,Google Vision OCR,Image Preprocessing,Google Gemini,EasyOCR,OpenAI,Anthropic Claude,Model Comparison Visualization,Roboflow Custom Metadata,Single-Label Classification Model,VLM As Classifier,Detections List Roll-Up,Stitch Images,Qwen 3.6 API,SIFT Comparison,Morphological Transformation,CogVLM,Crop Visualization,Camera Calibration,Florence-2 Model,Icon Visualization,Local File Sink,Image Contours,Reference Path Visualization,Dimension Collapse,Anthropic Claude,Clip Comparison,VLM As Detector,LMM,Classification Label Visualization,Image Slicer,Absolute Static Crop,Image Blur,Multi-Label Classification Model,Image Convert Grayscale,OpenAI,Corner Visualization,Dynamic Crop,Keypoint Visualization,QR Code Generator,Camera Focus,LMM For Classification,Morphological Transformation,Keypoint Detection Model,Contrast Enhancement,Background Color Visualization,Stitch OCR Detections - outputs:
CLIP Embedding Model,Detections Stitch,Roboflow Dataset Upload,Line Counter Visualization,Email Notification,Google Gemma API,Stability AI Outpainting,Google Vision OCR,Distance Measurement,Instance Segmentation Model,Google Gemini,Image Preprocessing,Color Visualization,Object Detection Model,Multi-Label Classification Model,OpenAI,Ellipse Visualization,Polygon Visualization,Anthropic Claude,Single-Label Classification Model,Time in Zone,Detections Classes Replacement,Cache Set,Webhook Sink,Model Comparison Visualization,Trace Visualization,Stitch OCR Detections,Qwen 3.5 API,Roboflow Custom Metadata,YOLO-World Model,OpenAI,SAM 3,Perception Encoder Embedding Model,Instance Segmentation Model,Size Measurement,Image Threshold,Heatmap Visualization,Qwen 3.6 API,SIFT Comparison,Morphological Transformation,Florence-2 Model,Halo Visualization,Instance Segmentation Model,CogVLM,Crop Visualization,Florence-2 Model,Path Deviation,Time in Zone,GLM-OCR,Dot Visualization,S3 Sink,Path Deviation,SAM 3,Semantic Segmentation Model,Twilio SMS Notification,Seg Preview,Model Monitoring Inference Aggregator,Local File Sink,Google Gemini,Roboflow Dataset Upload,Icon Visualization,Line Counter,Twilio SMS/MMS Notification,Time in Zone,Polygon Zone Visualization,Reference Path Visualization,Anthropic Claude,Text Display,Clip Comparison,LMM,Stability AI Image Generation,Perspective Correction,Anthropic Claude,Line Counter,Bounding Box Visualization,Pixel Color Count,Depth Estimation,Classification Label Visualization,Image Blur,Stability AI Inpainting,Polygon Visualization,SAM 3,Roboflow Vision Events,OpenAI,Google Gemini,Label Visualization,Corner Visualization,Keypoint Detection Model,Contrast Equalization,Dynamic Crop,Triangle Visualization,Keypoint Visualization,Moondream2,QR Code Generator,Halo Visualization,Circle Visualization,Segment Anything 2 Model,Mask Visualization,LMM For Classification,Morphological Transformation,OpenAI,MoonshotAI Kimi,Llama 3.2 Vision,Background Color Visualization,Email Notification,PTZ Tracking (ONVIF),Slack Notification,Stitch OCR Detections,Cache Get
Input and Output Bindings¶
The available connections depend on its binding kinds. Check what binding kinds
LMM For Classification in version v1 has.
Bindings
-
input
images(image): The image to infer on..lmm_type(string): Type of LMM to be used.classes(list_of_values): List of classes that LMM shall classify against.remote_api_key(Union[secret,string]): Holds API key required to call LMM model - in current state of development, we require OpenAI key whenlmm_type=gpt_4v..
-
output
raw_output(string): String value.top(top_class): String value representing top class predicted by classification model.parent_id(parent_id): Identifier of parent for step output.root_parent_id(parent_id): Identifier of parent for step output.image(image_metadata): Dictionary with image metadata required by supervision.prediction_type(prediction_type): String value with type of prediction.
Example JSON definition of step LMM For Classification in version v1
{
"name": "<your_step_name_here>",
"type": "roboflow_core/lmm_for_classification@v1",
"images": "$inputs.image",
"lmm_type": "gpt_4v",
"classes": [
"a",
"b"
],
"lmm_config": {
"gpt_image_detail": "low",
"gpt_model_version": "gpt-4o",
"max_tokens": 200
},
"remote_api_key": "xxx-xxx"
}