LMM For Classification¶
Deprecated
This block is deprecated and may be removed in a future release.
Class: LMMForClassificationBlockV1
Source: inference.core.workflows.core_steps.models.foundation.lmm_classifier.v1.LMMForClassificationBlockV1
Classify an image into one or more categories using a Large Multimodal Model (LMM).
You can specify arbitrary classes to an LMMBlock.
The LLMBlock supports two LMMs:
- OpenAI's GPT-4 with Vision.
You need to provide your OpenAI API key to use the GPT-4 with Vision model.
Type identifier¶
Use the following identifier in step "type" field: roboflow_core/lmm_for_classification@v1to add the block as
as step in your workflow.
Properties¶
| Name | Type | Description | Refs |
|---|---|---|---|
name |
str |
Enter a unique identifier for this step.. | ❌ |
lmm_type |
str |
Type of LMM to be used. | ✅ |
classes |
List[str] |
List of classes that LMM shall classify against. | ✅ |
lmm_config |
LMMConfig |
Configuration of LMM. | ❌ |
remote_api_key |
str |
Holds API key required to call LMM model - in current state of development, we require OpenAI key when lmm_type=gpt_4v.. |
✅ |
The Refs column marks possibility to parametrise the property with dynamic values available
in workflow runtime. See Bindings for more info.
Runtime compatibility¶
-
requires_internet— air-gapped / offline deployments - This block depends on a service that is not reachable from fully offline / air-gapped deployments.
-
hard— runtimehosted_serverless; executionremote - LMM_ENABLED=False on Roboflow Hosted Serverless: the /llm_v1 endpoint is not registered, so run_remotely() returns 404.
Available Connections¶
Compatible Blocks
Check what blocks you can connect to LMM For Classification in version v1.
- inputs:
Keypoint Visualization,Twilio SMS/MMS Notification,OPC UA Writer Sink,Qwen 3.6 API,Grid Visualization,Llama 3.2 Vision,PLC Writer,Absolute Static Crop,Roboflow Visual Search Classifier,Image Threshold,Polygon Zone Visualization,CogVLM,MQTT Writer,Contrast Equalization,Google Gemini,Detections List Roll-Up,OCR Model,GLM-OCR,Background Subtraction,Contrast Enhancement,Reference Path Visualization,Twilio SMS Notification,Instance Segmentation Model,Google Gemma API,VLM As Detector,Florence-2 Model,Dynamic Crop,Halo Visualization,Image Blur,LMM,Ellipse Visualization,OpenAI,Florence-2 Model,LMM For Classification,Event Writer,OpenAI,Keypoint Detection Model,Google Gemma,Buffer,Slack Notification,Heatmap Visualization,Email Notification,Circle Visualization,Perspective Correction,Camera Focus,Crop Visualization,Polygon Visualization,Multi-Label Classification Model,Stability AI Inpainting,OpenAI-Compatible LLM,Object Detection Model,VLM As Classifier,PLC ModbusTCP,Image Slicer,Google Gemini,Roboflow Vision Events,Size Measurement,Local File Sink,Dot Visualization,Line Counter Visualization,Depth Estimation,Stitch OCR Detections,PP-OCR,Blur Visualization,EasyOCR,Clip Comparison,Anthropic Claude,PLC EthernetIP,CSV Formatter,Color Visualization,Dynamic Zone,Image Convert Grayscale,Motion Detection,Stitch OCR Detections,Single-Label Classification Model,Qwen 3.5 API,Trace Visualization,Icon Visualization,Current Time,Model Comparison Visualization,Camera Focus,QR Code Generator,Google Gemini,Image Stack,Qwen-VL,Mask Visualization,MoonshotAI Kimi,Microsoft SQL Server Sink,Text Display,Stability AI Image Generation,Google Vision OCR,Webhook Sink,OpenAI,Pixelate Visualization,Stitch Images,S3 Sink,Classification Label Visualization,MoonshotAI Kimi,Polygon Visualization,Anthropic Claude,Dimension Collapse,Roboflow Dataset Upload,Morphological Transformation,Bounding Box Visualization,Image Contours,Image Slicer,Stability AI Outpainting,Anthropic Claude,Roboflow Custom Metadata,Camera Calibration,Roboflow Asset Library Attributes,Morphological Transformation,Relative Static Crop,Background Color Visualization,Model Monitoring Inference Aggregator,Clip Comparison,Corner Visualization,GeoTag Detection,SIFT,Roboflow Visual Search,OpenRouter,Llama 3.2 Vision,Qwen3.5-VL,Email Notification,Triangle Visualization,OpenAI,Roboflow Dataset Upload,Image Preprocessing,SIFT Comparison,Halo Visualization,Label Visualization - outputs:
Keypoint Visualization,Twilio SMS/MMS Notification,OPC UA Writer Sink,Keypoint Detection Model,Stitch OCR Detections,Perception Encoder Embedding Model,Qwen 3.6 API,Anthropic Claude,Segment Anything 2 Model,SAM 3,Color Visualization,Llama 3.2 Vision,Stitch OCR Detections,YOLO-World Model,Distance Measurement,Roboflow Visual Search Classifier,Qwen 3.5 API,Image Threshold,Polygon Zone Visualization,CogVLM,Trace Visualization,Icon Visualization,Current Time,Cache Get,Model Comparison Visualization,QR Code Generator,Google Gemini,Detections Stitch,MQTT Writer,Contrast Equalization,Google Gemini,GLM-OCR,Instance Segmentation Model,Qwen-VL,Mask Visualization,Reference Path Visualization,Twilio SMS Notification,Google Gemma API,Instance Segmentation Model,MoonshotAI Kimi,PTZ Tracking (ONVIF),Detections Classes Replacement,Microsoft SQL Server Sink,Text Display,Florence-2 Model,Dynamic Crop,Halo Visualization,Image Blur,LMM,Stability AI Image Generation,Google Vision OCR,Seg Preview,SAM 3,Webhook Sink,OpenAI,Ellipse Visualization,OpenAI,Florence-2 Model,Line Counter,LMM For Classification,S3 Sink,Time in Zone,Classification Label Visualization,Instance Segmentation Model,MoonshotAI Kimi,Polygon Visualization,Anthropic Claude,Cache Set,Semantic Segmentation Model,OpenAI,Path Deviation,Event Writer,Instance Segmentation Model,Roboflow Dataset Upload,Time in Zone,Google Gemma,Morphological Transformation,Halo Visualization,Bounding Box Visualization,Slack Notification,Heatmap Visualization,Stability AI Outpainting,Anthropic Claude,Email Notification,Circle Visualization,Roboflow Custom Metadata,Perspective Correction,Roboflow Asset Library Attributes,Crop Visualization,Polygon Visualization,Morphological Transformation,Line Counter,Path Deviation,Background Color Visualization,Model Monitoring Inference Aggregator,Stability AI Inpainting,Clip Comparison,OpenAI-Compatible LLM,Corner Visualization,SAM 3,Google Gemini,Roboflow Vision Events,Moondream2,Roboflow Visual Search,Multi-Label Classification Model,OpenRouter,Qwen3.5-VL,Size Measurement,Llama 3.2 Vision,Local File Sink,Object Detection Model,Email Notification,CLIP Embedding Model,SAM3 Video Tracker,Single-Label Classification Model,Triangle Visualization,OpenAI,Pixel Color Count,Dot Visualization,Roboflow Dataset Upload,Image Preprocessing,SIFT Comparison,Line Counter Visualization,Time in Zone,Depth Estimation,Label Visualization
Input and Output Bindings¶
The available connections depend on its binding kinds. Check what binding kinds
LMM For Classification in version v1 has.
Bindings
-
input
images(image): The image to infer on..lmm_type(string): Type of LMM to be used.classes(list_of_values): List of classes that LMM shall classify against.remote_api_key(Union[secret,string]): Holds API key required to call LMM model - in current state of development, we require OpenAI key whenlmm_type=gpt_4v..
-
output
raw_output(string): String value.top(top_class): String value representing top class predicted by classification model.parent_id(parent_id): Identifier of parent for step output.root_parent_id(parent_id): Identifier of parent for step output.image(image_metadata): Dictionary with image metadata required by supervision.prediction_type(prediction_type): String value with type of prediction.
Example JSON definition of step LMM For Classification in version v1
{
"name": "<your_step_name_here>",
"type": "roboflow_core/lmm_for_classification@v1",
"images": "$inputs.image",
"lmm_type": "gpt_4v",
"classes": [
"a",
"b"
],
"lmm_config": {
"gpt_image_detail": "low",
"gpt_model_version": "gpt-4o",
"max_tokens": 200
},
"remote_api_key": "xxx-xxx"
}