CogVLM

Class: CogVLMBlockV1

Source: inference.core.workflows.core_steps.models.foundation.cog_vlm.v1.CogVLMBlockV1

CogVLM reached End Of Life

Due to dependencies conflicts with newer models and security vulnerabilities discovered in transformers library patched in the versions of library incompatible with the model we announced End Of Life for CogVLM support in inference, effective since release 0.38.0.

We are leaving this block in ecosystem until release 0.42.0 for clients to get informed about change that was introduced.

Starting as of now, all Workflows using the block stop being functional (runtime error will be raised), after inference release 0.42.0 - this block will be removed and Execution Engine will raise compilation error seeing the block in Workflow definition.

Ask a question to CogVLM, an open source vision-language model.

This model requires a GPU and can only be run on self-hosted devices, and is not available on the Roboflow Hosted API.

This model was previously part of the LMM block.

Type identifier

Use the following identifier in step "type" field: roboflow_core/cog_vlm@v1to add the block as as step in your workflow.

Properties

Name	Type	Description	Refs
name	str	Enter a unique identifier for this step..	❌
prompt	str	Text prompt to the CogVLM model.	✅
json_output_format	Dict[str, str]	Holds dictionary that maps name of requested output field into its description.	❌

The Refs column marks possibility to parametrise the property with dynamic values available in workflow runtime. See Bindings for more info.

Available Connections

Compatible Blocks

Check what blocks you can connect to CogVLM in version v1.

• inputs: Clip Comparison, Halo Visualization, Anthropic Claude, Image Blur, Email Notification, Camera Focus, Text Display, CSV Formatter, Contrast Equalization, Object Detection Model, Blur Visualization, Roboflow Dataset Upload, Corner Visualization, Dynamic Crop, Classification Label Visualization, Relative Static Crop, Trace Visualization, Multi-Label Classification Model, Camera Focus, Mask Visualization, SIFT, Stitch OCR Detections, Background Color Visualization, Polygon Visualization, Local File Sink, Florence-2 Model, Stitch OCR Detections, Ellipse Visualization, Pixelate Visualization, SIFT Comparison, Roboflow Custom Metadata, LMM, OpenAI, Circle Visualization, Dot Visualization, Twilio SMS Notification, Absolute Static Crop, Morphological Transformation, Crop Visualization, Single-Label Classification Model, Google Gemini, Keypoint Visualization, Polygon Visualization, Google Vision OCR, Icon Visualization, Llama 3.2 Vision, Color Visualization, Image Contours, Stitch Images, OpenAI, VLM As Classifier, LMM For Classification, Email Notification, VLM As Detector, Anthropic Claude, OpenAI, Bounding Box Visualization, Triangle Visualization, Background Subtraction, Grid Visualization, Model Comparison Visualization, Reference Path Visualization, Line Counter Visualization, EasyOCR, Halo Visualization, OCR Model, Instance Segmentation Model, Slack Notification, Stability AI Inpainting, Image Slicer, Google Gemini, Florence-2 Model, Image Convert Grayscale, Stability AI Image Generation, Heatmap Visualization, Polygon Zone Visualization, Image Slicer, Label Visualization, Google Gemini, Depth Estimation, Image Preprocessing, OpenAI, Stability AI Outpainting, CogVLM, Webhook Sink, Image Threshold, Anthropic Claude, Camera Calibration, Perspective Correction, QR Code Generator, Keypoint Detection Model, Model Monitoring Inference Aggregator, Twilio SMS/MMS Notification, Roboflow Dataset Upload
• outputs: Anthropic Claude, Qwen3-VL, CLIP Embedding Model, Camera Focus, Continue If, SAM 3, Camera Focus, VLM As Classifier, Path Deviation, Detections Combine, Polygon Visualization, Size Measurement, SIFT Comparison, Byte Tracker, Detections List Roll-Up, OpenAI, Identify Outliers, Crop Visualization, Single-Label Classification Model, Keypoint Visualization, Llama 3.2 Vision, Gaze Detection, Image Contours, Stitch Images, Distance Measurement, Expression, Background Subtraction, EasyOCR, Slack Notification, Line Counter, Detection Event Log, Image Convert Grayscale, Pixel Color Count, Time in Zone, OpenAI, Object Detection Model, Stability AI Outpainting, Detections Merge, Image Threshold, Anthropic Claude, Cache Set, Clip Comparison, Image Blur, Contrast Equalization, Blur Visualization, Detections Stabilizer, VLM As Detector, Roboflow Dataset Upload, Dimension Collapse, Keypoint Detection Model, Stitch OCR Detections, YOLO-World Model, Bounding Rectangle, Florence-2 Model, Ellipse Visualization, Pixelate Visualization, SAM 3, Circle Visualization, Velocity, Absolute Static Crop, Morphological Transformation, Template Matching, Dominant Color, Overlap Filter, Google Gemini, Dynamic Zone, Polygon Visualization, Color Visualization, LMM For Classification, Email Notification, VLM As Detector, Bounding Box Visualization, Grid Visualization, Detections Consensus, Halo Visualization, Stability AI Inpainting, Florence-2 Model, Heatmap Visualization, Polygon Zone Visualization, Image Slicer, Label Visualization, Cache Get, QR Code Detection, Byte Tracker, QR Code Generator, Keypoint Detection Model, Roboflow Dataset Upload, Text Display, CSV Formatter, Segment Anything 2 Model, Rate Limiter, Dynamic Crop, Relative Static Crop, Qwen2.5-VL, Mask Visualization, SIFT, Background Color Visualization, PTZ Tracking (ONVIF).md), Moondream2, Stitch OCR Detections, Line Counter, Icon Visualization, Multi-Label Classification Model, OpenAI, VLM As Classifier, Anthropic Claude, SmolVLM2, OpenAI, Triangle Visualization, Detections Classes Replacement, Reference Path Visualization, Instance Segmentation Model, Line Counter Visualization, Time in Zone, Google Gemini, JSON Parser, Mask Area Measurement, Stability AI Image Generation, Seg Preview, Delta Filter, Buffer, Webhook Sink, Instance Segmentation Model, Perception Encoder Embedding Model, Perspective Correction, SAM 3, First Non Empty Or Default, Detection Offset, Clip Comparison, Halo Visualization, Email Notification, Object Detection Model, Corner Visualization, Classification Label Visualization, Trace Visualization, Multi-Label Classification Model, Barcode Detection, Local File Sink, Roboflow Custom Metadata, Single-Label Classification Model, Identify Changes, LMM, Dot Visualization, Twilio SMS Notification, Byte Tracker, SIFT Comparison, Property Definition, Google Vision OCR, Cosine Similarity, Model Comparison Visualization, OCR Model, Data Aggregator, Image Slicer, Detections Filter, Google Gemini, Depth Estimation, Image Preprocessing, Detections Stitch, Time in Zone, CogVLM, Path Deviation, Camera Calibration, Detections Transformation, Motion Detection, Model Monitoring Inference Aggregator, Twilio SMS/MMS Notification

Input and Output Bindings

The available connections depend on its binding kinds. Check what binding kinds CogVLM in version v1 has.

Bindings

• input

  • images (image): The image to infer on..
  • prompt (string): Text prompt to the CogVLM model.

• output

  • parent_id (parent_id): Identifier of parent for step output.
  • root_parent_id (parent_id): Identifier of parent for step output.
  • image (image_metadata): Dictionary with image metadata required by supervision.
  • structured_output (dictionary): Dictionary.
  • raw_output (string): String value.
  • * (*): Equivalent of any element.

Example JSON definition of step CogVLM in version v1

{
    "name": "<your_step_name_here>",
    "type": "roboflow_core/cog_vlm@v1",
    "images": "$inputs.image",
    "prompt": "my prompt",
    "json_output_format": {
        "count": "number of cats in the picture"
    }
}