CogVLM

Class: CogVLMBlockV1

Source: inference.core.workflows.core_steps.models.foundation.cog_vlm.v1.CogVLMBlockV1

CogVLM reached End Of Life

Due to dependencies conflicts with newer models and security vulnerabilities discovered in transformers library patched in the versions of library incompatible with the model we announced End Of Life for CogVLM support in inference, effective since release 0.38.0.

We are leaving this block in ecosystem until release 0.42.0 for clients to get informed about change that was introduced.

Starting as of now, all Workflows using the block stop being functional (runtime error will be raised), after inference release 0.42.0 - this block will be removed and Execution Engine will raise compilation error seeing the block in Workflow definition.

Ask a question to CogVLM, an open source vision-language model.

This model requires a GPU and can only be run on self-hosted devices, and is not available on the Roboflow Hosted API.

This model was previously part of the LMM block.

Type identifier

Use the following identifier in step "type" field: roboflow_core/cog_vlm@v1to add the block as as step in your workflow.

Properties

Name	Type	Description	Refs
name	str	Enter a unique identifier for this step..	❌
prompt	str	Text prompt to the CogVLM model.	✅
json_output_format	Dict[str, str]	Holds dictionary that maps name of requested output field into its description.	❌

The Refs column marks possibility to parametrise the property with dynamic values available in workflow runtime. See Bindings for more info.

Available Connections

Compatible Blocks

Check what blocks you can connect to CogVLM in version v1.

• inputs: Anthropic Claude, Mask Visualization, Classification Label Visualization, Instance Segmentation Model, Webhook Sink, Multi-Label Classification Model, Email Notification, QR Code Generator, Dynamic Crop, VLM As Detector, Google Gemini, LMM, Image Blur, Corner Visualization, Image Convert Grayscale, Stability AI Outpainting, Halo Visualization, Stability AI Inpainting, Object Detection Model, Image Contours, Trace Visualization, Google Vision OCR, Morphological Transformation, Triangle Visualization, Relative Static Crop, CSV Formatter, Text Display, Stitch Images, Google Gemini, Camera Calibration, Grid Visualization, Local File Sink, Slack Notification, VLM As Classifier, Roboflow Dataset Upload, Camera Focus, Color Visualization, Dot Visualization, Image Slicer, Polygon Visualization, Anthropic Claude, LMM For Classification, Line Counter Visualization, Llama 3.2 Vision, Keypoint Detection Model, Contrast Equalization, SIFT Comparison, Camera Focus, Background Subtraction, Image Slicer, Circle Visualization, Halo Visualization, Florence-2 Model, Blur Visualization, Label Visualization, Twilio SMS/MMS Notification, Clip Comparison, Email Notification, Ellipse Visualization, OpenAI, SIFT, Image Preprocessing, Model Monitoring Inference Aggregator, Single-Label Classification Model, OpenAI, Image Threshold, Background Color Visualization, Model Comparison Visualization, Depth Estimation, OpenAI, CogVLM, Absolute Static Crop, Roboflow Custom Metadata, EasyOCR, Stitch OCR Detections, Perspective Correction, Anthropic Claude, Pixelate Visualization, Stability AI Image Generation, Reference Path Visualization, Keypoint Visualization, Polygon Visualization, Twilio SMS Notification, Bounding Box Visualization, Polygon Zone Visualization, OCR Model, Icon Visualization, Crop Visualization, Stitch OCR Detections, Google Gemini, OpenAI, Florence-2 Model, Roboflow Dataset Upload
• outputs: Detections Consensus, Detections Merge, Webhook Sink, Multi-Label Classification Model, VLM As Detector, Multi-Label Classification Model, SAM 3, Corner Visualization, Image Convert Grayscale, Segment Anything 2 Model, Halo Visualization, Object Detection Model, Trace Visualization, Google Vision OCR, Instance Segmentation Model, CSV Formatter, Google Gemini, Slack Notification, VLM As Classifier, Color Visualization, Dot Visualization, Polygon Visualization, Buffer, Contrast Equalization, Identify Changes, Dimension Collapse, Velocity, Continue If, Moondream2, SIFT Comparison, Halo Visualization, Florence-2 Model, Blur Visualization, Twilio SMS/MMS Notification, Detections List Roll-Up, OpenAI, Model Comparison Visualization, Keypoint Detection Model, Gaze Detection, Twilio SMS Notification, SAM 3, OCR Model, Google Gemini, Florence-2 Model, Roboflow Dataset Upload, CLIP Embedding Model, Image Blur, Cache Set, SmolVLM2, Template Matching, Image Contours, Path Deviation, Morphological Transformation, Detections Stitch, Property Definition, Camera Calibration, Camera Focus, Detections Combine, Llama 3.2 Vision, Line Counter Visualization, LMM For Classification, SIFT Comparison, Camera Focus, Dominant Color, Time in Zone, Background Subtraction, Image Slicer, Identify Outliers, Qwen3-VL, Byte Tracker, SAM 3, Depth Estimation, Cosine Similarity, Line Counter, CogVLM, EasyOCR, Stitch OCR Detections, Qwen2.5-VL, Keypoint Visualization, Detection Event Log, Stitch OCR Detections, Crop Visualization, Pixel Color Count, OpenAI, Barcode Detection, Mask Visualization, Classification Label Visualization, Instance Segmentation Model, Email Notification, QR Code Generator, LMM, Detection Offset, Stability AI Outpainting, JSON Parser, Single-Label Classification Model, Clip Comparison, Text Display, Stitch Images, Local File Sink, Roboflow Dataset Upload, PTZ Tracking (ONVIF).md), Object Detection Model, Anthropic Claude, Byte Tracker, Detections Classes Replacement, Perception Encoder Embedding Model, First Non Empty Or Default, Expression, Label Visualization, Ellipse Visualization, OpenAI, SIFT, Model Monitoring Inference Aggregator, Single-Label Classification Model, Image Threshold, Background Color Visualization, Size Measurement, OpenAI, Polygon Visualization, Bounding Box Visualization, Overlap Filter, Icon Visualization, Time in Zone, Anthropic Claude, Dynamic Zone, Dynamic Crop, VLM As Detector, Google Gemini, Path Deviation, Line Counter, Byte Tracker, Stability AI Inpainting, Triangle Visualization, Bounding Rectangle, Relative Static Crop, Detections Filter, Grid Visualization, Detections Stabilizer, Delta Filter, Image Slicer, Keypoint Detection Model, Distance Measurement, Circle Visualization, Seg Preview, Clip Comparison, Email Notification, QR Code Detection, Image Preprocessing, Cache Get, Time in Zone, Absolute Static Crop, Roboflow Custom Metadata, Perspective Correction, Anthropic Claude, Pixelate Visualization, Data Aggregator, Stability AI Image Generation, Reference Path Visualization, VLM As Classifier, Polygon Zone Visualization, YOLO-World Model, Motion Detection, Rate Limiter, Detections Transformation

Input and Output Bindings

The available connections depend on its binding kinds. Check what binding kinds CogVLM in version v1 has.

Bindings

• input

  • images (image): The image to infer on..
  • prompt (string): Text prompt to the CogVLM model.

• output

  • parent_id (parent_id): Identifier of parent for step output.
  • root_parent_id (parent_id): Identifier of parent for step output.
  • image (image_metadata): Dictionary with image metadata required by supervision.
  • structured_output (dictionary): Dictionary.
  • raw_output (string): String value.
  • * (*): Equivalent of any element.

Example JSON definition of step CogVLM in version v1

{
    "name": "<your_step_name_here>",
    "type": "roboflow_core/cog_vlm@v1",
    "images": "$inputs.image",
    "prompt": "my prompt",
    "json_output_format": {
        "count": "number of cats in the picture"
    }
}