CogVLM

Class: CogVLMBlockV1

Source: inference.core.workflows.core_steps.models.foundation.cog_vlm.v1.CogVLMBlockV1

CogVLM reached End Of Life

Due to dependencies conflicts with newer models and security vulnerabilities discovered in transformers library patched in the versions of library incompatible with the model we announced End Of Life for CogVLM support in inference, effective since release 0.38.0.

We are leaving this block in ecosystem until release 0.42.0 for clients to get informed about change that was introduced.

Starting as of now, all Workflows using the block stop being functional (runtime error will be raised), after inference release 0.42.0 - this block will be removed and Execution Engine will raise compilation error seeing the block in Workflow definition.

Ask a question to CogVLM, an open source vision-language model.

This model requires a GPU and can only be run on self-hosted devices, and is not available on the Roboflow Hosted API.

This model was previously part of the LMM block.

Type identifier

Use the following identifier in step "type" field: roboflow_core/cog_vlm@v1to add the block as as step in your workflow.

Properties

Name	Type	Description	Refs
name	str	Enter a unique identifier for this step..	❌
prompt	str	Text prompt to the CogVLM model.	✅
json_output_format	Dict[str, str]	Holds dictionary that maps name of requested output field into its description.	❌

The Refs column marks possibility to parametrise the property with dynamic values available in workflow runtime. See Bindings for more info.

Available Connections

Compatible Blocks

Check what blocks you can connect to CogVLM in version v1.

• inputs: Icon Visualization, Image Preprocessing, LMM, Blur Visualization, Twilio SMS Notification, Morphological Transformation, Roboflow Custom Metadata, Stitch Images, Color Visualization, Contrast Equalization, Llama 3.2 Vision, Circle Visualization, Stability AI Image Generation, Image Blur, Reference Path Visualization, SIFT, OpenAI, Email Notification, Halo Visualization, EasyOCR, Google Gemini, Trace Visualization, Roboflow Dataset Upload, Twilio SMS/MMS Notification, Single-Label Classification Model, Classification Label Visualization, Clip Comparison, Image Convert Grayscale, CogVLM, Google Vision OCR, Background Color Visualization, Stitch OCR Detections, Multi-Label Classification Model, Camera Calibration, VLM as Detector, LMM For Classification, Triangle Visualization, Text Display, Ellipse Visualization, Slack Notification, Mask Visualization, OpenAI, Local File Sink, Anthropic Claude, Polygon Zone Visualization, Polygon Visualization, Absolute Static Crop, Model Comparison Visualization, Label Visualization, Google Gemini, Webhook Sink, Line Counter Visualization, Perspective Correction, Florence-2 Model, Image Slicer, QR Code Generator, Instance Segmentation Model, Stability AI Outpainting, Anthropic Claude, Object Detection Model, VLM as Classifier, Grid Visualization, Relative Static Crop, CSV Formatter, Image Slicer, Image Contours, Stability AI Inpainting, Camera Focus, Google Gemini, Florence-2 Model, SIFT Comparison, Keypoint Detection Model, Dot Visualization, Camera Focus, Crop Visualization, Bounding Box Visualization, OCR Model, Background Subtraction, OpenAI, Roboflow Dataset Upload, Dynamic Crop, Keypoint Visualization, Email Notification, Model Monitoring Inference Aggregator, Image Threshold, Anthropic Claude, Corner Visualization, Depth Estimation, OpenAI, Pixelate Visualization
• outputs: Icon Visualization, LMM, Image Preprocessing, Blur Visualization, Detections Classes Replacement, Color Visualization, Contrast Equalization, Detections Merge, Cache Set, Llama 3.2 Vision, Velocity, Reference Path Visualization, SIFT, OpenAI, Buffer, SAM 3, Halo Visualization, Roboflow Dataset Upload, Trace Visualization, Dimension Collapse, Twilio SMS/MMS Notification, Detections Transformation, Single-Label Classification Model, VLM as Detector, Image Convert Grayscale, Path Deviation, Background Color Visualization, Multi-Label Classification Model, Qwen2.5-VL, Single-Label Classification Model, Camera Calibration, VLM as Detector, Triangle Visualization, Dynamic Zone, Ellipse Visualization, Seg Preview, Slack Notification, Absolute Static Crop, Time in Zone, Google Gemini, Webhook Sink, Line Counter Visualization, Florence-2 Model, Dominant Color, Detections Consensus, QR Code Generator, Property Definition, Anthropic Claude, Object Detection Model, Keypoint Detection Model, Image Slicer, Byte Tracker, Pixel Color Count, Stability AI Inpainting, VLM as Classifier, Image Contours, Line Counter, Google Gemini, Motion Detection, SIFT Comparison, Line Counter, Keypoint Detection Model, Dot Visualization, Byte Tracker, YOLO-World Model, Camera Focus, Bounding Box Visualization, OCR Model, Background Subtraction, OpenAI, SAM 3, Dynamic Crop, Qwen3-VL, Distance Measurement, Keypoint Visualization, Email Notification, Image Threshold, Anthropic Claude, Corner Visualization, Rate Limiter, Pixelate Visualization, SmolVLM2, SIFT Comparison, Twilio SMS Notification, Moondream2, Morphological Transformation, Roboflow Custom Metadata, Stitch Images, Detections Filter, Stability AI Image Generation, Circle Visualization, Image Blur, Template Matching, Detections List Roll-Up, Bounding Rectangle, Email Notification, Perception Encoder Embedding Model, Continue If, Detection Event Log, EasyOCR, Google Gemini, Instance Segmentation Model, Delta Filter, Classification Label Visualization, Clip Comparison, Detection Offset, Google Vision OCR, CogVLM, Data Aggregator, Stitch OCR Detections, QR Code Detection, Segment Anything 2 Model, LMM For Classification, Text Display, CLIP Embedding Model, SAM 3, Multi-Label Classification Model, Barcode Detection, Local File Sink, OpenAI, Mask Visualization, Anthropic Claude, Polygon Zone Visualization, Polygon Visualization, Model Comparison Visualization, Label Visualization, JSON Parser, Perspective Correction, Overlap Filter, Identify Changes, Image Slicer, Instance Segmentation Model, Cosine Similarity, Stability AI Outpainting, VLM as Classifier, Grid Visualization, Relative Static Crop, CSV Formatter, Size Measurement, Path Deviation, Camera Focus, Time in Zone, Florence-2 Model, Identify Outliers, Object Detection Model, Detections Stitch, Cache Get, First Non Empty Or Default, Crop Visualization, Clip Comparison, Expression, Byte Tracker, Detections Stabilizer, Roboflow Dataset Upload, PTZ Tracking (ONVIF).md), Model Monitoring Inference Aggregator, Time in Zone, Detections Combine, Depth Estimation, OpenAI, Gaze Detection

Input and Output Bindings

The available connections depend on its binding kinds. Check what binding kinds CogVLM in version v1 has.

Bindings

• input

  • images (image): The image to infer on..
  • prompt (string): Text prompt to the CogVLM model.

• output

  • parent_id (parent_id): Identifier of parent for step output.
  • root_parent_id (parent_id): Identifier of parent for step output.
  • image (image_metadata): Dictionary with image metadata required by supervision.
  • structured_output (dictionary): Dictionary.
  • raw_output (string): String value.
  • * (*): Equivalent of any element.

Example JSON definition of step CogVLM in version v1

{
    "name": "<your_step_name_here>",
    "type": "roboflow_core/cog_vlm@v1",
    "images": "$inputs.image",
    "prompt": "my prompt",
    "json_output_format": {
        "count": "number of cats in the picture"
    }
}