OpenAI

v3

Class: OpenAIBlockV3 (there are multiple versions of this block)

Source: inference.core.workflows.core_steps.models.foundation.openai.v3.OpenAIBlockV3

Warning: This block has multiple versions. Please refer to the specific version for details. You can learn more about how versions work here: Versioning

Ask a question to OpenAI's GPT models with vision capabilities (including GPT-5 and GPT-4o).

You can specify arbitrary text prompts or predefined ones, the block supports the following types of prompt:

• Open Prompt (unconstrained) - Use any prompt to generate a raw response

• Text Recognition (OCR) (ocr) - Model recognizes text in the image

• Visual Question Answering (visual-question-answering) - Model answers the question you submit in the prompt

• Captioning (short) (caption) - Model provides a short description of the image

• Captioning (detailed-caption) - Model provides a long description of the image

• Single-Label Classification (classification) - Model classifies the image content as one of the provided classes

• Multi-Label Classification (multi-label-classification) - Model classifies the image content as one or more of the provided classes

• Structured Output Generation (structured-answering) - Model returns a JSON response with the specified fields

Provide your OpenAI API key or set the value to rf_key:account (or rf_key:user:<id>) to proxy requests through Roboflow's API.

Type identifier

Use the following identifier in step "type" field: roboflow_core/open_ai@v3to add the block as as step in your workflow.

Properties

Name	Type	Description	Refs
name	str	Enter a unique identifier for this step..	❌
task_type	str	Task type to be performed by model. Value determines required parameters and output response..	❌
prompt	str	Text prompt to the OpenAI model.	✅
output_structure	Dict[str, str]	Dictionary with structure of expected JSON response.	❌
classes	List[str]	List of classes to be used.	✅
api_key	str	Your OpenAI API key.	✅
model_version	str	Model to be used.	✅
image_detail	str	Indicates the image's quality, with 'high' suggesting it is of high resolution and should be processed or displayed with high fidelity..	✅
max_tokens	int	Maximum number of tokens the model can generate in it's response..	❌
temperature	float	Temperature to sample from the model - value in range 0.0-2.0, the higher - the more random / "creative" the generations are..	✅
max_concurrent_requests	int	Number of concurrent requests that can be executed by block when batch of input images provided. If not given - block defaults to value configured globally in Workflows Execution Engine. Please restrict if you hit OpenAI limits..	❌

The Refs column marks possibility to parametrise the property with dynamic values available in workflow runtime. See Bindings for more info.

Available Connections

Compatible Blocks

Check what blocks you can connect to OpenAI in version v3.

• inputs: Grid Visualization, Image Blur, Image Preprocessing, Image Slicer, OpenAI, Dynamic Crop, Absolute Static Crop, Roboflow Dataset Upload, Color Visualization, LMM, Corner Visualization, Google Gemini, Depth Estimation, Stability AI Outpainting, Keypoint Visualization, Trace Visualization, Clip Comparison, Google Vision OCR, Keypoint Detection Model, Single-Label Classification Model, Email Notification, Model Comparison Visualization, Dimension Collapse, Mask Visualization, Image Slicer, Model Monitoring Inference Aggregator, Clip Comparison, Size Measurement, Multi-Label Classification Model, Buffer, Image Threshold, Contrast Equalization, OpenAI, Morphological Transformation, Classification Label Visualization, Relative Static Crop, Camera Calibration, Dynamic Zone, Florence-2 Model, Blur Visualization, Stitch Images, Roboflow Dataset Upload, Triangle Visualization, Perspective Correction, SIFT, Icon Visualization, Label Visualization, Stability AI Image Generation, Stitch OCR Detections, CogVLM, Ellipse Visualization, Llama 3.2 Vision, VLM as Detector, Line Counter Visualization, Florence-2 Model, SIFT Comparison, Local File Sink, Slack Notification, Image Convert Grayscale, Roboflow Custom Metadata, Gaze Detection, Twilio SMS Notification, Background Color Visualization, VLM as Classifier, QR Code Generator, Identify Changes, Polygon Zone Visualization, Anthropic Claude, Polygon Visualization, Camera Focus, Dot Visualization, LMM For Classification, Cosine Similarity, Instance Segmentation Model, Circle Visualization, Bounding Box Visualization, Image Contours, OpenAI, Object Detection Model, OCR Model, Halo Visualization, Reference Path Visualization, CSV Formatter, Pixelate Visualization, Webhook Sink, EasyOCR, Stability AI Inpainting, Crop Visualization
• outputs: Image Blur, Grid Visualization, OpenAI, Image Preprocessing, Instance Segmentation Model, Dynamic Crop, Time in Zone, Roboflow Dataset Upload, LMM, Moondream2, Color Visualization, Corner Visualization, Google Gemini, Stability AI Outpainting, Keypoint Visualization, Keypoint Detection Model, PTZ Tracking (ONVIF).md), Trace Visualization, Clip Comparison, Google Vision OCR, Email Notification, YOLO-World Model, Time in Zone, Model Comparison Visualization, Keypoint Detection Model, Mask Visualization, Model Monitoring Inference Aggregator, Clip Comparison, Size Measurement, Buffer, Detections Consensus, Image Threshold, Contrast Equalization, Line Counter, OpenAI, Path Deviation, Morphological Transformation, Classification Label Visualization, Time in Zone, Path Deviation, Florence-2 Model, Cache Set, Roboflow Dataset Upload, JSON Parser, Triangle Visualization, Perspective Correction, Icon Visualization, Label Visualization, Pixel Color Count, Stability AI Image Generation, Stitch OCR Detections, Object Detection Model, Llama 3.2 Vision, Ellipse Visualization, CogVLM, VLM as Detector, Line Counter Visualization, Florence-2 Model, Line Counter, Local File Sink, Distance Measurement, Slack Notification, SIFT Comparison, Roboflow Custom Metadata, Perception Encoder Embedding Model, Twilio SMS Notification, Background Color Visualization, VLM as Classifier, QR Code Generator, Segment Anything 2 Model, Cache Get, Polygon Zone Visualization, Anthropic Claude, VLM as Detector, Detections Stitch, Polygon Visualization, Dot Visualization, LMM For Classification, CLIP Embedding Model, Detections Classes Replacement, Instance Segmentation Model, Circle Visualization, Bounding Box Visualization, OpenAI, Object Detection Model, Halo Visualization, Reference Path Visualization, VLM as Classifier, Webhook Sink, Stability AI Inpainting, Crop Visualization

Input and Output Bindings

The available connections depend on its binding kinds. Check what binding kinds OpenAI in version v3 has.

Bindings

• input

  • images (image): The image to infer on..
  • prompt (string): Text prompt to the OpenAI model.
  • classes (list_of_values): List of classes to be used.
  • api_key (Union[secret, string, ROBOFLOW_MANAGED_KEY]): Your OpenAI API key.
  • model_version (string): Model to be used.
  • image_detail (string): Indicates the image's quality, with 'high' suggesting it is of high resolution and should be processed or displayed with high fidelity..
  • temperature (float): Temperature to sample from the model - value in range 0.0-2.0, the higher - the more random / "creative" the generations are..

• output

  • output (Union[string, language_model_output]): String value if string or LLM / VLM output if language_model_output.
  • classes (list_of_values): List of values of any type.

Example JSON definition of step OpenAI in version v3

{
    "name": "<your_step_name_here>",
    "type": "roboflow_core/open_ai@v3",
    "images": "$inputs.image",
    "task_type": "<block_does_not_provide_example>",
    "prompt": "my prompt",
    "output_structure": {
        "my_key": "description"
    },
    "classes": [
        "class-a",
        "class-b"
    ],
    "api_key": "xxx-xxx",
    "model_version": "gpt-5",
    "image_detail": "auto",
    "max_tokens": "<block_does_not_provide_example>",
    "temperature": "<block_does_not_provide_example>",
    "max_concurrent_requests": "<block_does_not_provide_example>"
}

v2

Class: OpenAIBlockV2 (there are multiple versions of this block)

Source: inference.core.workflows.core_steps.models.foundation.openai.v2.OpenAIBlockV2

Warning: This block has multiple versions. Please refer to the specific version for details. You can learn more about how versions work here: Versioning

Ask a question to OpenAI's GPT models with vision capabilities (including GPT-4o and GPT-5).

You can specify arbitrary text prompts or predefined ones, the block supports the following types of prompt:

• Open Prompt (unconstrained) - Use any prompt to generate a raw response

• Text Recognition (OCR) (ocr) - Model recognizes text in the image

• Visual Question Answering (visual-question-answering) - Model answers the question you submit in the prompt

• Captioning (short) (caption) - Model provides a short description of the image

• Captioning (detailed-caption) - Model provides a long description of the image

• Single-Label Classification (classification) - Model classifies the image content as one of the provided classes

• Multi-Label Classification (multi-label-classification) - Model classifies the image content as one or more of the provided classes

• Structured Output Generation (structured-answering) - Model returns a JSON response with the specified fields

You need to provide your OpenAI API key to use the GPT-4 with Vision model.

Type identifier

Use the following identifier in step "type" field: roboflow_core/open_ai@v2to add the block as as step in your workflow.

Properties

Name	Type	Description	Refs
name	str	Enter a unique identifier for this step..	❌
task_type	str	Task type to be performed by model. Value determines required parameters and output response..	❌
prompt	str	Text prompt to the OpenAI model.	✅
output_structure	Dict[str, str]	Dictionary with structure of expected JSON response.	❌
classes	List[str]	List of classes to be used.	✅
api_key	str	Your OpenAI API key.	✅
model_version	str	Model to be used.	✅
image_detail	str	Indicates the image's quality, with 'high' suggesting it is of high resolution and should be processed or displayed with high fidelity..	✅
max_tokens	int	Maximum number of tokens the model can generate in it's response..	❌
temperature	float	Temperature to sample from the model - value in range 0.0-2.0, the higher - the more random / "creative" the generations are..	✅
max_concurrent_requests	int	Number of concurrent requests that can be executed by block when batch of input images provided. If not given - block defaults to value configured globally in Workflows Execution Engine. Please restrict if you hit OpenAI limits..	❌

The Refs column marks possibility to parametrise the property with dynamic values available in workflow runtime. See Bindings for more info.

Available Connections

Compatible Blocks

Check what blocks you can connect to OpenAI in version v2.

• inputs: Grid Visualization, Image Blur, Image Preprocessing, Image Slicer, OpenAI, Dynamic Crop, Absolute Static Crop, Roboflow Dataset Upload, Color Visualization, LMM, Corner Visualization, Google Gemini, Depth Estimation, Stability AI Outpainting, Keypoint Visualization, Trace Visualization, Clip Comparison, Google Vision OCR, Keypoint Detection Model, Single-Label Classification Model, Email Notification, Model Comparison Visualization, Dimension Collapse, Mask Visualization, Image Slicer, Model Monitoring Inference Aggregator, Clip Comparison, Size Measurement, Multi-Label Classification Model, Buffer, Image Threshold, Contrast Equalization, OpenAI, Morphological Transformation, Classification Label Visualization, Relative Static Crop, Camera Calibration, Dynamic Zone, Florence-2 Model, Blur Visualization, Stitch Images, Roboflow Dataset Upload, Triangle Visualization, Perspective Correction, SIFT, Icon Visualization, Label Visualization, Stability AI Image Generation, Stitch OCR Detections, CogVLM, Ellipse Visualization, Llama 3.2 Vision, VLM as Detector, Line Counter Visualization, Florence-2 Model, SIFT Comparison, Local File Sink, Slack Notification, Image Convert Grayscale, Roboflow Custom Metadata, Gaze Detection, Twilio SMS Notification, Background Color Visualization, VLM as Classifier, QR Code Generator, Identify Changes, Polygon Zone Visualization, Anthropic Claude, Polygon Visualization, Camera Focus, Dot Visualization, LMM For Classification, Cosine Similarity, Instance Segmentation Model, Circle Visualization, Bounding Box Visualization, Image Contours, OpenAI, Object Detection Model, OCR Model, Halo Visualization, Reference Path Visualization, CSV Formatter, Pixelate Visualization, Webhook Sink, EasyOCR, Stability AI Inpainting, Crop Visualization
• outputs: Image Blur, Grid Visualization, OpenAI, Image Preprocessing, Instance Segmentation Model, Dynamic Crop, Time in Zone, Roboflow Dataset Upload, LMM, Moondream2, Color Visualization, Corner Visualization, Google Gemini, Stability AI Outpainting, Keypoint Visualization, Keypoint Detection Model, PTZ Tracking (ONVIF).md), Trace Visualization, Clip Comparison, Google Vision OCR, Email Notification, YOLO-World Model, Time in Zone, Model Comparison Visualization, Keypoint Detection Model, Mask Visualization, Model Monitoring Inference Aggregator, Clip Comparison, Size Measurement, Buffer, Detections Consensus, Image Threshold, Contrast Equalization, Line Counter, OpenAI, Path Deviation, Morphological Transformation, Classification Label Visualization, Time in Zone, Path Deviation, Florence-2 Model, Cache Set, Roboflow Dataset Upload, JSON Parser, Triangle Visualization, Perspective Correction, Icon Visualization, Label Visualization, Pixel Color Count, Stability AI Image Generation, Stitch OCR Detections, Object Detection Model, Llama 3.2 Vision, Ellipse Visualization, CogVLM, VLM as Detector, Line Counter Visualization, Florence-2 Model, Line Counter, Local File Sink, Distance Measurement, Slack Notification, SIFT Comparison, Roboflow Custom Metadata, Perception Encoder Embedding Model, Twilio SMS Notification, Background Color Visualization, VLM as Classifier, QR Code Generator, Segment Anything 2 Model, Cache Get, Polygon Zone Visualization, Anthropic Claude, VLM as Detector, Detections Stitch, Polygon Visualization, Dot Visualization, LMM For Classification, CLIP Embedding Model, Detections Classes Replacement, Instance Segmentation Model, Circle Visualization, Bounding Box Visualization, OpenAI, Object Detection Model, Halo Visualization, Reference Path Visualization, VLM as Classifier, Webhook Sink, Stability AI Inpainting, Crop Visualization

Input and Output Bindings

The available connections depend on its binding kinds. Check what binding kinds OpenAI in version v2 has.

Bindings

• input

  • images (image): The image to infer on..
  • prompt (string): Text prompt to the OpenAI model.
  • classes (list_of_values): List of classes to be used.
  • api_key (Union[secret, string]): Your OpenAI API key.
  • model_version (string): Model to be used.
  • image_detail (string): Indicates the image's quality, with 'high' suggesting it is of high resolution and should be processed or displayed with high fidelity..
  • temperature (float): Temperature to sample from the model - value in range 0.0-2.0, the higher - the more random / "creative" the generations are..

• output

  • output (Union[string, language_model_output]): String value if string or LLM / VLM output if language_model_output.
  • classes (list_of_values): List of values of any type.

Example JSON definition of step OpenAI in version v2

{
    "name": "<your_step_name_here>",
    "type": "roboflow_core/open_ai@v2",
    "images": "$inputs.image",
    "task_type": "<block_does_not_provide_example>",
    "prompt": "my prompt",
    "output_structure": {
        "my_key": "description"
    },
    "classes": [
        "class-a",
        "class-b"
    ],
    "api_key": "xxx-xxx",
    "model_version": "gpt-4o",
    "image_detail": "auto",
    "max_tokens": "<block_does_not_provide_example>",
    "temperature": "<block_does_not_provide_example>",
    "max_concurrent_requests": "<block_does_not_provide_example>"
}

v1

Class: OpenAIBlockV1 (there are multiple versions of this block)

Source: inference.core.workflows.core_steps.models.foundation.openai.v1.OpenAIBlockV1

Warning: This block has multiple versions. Please refer to the specific version for details. You can learn more about how versions work here: Versioning

Ask a question to OpenAI's GPT-4 with Vision model.

You can specify arbitrary text prompts to the OpenAIBlock.

You need to provide your OpenAI API key to use the GPT-4 with Vision model.

This model was previously part of the LMM block.

Type identifier

Use the following identifier in step "type" field: roboflow_core/open_ai@v1to add the block as as step in your workflow.

Properties

Name	Type	Description	Refs
name	str	Enter a unique identifier for this step..	❌
prompt	str	Text prompt to the OpenAI model.	✅
openai_api_key	str	Your OpenAI API key.	✅
openai_model	str	Model to be used.	✅
json_output_format	Dict[str, str]	Holds dictionary that maps name of requested output field into its description.	❌
image_detail	str	Indicates the image's quality, with 'high' suggesting it is of high resolution and should be processed or displayed with high fidelity..	✅
max_tokens	int	Maximum number of tokens the model can generate in it's response..	❌

The Refs column marks possibility to parametrise the property with dynamic values available in workflow runtime. See Bindings for more info.

Available Connections

Compatible Blocks

Check what blocks you can connect to OpenAI in version v1.

• inputs: Grid Visualization, Image Blur, Image Preprocessing, Image Slicer, OpenAI, Dynamic Crop, Absolute Static Crop, Roboflow Dataset Upload, Color Visualization, LMM, Corner Visualization, Google Gemini, Depth Estimation, Stability AI Outpainting, Keypoint Visualization, Trace Visualization, Clip Comparison, Google Vision OCR, Keypoint Detection Model, Single-Label Classification Model, Email Notification, Model Comparison Visualization, Mask Visualization, Image Slicer, Model Monitoring Inference Aggregator, Multi-Label Classification Model, Image Threshold, Contrast Equalization, OpenAI, Morphological Transformation, Classification Label Visualization, Relative Static Crop, Camera Calibration, Florence-2 Model, Blur Visualization, Stitch Images, Roboflow Dataset Upload, Triangle Visualization, Perspective Correction, SIFT, Icon Visualization, Label Visualization, Stability AI Image Generation, Stitch OCR Detections, CogVLM, Ellipse Visualization, Llama 3.2 Vision, VLM as Detector, Line Counter Visualization, Florence-2 Model, SIFT Comparison, Local File Sink, Slack Notification, Image Convert Grayscale, Roboflow Custom Metadata, Twilio SMS Notification, Background Color Visualization, VLM as Classifier, QR Code Generator, Polygon Zone Visualization, Anthropic Claude, Polygon Visualization, Camera Focus, Dot Visualization, LMM For Classification, Instance Segmentation Model, Circle Visualization, Bounding Box Visualization, Image Contours, OpenAI, Object Detection Model, OCR Model, Halo Visualization, Reference Path Visualization, CSV Formatter, Pixelate Visualization, Webhook Sink, EasyOCR, Stability AI Inpainting, Crop Visualization
• outputs: OpenAI, Image Preprocessing, Image Slicer, Dynamic Crop, Multi-Label Classification Model, Roboflow Dataset Upload, Moondream2, Corner Visualization, Google Gemini, Keypoint Detection Model, Depth Estimation, PTZ Tracking (ONVIF).md), Email Notification, Keypoint Detection Model, Time in Zone, Model Comparison Visualization, Single-Label Classification Model, Dimension Collapse, Mask Visualization, Model Monitoring Inference Aggregator, Property Definition, Line Counter, OpenAI, Detections Filter, Barcode Detection, Morphological Transformation, Classification Label Visualization, Time in Zone, Dynamic Zone, Florence-2 Model, Stitch Images, Cache Set, Triangle Visualization, Pixel Color Count, Stability AI Image Generation, Llama 3.2 Vision, Ellipse Visualization, CogVLM, SIFT Comparison, Detections Stabilizer, Florence-2 Model, Overlap Filter, Local File Sink, Distance Measurement, Byte Tracker, Detection Offset, Background Color Visualization, QR Code Generator, Segment Anything 2 Model, Anthropic Claude, VLM as Detector, Byte Tracker, Polygon Visualization, Camera Focus, Cosine Similarity, Detections Classes Replacement, Instance Segmentation Model, Identify Outliers, Image Contours, OpenAI, Object Detection Model, Dominant Color, Halo Visualization, VLM as Classifier, Detections Merge, EasyOCR, Stability AI Inpainting, Image Blur, Grid Visualization, Instance Segmentation Model, Time in Zone, Delta Filter, LMM, Absolute Static Crop, Color Visualization, Stability AI Outpainting, Keypoint Visualization, Trace Visualization, Clip Comparison, Google Vision OCR, First Non Empty Or Default, YOLO-World Model, Image Slicer, Clip Comparison, Size Measurement, Multi-Label Classification Model, Buffer, Detections Consensus, Image Threshold, Contrast Equalization, Path Deviation, Velocity, Relative Static Crop, Path Deviation, Camera Calibration, Blur Visualization, Roboflow Dataset Upload, JSON Parser, Qwen2.5-VL, Perspective Correction, SIFT, Icon Visualization, Label Visualization, Object Detection Model, Stitch OCR Detections, Detections Transformation, QR Code Detection, VLM as Detector, SmolVLM2, Byte Tracker, Single-Label Classification Model, Line Counter Visualization, Line Counter, Slack Notification, SIFT Comparison, Image Convert Grayscale, Detections Combine, Roboflow Custom Metadata, Gaze Detection, Perception Encoder Embedding Model, Twilio SMS Notification, Expression, VLM as Classifier, Cache Get, Identify Changes, Polygon Zone Visualization, Rate Limiter, Detections Stitch, Bounding Rectangle, Data Aggregator, Dot Visualization, LMM For Classification, Template Matching, CLIP Embedding Model, Circle Visualization, Bounding Box Visualization, OCR Model, Reference Path Visualization, CSV Formatter, Pixelate Visualization, Continue If, Webhook Sink, Crop Visualization

Input and Output Bindings

The available connections depend on its binding kinds. Check what binding kinds OpenAI in version v1 has.

Bindings

• input

  • images (image): The image to infer on..
  • prompt (string): Text prompt to the OpenAI model.
  • openai_api_key (Union[secret, string]): Your OpenAI API key.
  • openai_model (string): Model to be used.
  • image_detail (string): Indicates the image's quality, with 'high' suggesting it is of high resolution and should be processed or displayed with high fidelity..

• output

  • parent_id (parent_id): Identifier of parent for step output.
  • root_parent_id (parent_id): Identifier of parent for step output.
  • image (image_metadata): Dictionary with image metadata required by supervision.
  • structured_output (dictionary): Dictionary.
  • raw_output (string): String value.
  • * (*): Equivalent of any element.

Example JSON definition of step OpenAI in version v1

{
    "name": "<your_step_name_here>",
    "type": "roboflow_core/open_ai@v1",
    "images": "$inputs.image",
    "prompt": "my prompt",
    "openai_api_key": "xxx-xxx",
    "openai_model": "gpt-4o",
    "json_output_format": {
        "count": "number of cats in the picture"
    },
    "image_detail": "auto",
    "max_tokens": 450
}