OpenAI¶
v4¶
Class: OpenAIBlockV4 (there are multiple versions of this block)
Source: inference.core.workflows.core_steps.models.foundation.openai.v4.OpenAIBlockV4
Warning: This block has multiple versions. Please refer to the specific version for details. You can learn more about how versions work here: Versioning
Ask a question to OpenAI's GPT models with vision capabilities (including GPT-5 and GPT-4o).
You can specify arbitrary text prompts or predefined ones, the block supports the following types of prompt:
-
Open Prompt (
unconstrained) - Use any prompt to generate a raw response -
Text Recognition (OCR) (
ocr) - Model recognizes text in the image -
Visual Question Answering (
visual-question-answering) - Model answers the question you submit in the prompt -
Captioning (short) (
caption) - Model provides a short description of the image -
Captioning (
detailed-caption) - Model provides a long description of the image -
Single-Label Classification (
classification) - Model classifies the image content as one of the provided classes -
Multi-Label Classification (
multi-label-classification) - Model classifies the image content as one or more of the provided classes -
Unprompted Object Detection (
object-detection) - Model detects and returns the bounding boxes for prominent objects in the image -
Structured Output Generation (
structured-answering) - Model returns a JSON response with the specified fields
Provide your OpenAI API key or set the value to rf_key:account (or
rf_key:user:<id>) to proxy requests through Roboflow's API.
Type identifier¶
Use the following identifier in step "type" field: roboflow_core/open_ai@v4to add the block as
as step in your workflow.
Properties¶
| Name | Type | Description | Refs |
|---|---|---|---|
name |
str |
Enter a unique identifier for this step.. | ❌ |
task_type |
str |
Task type to be performed by model. Value determines required parameters and output response.. | ❌ |
prompt |
str |
Text prompt to the OpenAI model. | ✅ |
output_structure |
Dict[str, str] |
Dictionary with structure of expected JSON response. | ❌ |
classes |
List[str] |
List of classes to be used. | ✅ |
api_key |
str |
Your OpenAI API key. | ✅ |
model_version |
str |
Model to be used. | ✅ |
reasoning_effort |
str |
Controls reasoning. Reducing can result in faster responses and fewer tokens. GPT-5.1 and higher models default to 'none' (no reasoning) and support 'none', 'low', 'medium', 'high'. GPT-5.2 also supports 'xhigh'. GPT-5 models default to 'medium' and support 'minimal', 'low', 'medium', 'high'.. | ✅ |
image_detail |
str |
Indicates the image's quality, with 'high' suggesting it is of high resolution and should be processed or displayed with high fidelity.. | ✅ |
max_tokens |
int |
Maximum number of tokens the model can generate in its response. If not specified, the model will use its default limit. Minimum value is 16.. | ❌ |
temperature |
float |
Temperature to sample from the model - value in range 0.0-2.0, the higher - the more random / "creative" the generations are.. | ✅ |
max_concurrent_requests |
int |
Number of concurrent requests that can be executed by block when batch of input images provided. If not given - block defaults to value configured globally in Workflows Execution Engine. Please restrict if you hit OpenAI limits.. | ❌ |
The Refs column marks possibility to parametrise the property with dynamic values available
in workflow runtime. See Bindings for more info.
Available Connections¶
Compatible Blocks
Check what blocks you can connect to OpenAI in version v4.
- inputs:
Anthropic Claude,Instance Segmentation Model,Google Vision OCR,Circle Visualization,Image Slicer,Google Gemini,Image Contours,Qwen 3.6 API,Single-Label Classification Model,Roboflow Vision Events,Depth Estimation,Line Counter Visualization,Stitch Images,Morphological Transformation,LMM,Model Comparison Visualization,Buffer,MoonshotAI Kimi,Grid Visualization,Twilio SMS/MMS Notification,OpenAI,Clip Comparison,Twilio SMS Notification,Qwen-VL,S3 Sink,Halo Visualization,Camera Focus,SIFT Comparison,Keypoint Detection Model,Local File Sink,Mask Visualization,SIFT,Anthropic Claude,MoonshotAI Kimi,Roboflow Dataset Upload,Text Display,Image Slicer,Multi-Label Classification Model,Absolute Static Crop,Llama 3.2 Vision,GLM-OCR,Object Detection Model,Roboflow Custom Metadata,Email Notification,Dynamic Crop,OpenRouter,Cosine Similarity,Model Monitoring Inference Aggregator,Contrast Enhancement,Motion Detection,Webhook Sink,Google Gemma API,Stability AI Image Generation,Color Visualization,Heatmap Visualization,Contrast Equalization,Google Gemini,Roboflow Dataset Upload,Slack Notification,Anthropic Claude,QR Code Generator,Clip Comparison,OpenAI,Bounding Box Visualization,Florence-2 Model,VLM As Classifier,Image Blur,Dot Visualization,Polygon Zone Visualization,Label Visualization,Icon Visualization,Image Threshold,LMM For Classification,Blur Visualization,Relative Static Crop,Dimension Collapse,Trace Visualization,Qwen3.5-VL,Size Measurement,Dynamic Zone,Florence-2 Model,Camera Focus,CogVLM,Pixelate Visualization,Llama 3.2 Vision,Image Convert Grayscale,Keypoint Visualization,Polygon Visualization,Google Gemma,Classification Label Visualization,Image Stack,Morphological Transformation,Gaze Detection,Camera Calibration,Google Gemini,Email Notification,Image Preprocessing,Corner Visualization,Stitch OCR Detections,Halo Visualization,Roboflow Asset Library Attributes,Reference Path Visualization,OpenAI,Background Color Visualization,Identify Changes,Ellipse Visualization,CSV Formatter,Stability AI Outpainting,VLM As Detector,EasyOCR,Triangle Visualization,OCR Model,Crop Visualization,Perspective Correction,Qwen 3.5 API,Stitch OCR Detections,OpenAI-Compatible LLM,Detections List Roll-Up,Polygon Visualization,Background Subtraction,OpenAI,Stability AI Inpainting - outputs:
Distance Measurement,Keypoint Detection Model,Instance Segmentation Model,Anthropic Claude,Google Vision OCR,Circle Visualization,Google Gemini,Qwen 3.6 API,CLIP Embedding Model,Roboflow Vision Events,Depth Estimation,Line Counter Visualization,VLM As Detector,Morphological Transformation,LMM,Model Comparison Visualization,Buffer,MoonshotAI Kimi,Segment Anything 2 Model,Grid Visualization,Cache Set,Instance Segmentation Model,Twilio SMS/MMS Notification,OpenAI,Detections Classes Replacement,Clip Comparison,Twilio SMS Notification,SAM 3,Qwen-VL,S3 Sink,SAM 3,Halo Visualization,Local File Sink,Semantic Segmentation Model,SIFT Comparison,Multi-Label Classification Model,Keypoint Detection Model,Mask Visualization,Path Deviation,Anthropic Claude,MoonshotAI Kimi,Roboflow Dataset Upload,Text Display,Llama 3.2 Vision,VLM As Classifier,PTZ Tracking (ONVIF),Path Deviation,GLM-OCR,Object Detection Model,Email Notification,Seg Preview,Dynamic Crop,Roboflow Custom Metadata,Instance Segmentation Model,Time in Zone,OpenRouter,Model Monitoring Inference Aggregator,Motion Detection,Webhook Sink,Google Gemma API,Stability AI Image Generation,Color Visualization,Heatmap Visualization,Contrast Equalization,Object Detection Model,YOLO-World Model,Google Gemini,Roboflow Dataset Upload,Slack Notification,Anthropic Claude,QR Code Generator,Clip Comparison,OpenAI,Bounding Box Visualization,SAM 3,Florence-2 Model,VLM As Classifier,Image Blur,Keypoint Detection Model,Detections Consensus,Dot Visualization,Polygon Zone Visualization,Label Visualization,Icon Visualization,Image Threshold,Cache Get,LMM For Classification,Object Detection Model,Line Counter,Trace Visualization,Moondream2,Size Measurement,Perception Encoder Embedding Model,Florence-2 Model,CogVLM,Llama 3.2 Vision,Time in Zone,Keypoint Visualization,Polygon Visualization,Line Counter,Google Gemma,Classification Label Visualization,Morphological Transformation,Email Notification,Google Gemini,Image Preprocessing,Corner Visualization,Stitch OCR Detections,Halo Visualization,Roboflow Asset Library Attributes,OpenAI,Reference Path Visualization,Detections Stitch,Background Color Visualization,Single-Label Classification Model,JSON Parser,Ellipse Visualization,Stability AI Outpainting,VLM As Detector,Triangle Visualization,Crop Visualization,Qwen 3.5 API,Perspective Correction,Stitch OCR Detections,OpenAI-Compatible LLM,Detections List Roll-Up,Instance Segmentation Model,Time in Zone,Polygon Visualization,Pixel Color Count,OpenAI,Stability AI Inpainting
Input and Output Bindings¶
The available connections depend on its binding kinds. Check what binding kinds
OpenAI in version v4 has.
Bindings
-
input
images(image): The image to infer on..prompt(string): Text prompt to the OpenAI model.classes(list_of_values): List of classes to be used.api_key(Union[secret,ROBOFLOW_MANAGED_KEY,string]): Your OpenAI API key.model_version(string): Model to be used.reasoning_effort(string): Controls reasoning. Reducing can result in faster responses and fewer tokens. GPT-5.1 and higher models default to 'none' (no reasoning) and support 'none', 'low', 'medium', 'high'. GPT-5.2 also supports 'xhigh'. GPT-5 models default to 'medium' and support 'minimal', 'low', 'medium', 'high'..image_detail(string): Indicates the image's quality, with 'high' suggesting it is of high resolution and should be processed or displayed with high fidelity..temperature(float): Temperature to sample from the model - value in range 0.0-2.0, the higher - the more random / "creative" the generations are..
-
output
output(Union[string,language_model_output]): String value ifstringor LLM / VLM output iflanguage_model_output.classes(list_of_values): List of values of any type.
Example JSON definition of step OpenAI in version v4
{
"name": "<your_step_name_here>",
"type": "roboflow_core/open_ai@v4",
"images": "$inputs.image",
"task_type": "<block_does_not_provide_example>",
"prompt": "my prompt",
"output_structure": {
"my_key": "description"
},
"classes": [
"class-a",
"class-b"
],
"api_key": "xxx-xxx",
"model_version": "gpt-5.1",
"reasoning_effort": "<block_does_not_provide_example>",
"image_detail": "auto",
"max_tokens": "<block_does_not_provide_example>",
"temperature": "<block_does_not_provide_example>",
"max_concurrent_requests": "<block_does_not_provide_example>"
}
v3¶
Class: OpenAIBlockV3 (there are multiple versions of this block)
Source: inference.core.workflows.core_steps.models.foundation.openai.v3.OpenAIBlockV3
Warning: This block has multiple versions. Please refer to the specific version for details. You can learn more about how versions work here: Versioning
Ask a question to OpenAI's GPT models with vision capabilities (including GPT-5 and GPT-4o).
You can specify arbitrary text prompts or predefined ones, the block supports the following types of prompt:
-
Open Prompt (
unconstrained) - Use any prompt to generate a raw response -
Text Recognition (OCR) (
ocr) - Model recognizes text in the image -
Visual Question Answering (
visual-question-answering) - Model answers the question you submit in the prompt -
Captioning (short) (
caption) - Model provides a short description of the image -
Captioning (
detailed-caption) - Model provides a long description of the image -
Single-Label Classification (
classification) - Model classifies the image content as one of the provided classes -
Multi-Label Classification (
multi-label-classification) - Model classifies the image content as one or more of the provided classes -
Structured Output Generation (
structured-answering) - Model returns a JSON response with the specified fields
Provide your OpenAI API key or set the value to rf_key:account (or
rf_key:user:<id>) to proxy requests through Roboflow's API.
Type identifier¶
Use the following identifier in step "type" field: roboflow_core/open_ai@v3to add the block as
as step in your workflow.
Properties¶
| Name | Type | Description | Refs |
|---|---|---|---|
name |
str |
Enter a unique identifier for this step.. | ❌ |
task_type |
str |
Task type to be performed by model. Value determines required parameters and output response.. | ❌ |
prompt |
str |
Text prompt to the OpenAI model. | ✅ |
output_structure |
Dict[str, str] |
Dictionary with structure of expected JSON response. | ❌ |
classes |
List[str] |
List of classes to be used. | ✅ |
api_key |
str |
Your OpenAI API key. | ✅ |
model_version |
str |
Model to be used. | ✅ |
image_detail |
str |
Indicates the image's quality, with 'high' suggesting it is of high resolution and should be processed or displayed with high fidelity.. | ✅ |
max_tokens |
int |
Maximum number of tokens the model can generate in it's response.. | ❌ |
temperature |
float |
Temperature to sample from the model - value in range 0.0-2.0, the higher - the more random / "creative" the generations are.. | ✅ |
max_concurrent_requests |
int |
Number of concurrent requests that can be executed by block when batch of input images provided. If not given - block defaults to value configured globally in Workflows Execution Engine. Please restrict if you hit OpenAI limits.. | ❌ |
The Refs column marks possibility to parametrise the property with dynamic values available
in workflow runtime. See Bindings for more info.
Available Connections¶
Compatible Blocks
Check what blocks you can connect to OpenAI in version v3.
- inputs:
Anthropic Claude,Instance Segmentation Model,Google Vision OCR,Circle Visualization,Image Slicer,Google Gemini,Image Contours,Qwen 3.6 API,Single-Label Classification Model,Roboflow Vision Events,Depth Estimation,Line Counter Visualization,Stitch Images,Morphological Transformation,LMM,Model Comparison Visualization,Buffer,MoonshotAI Kimi,Grid Visualization,Twilio SMS/MMS Notification,OpenAI,Clip Comparison,Twilio SMS Notification,Qwen-VL,S3 Sink,Halo Visualization,Camera Focus,SIFT Comparison,Keypoint Detection Model,Local File Sink,Mask Visualization,SIFT,Anthropic Claude,MoonshotAI Kimi,Roboflow Dataset Upload,Text Display,Image Slicer,Multi-Label Classification Model,Absolute Static Crop,Llama 3.2 Vision,GLM-OCR,Object Detection Model,Roboflow Custom Metadata,Email Notification,Dynamic Crop,OpenRouter,Cosine Similarity,Model Monitoring Inference Aggregator,Contrast Enhancement,Motion Detection,Webhook Sink,Google Gemma API,Stability AI Image Generation,Color Visualization,Heatmap Visualization,Contrast Equalization,Google Gemini,Roboflow Dataset Upload,Slack Notification,Anthropic Claude,QR Code Generator,Clip Comparison,OpenAI,Bounding Box Visualization,Florence-2 Model,VLM As Classifier,Image Blur,Dot Visualization,Polygon Zone Visualization,Label Visualization,Icon Visualization,Image Threshold,LMM For Classification,Blur Visualization,Relative Static Crop,Dimension Collapse,Trace Visualization,Qwen3.5-VL,Size Measurement,Dynamic Zone,Florence-2 Model,Camera Focus,CogVLM,Pixelate Visualization,Llama 3.2 Vision,Image Convert Grayscale,Keypoint Visualization,Polygon Visualization,Google Gemma,Classification Label Visualization,Image Stack,Morphological Transformation,Gaze Detection,Camera Calibration,Google Gemini,Email Notification,Image Preprocessing,Corner Visualization,Stitch OCR Detections,Halo Visualization,Roboflow Asset Library Attributes,Reference Path Visualization,OpenAI,Background Color Visualization,Identify Changes,Ellipse Visualization,CSV Formatter,Stability AI Outpainting,VLM As Detector,EasyOCR,Triangle Visualization,OCR Model,Crop Visualization,Perspective Correction,Qwen 3.5 API,Stitch OCR Detections,OpenAI-Compatible LLM,Detections List Roll-Up,Polygon Visualization,Background Subtraction,OpenAI,Stability AI Inpainting - outputs:
Distance Measurement,Keypoint Detection Model,Instance Segmentation Model,Anthropic Claude,Google Vision OCR,Circle Visualization,Google Gemini,Qwen 3.6 API,CLIP Embedding Model,Roboflow Vision Events,Depth Estimation,Line Counter Visualization,VLM As Detector,Morphological Transformation,LMM,Model Comparison Visualization,Buffer,MoonshotAI Kimi,Segment Anything 2 Model,Grid Visualization,Cache Set,Instance Segmentation Model,Twilio SMS/MMS Notification,OpenAI,Detections Classes Replacement,Clip Comparison,Twilio SMS Notification,SAM 3,Qwen-VL,S3 Sink,SAM 3,Halo Visualization,Local File Sink,Semantic Segmentation Model,SIFT Comparison,Multi-Label Classification Model,Keypoint Detection Model,Mask Visualization,Path Deviation,Anthropic Claude,MoonshotAI Kimi,Roboflow Dataset Upload,Text Display,Llama 3.2 Vision,VLM As Classifier,PTZ Tracking (ONVIF),Path Deviation,GLM-OCR,Object Detection Model,Email Notification,Seg Preview,Dynamic Crop,Roboflow Custom Metadata,Instance Segmentation Model,Time in Zone,OpenRouter,Model Monitoring Inference Aggregator,Motion Detection,Webhook Sink,Google Gemma API,Stability AI Image Generation,Color Visualization,Heatmap Visualization,Contrast Equalization,Object Detection Model,YOLO-World Model,Google Gemini,Roboflow Dataset Upload,Slack Notification,Anthropic Claude,QR Code Generator,Clip Comparison,OpenAI,Bounding Box Visualization,SAM 3,Florence-2 Model,VLM As Classifier,Image Blur,Keypoint Detection Model,Detections Consensus,Dot Visualization,Polygon Zone Visualization,Label Visualization,Icon Visualization,Image Threshold,Cache Get,LMM For Classification,Object Detection Model,Line Counter,Trace Visualization,Moondream2,Size Measurement,Perception Encoder Embedding Model,Florence-2 Model,CogVLM,Llama 3.2 Vision,Time in Zone,Keypoint Visualization,Polygon Visualization,Line Counter,Google Gemma,Classification Label Visualization,Morphological Transformation,Email Notification,Google Gemini,Image Preprocessing,Corner Visualization,Stitch OCR Detections,Halo Visualization,Roboflow Asset Library Attributes,OpenAI,Reference Path Visualization,Detections Stitch,Background Color Visualization,Single-Label Classification Model,JSON Parser,Ellipse Visualization,Stability AI Outpainting,VLM As Detector,Triangle Visualization,Crop Visualization,Qwen 3.5 API,Perspective Correction,Stitch OCR Detections,OpenAI-Compatible LLM,Detections List Roll-Up,Instance Segmentation Model,Time in Zone,Polygon Visualization,Pixel Color Count,OpenAI,Stability AI Inpainting
Input and Output Bindings¶
The available connections depend on its binding kinds. Check what binding kinds
OpenAI in version v3 has.
Bindings
-
input
images(image): The image to infer on..prompt(string): Text prompt to the OpenAI model.classes(list_of_values): List of classes to be used.api_key(Union[secret,ROBOFLOW_MANAGED_KEY,string]): Your OpenAI API key.model_version(string): Model to be used.image_detail(string): Indicates the image's quality, with 'high' suggesting it is of high resolution and should be processed or displayed with high fidelity..temperature(float): Temperature to sample from the model - value in range 0.0-2.0, the higher - the more random / "creative" the generations are..
-
output
output(Union[string,language_model_output]): String value ifstringor LLM / VLM output iflanguage_model_output.classes(list_of_values): List of values of any type.
Example JSON definition of step OpenAI in version v3
{
"name": "<your_step_name_here>",
"type": "roboflow_core/open_ai@v3",
"images": "$inputs.image",
"task_type": "<block_does_not_provide_example>",
"prompt": "my prompt",
"output_structure": {
"my_key": "description"
},
"classes": [
"class-a",
"class-b"
],
"api_key": "xxx-xxx",
"model_version": "gpt-5",
"image_detail": "auto",
"max_tokens": "<block_does_not_provide_example>",
"temperature": "<block_does_not_provide_example>",
"max_concurrent_requests": "<block_does_not_provide_example>"
}
v2¶
Class: OpenAIBlockV2 (there are multiple versions of this block)
Source: inference.core.workflows.core_steps.models.foundation.openai.v2.OpenAIBlockV2
Warning: This block has multiple versions. Please refer to the specific version for details. You can learn more about how versions work here: Versioning
Ask a question to OpenAI's GPT models with vision capabilities (including GPT-4o and GPT-5).
You can specify arbitrary text prompts or predefined ones, the block supports the following types of prompt:
-
Open Prompt (
unconstrained) - Use any prompt to generate a raw response -
Text Recognition (OCR) (
ocr) - Model recognizes text in the image -
Visual Question Answering (
visual-question-answering) - Model answers the question you submit in the prompt -
Captioning (short) (
caption) - Model provides a short description of the image -
Captioning (
detailed-caption) - Model provides a long description of the image -
Single-Label Classification (
classification) - Model classifies the image content as one of the provided classes -
Multi-Label Classification (
multi-label-classification) - Model classifies the image content as one or more of the provided classes -
Structured Output Generation (
structured-answering) - Model returns a JSON response with the specified fields
You need to provide your OpenAI API key to use the GPT-4 with Vision model.
Type identifier¶
Use the following identifier in step "type" field: roboflow_core/open_ai@v2to add the block as
as step in your workflow.
Properties¶
| Name | Type | Description | Refs |
|---|---|---|---|
name |
str |
Enter a unique identifier for this step.. | ❌ |
task_type |
str |
Task type to be performed by model. Value determines required parameters and output response.. | ❌ |
prompt |
str |
Text prompt to the OpenAI model. | ✅ |
output_structure |
Dict[str, str] |
Dictionary with structure of expected JSON response. | ❌ |
classes |
List[str] |
List of classes to be used. | ✅ |
api_key |
str |
Your OpenAI API key. | ✅ |
model_version |
str |
Model to be used. | ✅ |
image_detail |
str |
Indicates the image's quality, with 'high' suggesting it is of high resolution and should be processed or displayed with high fidelity.. | ✅ |
max_tokens |
int |
Maximum number of tokens the model can generate in it's response.. | ❌ |
temperature |
float |
Temperature to sample from the model - value in range 0.0-2.0, the higher - the more random / "creative" the generations are.. | ✅ |
max_concurrent_requests |
int |
Number of concurrent requests that can be executed by block when batch of input images provided. If not given - block defaults to value configured globally in Workflows Execution Engine. Please restrict if you hit OpenAI limits.. | ❌ |
The Refs column marks possibility to parametrise the property with dynamic values available
in workflow runtime. See Bindings for more info.
Available Connections¶
Compatible Blocks
Check what blocks you can connect to OpenAI in version v2.
- inputs:
Anthropic Claude,Instance Segmentation Model,Google Vision OCR,Circle Visualization,Image Slicer,Google Gemini,Image Contours,Qwen 3.6 API,Single-Label Classification Model,Roboflow Vision Events,Depth Estimation,Line Counter Visualization,Stitch Images,Morphological Transformation,LMM,Model Comparison Visualization,Buffer,MoonshotAI Kimi,Grid Visualization,Twilio SMS/MMS Notification,OpenAI,Clip Comparison,Twilio SMS Notification,Qwen-VL,S3 Sink,Halo Visualization,Camera Focus,SIFT Comparison,Keypoint Detection Model,Local File Sink,Mask Visualization,SIFT,Anthropic Claude,MoonshotAI Kimi,Roboflow Dataset Upload,Text Display,Image Slicer,Multi-Label Classification Model,Absolute Static Crop,Llama 3.2 Vision,GLM-OCR,Object Detection Model,Roboflow Custom Metadata,Email Notification,Dynamic Crop,OpenRouter,Cosine Similarity,Model Monitoring Inference Aggregator,Contrast Enhancement,Motion Detection,Webhook Sink,Google Gemma API,Stability AI Image Generation,Color Visualization,Heatmap Visualization,Contrast Equalization,Google Gemini,Roboflow Dataset Upload,Slack Notification,Anthropic Claude,QR Code Generator,Clip Comparison,OpenAI,Bounding Box Visualization,Florence-2 Model,VLM As Classifier,Image Blur,Dot Visualization,Polygon Zone Visualization,Label Visualization,Icon Visualization,Image Threshold,LMM For Classification,Blur Visualization,Relative Static Crop,Dimension Collapse,Trace Visualization,Qwen3.5-VL,Size Measurement,Dynamic Zone,Florence-2 Model,Camera Focus,CogVLM,Pixelate Visualization,Llama 3.2 Vision,Image Convert Grayscale,Keypoint Visualization,Polygon Visualization,Google Gemma,Classification Label Visualization,Image Stack,Morphological Transformation,Gaze Detection,Camera Calibration,Google Gemini,Email Notification,Image Preprocessing,Corner Visualization,Stitch OCR Detections,Halo Visualization,Roboflow Asset Library Attributes,Reference Path Visualization,OpenAI,Background Color Visualization,Identify Changes,Ellipse Visualization,CSV Formatter,Stability AI Outpainting,VLM As Detector,EasyOCR,Triangle Visualization,OCR Model,Crop Visualization,Perspective Correction,Qwen 3.5 API,Stitch OCR Detections,OpenAI-Compatible LLM,Detections List Roll-Up,Polygon Visualization,Background Subtraction,OpenAI,Stability AI Inpainting - outputs:
Distance Measurement,Keypoint Detection Model,Instance Segmentation Model,Anthropic Claude,Google Vision OCR,Circle Visualization,Google Gemini,Qwen 3.6 API,CLIP Embedding Model,Roboflow Vision Events,Depth Estimation,Line Counter Visualization,VLM As Detector,Morphological Transformation,LMM,Model Comparison Visualization,Buffer,MoonshotAI Kimi,Segment Anything 2 Model,Grid Visualization,Cache Set,Instance Segmentation Model,Twilio SMS/MMS Notification,OpenAI,Detections Classes Replacement,Clip Comparison,Twilio SMS Notification,SAM 3,Qwen-VL,S3 Sink,SAM 3,Halo Visualization,Local File Sink,Semantic Segmentation Model,SIFT Comparison,Multi-Label Classification Model,Keypoint Detection Model,Mask Visualization,Path Deviation,Anthropic Claude,MoonshotAI Kimi,Roboflow Dataset Upload,Text Display,Llama 3.2 Vision,VLM As Classifier,PTZ Tracking (ONVIF),Path Deviation,GLM-OCR,Object Detection Model,Email Notification,Seg Preview,Dynamic Crop,Roboflow Custom Metadata,Instance Segmentation Model,Time in Zone,OpenRouter,Model Monitoring Inference Aggregator,Motion Detection,Webhook Sink,Google Gemma API,Stability AI Image Generation,Color Visualization,Heatmap Visualization,Contrast Equalization,Object Detection Model,YOLO-World Model,Google Gemini,Roboflow Dataset Upload,Slack Notification,Anthropic Claude,QR Code Generator,Clip Comparison,OpenAI,Bounding Box Visualization,SAM 3,Florence-2 Model,VLM As Classifier,Image Blur,Keypoint Detection Model,Detections Consensus,Dot Visualization,Polygon Zone Visualization,Label Visualization,Icon Visualization,Image Threshold,Cache Get,LMM For Classification,Object Detection Model,Line Counter,Trace Visualization,Moondream2,Size Measurement,Perception Encoder Embedding Model,Florence-2 Model,CogVLM,Llama 3.2 Vision,Time in Zone,Keypoint Visualization,Polygon Visualization,Line Counter,Google Gemma,Classification Label Visualization,Morphological Transformation,Email Notification,Google Gemini,Image Preprocessing,Corner Visualization,Stitch OCR Detections,Halo Visualization,Roboflow Asset Library Attributes,OpenAI,Reference Path Visualization,Detections Stitch,Background Color Visualization,Single-Label Classification Model,JSON Parser,Ellipse Visualization,Stability AI Outpainting,VLM As Detector,Triangle Visualization,Crop Visualization,Qwen 3.5 API,Perspective Correction,Stitch OCR Detections,OpenAI-Compatible LLM,Detections List Roll-Up,Instance Segmentation Model,Time in Zone,Polygon Visualization,Pixel Color Count,OpenAI,Stability AI Inpainting
Input and Output Bindings¶
The available connections depend on its binding kinds. Check what binding kinds
OpenAI in version v2 has.
Bindings
-
input
images(image): The image to infer on..prompt(string): Text prompt to the OpenAI model.classes(list_of_values): List of classes to be used.api_key(Union[secret,string]): Your OpenAI API key.model_version(string): Model to be used.image_detail(string): Indicates the image's quality, with 'high' suggesting it is of high resolution and should be processed or displayed with high fidelity..temperature(float): Temperature to sample from the model - value in range 0.0-2.0, the higher - the more random / "creative" the generations are..
-
output
output(Union[string,language_model_output]): String value ifstringor LLM / VLM output iflanguage_model_output.classes(list_of_values): List of values of any type.
Example JSON definition of step OpenAI in version v2
{
"name": "<your_step_name_here>",
"type": "roboflow_core/open_ai@v2",
"images": "$inputs.image",
"task_type": "<block_does_not_provide_example>",
"prompt": "my prompt",
"output_structure": {
"my_key": "description"
},
"classes": [
"class-a",
"class-b"
],
"api_key": "xxx-xxx",
"model_version": "gpt-4o",
"image_detail": "auto",
"max_tokens": "<block_does_not_provide_example>",
"temperature": "<block_does_not_provide_example>",
"max_concurrent_requests": "<block_does_not_provide_example>"
}
v1¶
Class: OpenAIBlockV1 (there are multiple versions of this block)
Source: inference.core.workflows.core_steps.models.foundation.openai.v1.OpenAIBlockV1
Warning: This block has multiple versions. Please refer to the specific version for details. You can learn more about how versions work here: Versioning
Ask a question to OpenAI's GPT-4 with Vision model.
You can specify arbitrary text prompts to the OpenAIBlock.
You need to provide your OpenAI API key to use the GPT-4 with Vision model.
This model was previously part of the LMM block.
Type identifier¶
Use the following identifier in step "type" field: roboflow_core/open_ai@v1to add the block as
as step in your workflow.
Properties¶
| Name | Type | Description | Refs |
|---|---|---|---|
name |
str |
Enter a unique identifier for this step.. | ❌ |
prompt |
str |
Text prompt to the OpenAI model. | ✅ |
openai_api_key |
str |
Your OpenAI API key. | ✅ |
openai_model |
str |
Model to be used. | ✅ |
json_output_format |
Dict[str, str] |
Holds dictionary that maps name of requested output field into its description. | ❌ |
image_detail |
str |
Indicates the image's quality, with 'high' suggesting it is of high resolution and should be processed or displayed with high fidelity.. | ✅ |
max_tokens |
int |
Maximum number of tokens the model can generate in it's response.. | ❌ |
The Refs column marks possibility to parametrise the property with dynamic values available
in workflow runtime. See Bindings for more info.
Available Connections¶
Compatible Blocks
Check what blocks you can connect to OpenAI in version v1.
- inputs:
Anthropic Claude,Instance Segmentation Model,Google Vision OCR,Circle Visualization,Image Slicer,Google Gemini,Image Contours,Qwen 3.6 API,Single-Label Classification Model,Roboflow Vision Events,Depth Estimation,Line Counter Visualization,Stitch Images,Morphological Transformation,LMM,Model Comparison Visualization,MoonshotAI Kimi,Grid Visualization,Twilio SMS/MMS Notification,OpenAI,Twilio SMS Notification,Qwen-VL,S3 Sink,Halo Visualization,Camera Focus,SIFT Comparison,Keypoint Detection Model,Local File Sink,Mask Visualization,SIFT,Anthropic Claude,MoonshotAI Kimi,Roboflow Dataset Upload,Text Display,Image Slicer,Multi-Label Classification Model,Absolute Static Crop,Llama 3.2 Vision,GLM-OCR,Object Detection Model,Roboflow Custom Metadata,Email Notification,Dynamic Crop,OpenRouter,Model Monitoring Inference Aggregator,Contrast Enhancement,Webhook Sink,Google Gemma API,Stability AI Image Generation,Color Visualization,Heatmap Visualization,Contrast Equalization,Google Gemini,Roboflow Dataset Upload,Slack Notification,Anthropic Claude,QR Code Generator,Clip Comparison,OpenAI,Bounding Box Visualization,Florence-2 Model,VLM As Classifier,Image Blur,Dot Visualization,Polygon Zone Visualization,Label Visualization,Icon Visualization,Image Threshold,LMM For Classification,Blur Visualization,Relative Static Crop,Trace Visualization,Qwen3.5-VL,Florence-2 Model,Camera Focus,CogVLM,Pixelate Visualization,Llama 3.2 Vision,Image Convert Grayscale,Keypoint Visualization,Polygon Visualization,Google Gemma,Classification Label Visualization,Morphological Transformation,Camera Calibration,Google Gemini,Email Notification,Image Preprocessing,Corner Visualization,Stitch OCR Detections,Halo Visualization,Roboflow Asset Library Attributes,Reference Path Visualization,OpenAI,Background Color Visualization,Ellipse Visualization,CSV Formatter,Stability AI Outpainting,VLM As Detector,EasyOCR,Triangle Visualization,OCR Model,Crop Visualization,Perspective Correction,Qwen 3.5 API,Stitch OCR Detections,OpenAI-Compatible LLM,Polygon Visualization,Background Subtraction,OpenAI,Stability AI Inpainting - outputs:
Detections Stabilizer,Keypoint Detection Model,Velocity,Instance Segmentation Model,Anthropic Claude,SIFT Comparison,Google Vision OCR,Circle Visualization,Detections Filter,Google Gemini,Detections Merge,Byte Tracker,Roboflow Vision Events,Line Counter Visualization,VLM As Detector,Morphological Transformation,LMM,Model Comparison Visualization,Buffer,MoonshotAI Kimi,Segment Anything 2 Model,Cache Set,Instance Segmentation Model,Twilio SMS/MMS Notification,Detections Classes Replacement,Twilio SMS Notification,SAM 3,S3 Sink,Halo Visualization,Local File Sink,Semantic Segmentation Model,SIFT Comparison,Camera Focus,Mask Visualization,Path Deviation,MoonshotAI Kimi,Text Display,Image Slicer,Absolute Static Crop,Inner Workflow,VLM As Classifier,Path Deviation,GLM-OCR,Roboflow Custom Metadata,Detections Combine,Dynamic Crop,Mask Area Measurement,Cosine Similarity,Model Monitoring Inference Aggregator,Contrast Enhancement,Motion Detection,Webhook Sink,SAM2 Video Tracker,Color Visualization,Object Detection Model,YOLO-World Model,Google Gemini,Single-Label Classification Model,Anthropic Claude,QR Code Generator,Clip Comparison,Qwen3.5,Bounding Box Visualization,Florence-2 Model,VLM As Classifier,Overlap Filter,Image Threshold,Cache Get,Line Counter,Relative Static Crop,Qwen3.5-VL,Moondream2,Perception Encoder Embedding Model,Llama 3.2 Vision,Time in Zone,Image Convert Grayscale,Keypoint Visualization,Byte Tracker,Google Gemma,Image Stack,Morphological Transformation,Email Notification,Rate Limiter,Corner Visualization,Stitch OCR Detections,Detections Transformation,OpenAI,Background Color Visualization,Identify Changes,Single-Label Classification Model,JSON Parser,Ellipse Visualization,Stability AI Outpainting,Data Aggregator,EasyOCR,OCR Model,Perspective Correction,Stitch OCR Detections,Instance Segmentation Model,Time in Zone,Background Subtraction,Template Matching,Pixel Color Count,Stability AI Inpainting,Distance Measurement,Barcode Detection,Mask Edge Snap,Image Slicer,Identify Outliers,Image Contours,Qwen 3.6 API,Single-Label Classification Model,CLIP Embedding Model,Depth Estimation,Stitch Images,Grid Visualization,OpenAI,Clip Comparison,Continue If,Dominant Color,Qwen-VL,SAM 3,Keypoint Detection Model,Multi-Label Classification Model,SmolVLM2,SIFT,Anthropic Claude,Roboflow Dataset Upload,Multi-Label Classification Model,Llama 3.2 Vision,PTZ Tracking (ONVIF),Object Detection Model,Email Notification,Seg Preview,Instance Segmentation Model,Overlap Analysis,Time in Zone,OpenRouter,Qwen3-VL,Per-Class Confidence Filter,Google Gemma API,Stability AI Image Generation,Heatmap Visualization,Contrast Equalization,Property Definition,Roboflow Dataset Upload,Slack Notification,Detection Offset,OpenAI,SAM 3,Image Blur,Expression,Keypoint Detection Model,Detections Consensus,Qwen2.5-VL,Dot Visualization,Polygon Zone Visualization,Label Visualization,Icon Visualization,LMM For Classification,Object Detection Model,OC-SORT Tracker,Blur Visualization,Bounding Rectangle,Delta Filter,Dimension Collapse,Trace Visualization,Size Measurement,Dynamic Zone,Florence-2 Model,Camera Focus,QR Code Detection,CogVLM,Pixelate Visualization,Polygon Visualization,Line Counter,Classification Label Visualization,Multi-Label Classification Model,Gaze Detection,Camera Calibration,Google Gemini,Image Preprocessing,Semantic Segmentation Model,Halo Visualization,Byte Tracker,Roboflow Asset Library Attributes,ByteTrack Tracker,Reference Path Visualization,Detections Stitch,CSV Formatter,VLM As Detector,Triangle Visualization,Crop Visualization,Qwen 3.5 API,Detection Event Log,OpenAI-Compatible LLM,First Non Empty Or Default,BoT-SORT Tracker,Detections List Roll-Up,SORT Tracker,Polygon Visualization,OpenAI
Input and Output Bindings¶
The available connections depend on its binding kinds. Check what binding kinds
OpenAI in version v1 has.
Bindings
-
input
images(image): The image to infer on..prompt(string): Text prompt to the OpenAI model.openai_api_key(Union[secret,string]): Your OpenAI API key.openai_model(string): Model to be used.image_detail(string): Indicates the image's quality, with 'high' suggesting it is of high resolution and should be processed or displayed with high fidelity..
-
output
parent_id(parent_id): Identifier of parent for step output.root_parent_id(parent_id): Identifier of parent for step output.image(image_metadata): Dictionary with image metadata required by supervision.structured_output(dictionary): Dictionary.raw_output(string): String value.*(*): Equivalent of any element.
Example JSON definition of step OpenAI in version v1
{
"name": "<your_step_name_here>",
"type": "roboflow_core/open_ai@v1",
"images": "$inputs.image",
"prompt": "my prompt",
"openai_api_key": "xxx-xxx",
"openai_model": "gpt-4o",
"json_output_format": {
"count": "number of cats in the picture"
},
"image_detail": "auto",
"max_tokens": 450
}