OpenAI-Compatible LLM¶

Send a prompt to any OpenAI-compatible API endpoint (e.g. local Qwen, vLLM, Ollama, LM Studio, or any service that implements the OpenAI chat completions API).

How this block works¶

1. You provide a Base URL (e.g. http://localhost:8000/v1) and a Model Name.
2. Write an Instruction — the text the model receives.
3. Add rows under Inputs to feed step outputs (images, detections, text) into the request. Image inputs are base64-encoded and sent as vision content parts. A list of images becomes one vision part per image.
4. Non-image inputs are converted to strings. To splice them into the instruction text, reference the input by name with the placeholder syntax shown in the Instruction field's help text.
5. Optionally apply UQL operations to transform input values before insertion.

Image handling¶

• A WorkflowImageData value is JPEG-encoded and sent as an image_url part.
• Raw JPEG bytes (e.g. from the Image Stack block) are sent directly.
• A list of either is fanned out into multiple image_url parts.

If an image input is also referenced in the instruction text by name, the placeholder is removed from the text — the image only travels as a vision part.

Type identifier¶

Use the following identifier in step "type" field: roboflow_core/openai_compatible@v1to add the block as as step in your workflow.

Properties¶

The Refs column marks possibility to parametrise the property with dynamic values available in workflow runtime. See Bindings for more info.

Available Connections¶

Input and Output Bindings¶

The available connections depend on its binding kinds. Check what binding kinds OpenAI-Compatible LLM in version v1 has.

{
    "name": "<your_step_name_here>",
    "type": "roboflow_core/openai_compatible@v1",
    "base_url": "http://localhost:8000/v1",
    "model_name": "Qwen/Qwen2.5-VL-7B-Instruct",
    "api_key": "xxx-xxx",
    "system_prompt": "You are a helpful assistant.",
    "prompt": "Describe what you see in the image.",
    "prompt_parameters": {
        "detections": "$steps.model.predictions",
        "frames": "$steps.image_stack.frames"
    },
    "prompt_parameters_operations": {
        "detections": [
            {
                "property_name": "class_name",
                "type": "DetectionsPropertyExtract"
            }
        ]
    },
    "max_tokens": "<block_does_not_provide_example>",
    "temperature": "<block_does_not_provide_example>",
    "extra_body": {
        "chat_template_kwargs": {
            "enable_thinking": false
        },
        "guided_choice": [
            "A",
            "B",
            "C",
            "D"
        ]
    }
}