Skip to content

Clip Comparison

Version v2

Use the OpenAI CLIP zero-shot classification model to classify images.

This block accepts an image and a list of text prompts. The block then returns the similarity of each text label to the provided image.

This block is useful for classifying images without having to train a fine-tuned classification model. For example, you could use CLIP to classify the type of vehicle in an image, or if an image contains NSFW material.

Type identifier

Use the following identifier in step "type" field: roboflow_core/clip_comparison@v2to add the block as as step in your workflow.

Properties

Name Type Description Refs
name str Unique name of step in workflows. ❌
classes List[str] List of classes to calculate similarity against each input image. ✅
version str Variant of CLIP model. ✅

The Refs column marks possibility to parametrise the property with dynamic values available in workflow runtime. See Bindings for more info.

Available Connections

Check what blocks you can connect to Clip Comparison in version v2.

The available connections depend on its binding kinds. Check what binding kinds Clip Comparison in version v2 has.

Bindings

  • input

    • images (image): The image to infer on.
    • classes (list_of_values): List of classes to calculate similarity against each input image.
    • version (string): Variant of CLIP model.

  • output

Example JSON definition of step Clip Comparison in version v2 
{
    "name": "<your_step_name_here>",
    "type": "roboflow_core/clip_comparison@v2",
    "images": "$inputs.image",
    "classes": [
        "a",
        "b",
        "c"
    ],
    "version": "ViT-B-16"
}

Version v1

Use the OpenAI CLIP zero-shot classification model to classify images.

This block accepts an image and a list of text prompts. The block then returns the similarity of each text label to the provided image.

This block is useful for classifying images without having to train a fine-tuned classification model. For example, you could use CLIP to classify the type of vehicle in an image, or if an image contains NSFW material.

Type identifier

Use the following identifier in step "type" field: roboflow_core/clip_comparison@v1to add the block as as step in your workflow.

Properties

Name Type Description Refs
name str Unique name of step in workflows. ❌
texts List[str] List of texts to calculate similarity against each input image. ✅

The Refs column marks possibility to parametrise the property with dynamic values available in workflow runtime. See Bindings for more info.

Available Connections

Check what blocks you can connect to Clip Comparison in version v1.

The available connections depend on its binding kinds. Check what binding kinds Clip Comparison in version v1 has.

Bindings

  • input

    • images (image): The image to infer on.
    • texts (list_of_values): List of texts to calculate similarity against each input image.

  • output

    • similarity (list_of_values): List of values of any type.
    • parent_id (parent_id): Identifier of parent for step output.
    • root_parent_id (parent_id): Identifier of parent for step output.
    • prediction_type (prediction_type): String value with type of prediction.
Example JSON definition of step Clip Comparison in version v1 
{
    "name": "<your_step_name_here>",
    "type": "roboflow_core/clip_comparison@v1",
    "images": "$inputs.image",
    "texts": [
        "a",
        "b",
        "c"
    ]
}