Florence-2 Model

v2

Class: Florence2BlockV2 (there are multiple versions of this block)

Source: inference.core.workflows.core_steps.models.foundation.florence2.v2.Florence2BlockV2

Warning: This block has multiple versions. Please refer to the specific version for details. You can learn more about how versions work here: Versioning

Dedicated inference server required (GPU recommended) - you may want to use dedicated deployment

This Workflow block introduces Florence 2, a Visual Language Model (VLM) capable of performing a wide range of tasks, including:

• Object Detection

• Instance Segmentation

• Image Captioning

• Optical Character Recognition (OCR)

• and more...

Below is a comprehensive list of tasks supported by the model, along with descriptions on how to utilize their outputs within the Workflows ecosystem:

Task Descriptions:

• Custom Prompt (custom) - Use free-form prompt to generate a response. Useful with finetuned models.

• Text Recognition (OCR) (ocr) - Model recognizes text in the image

• Text Detection & Recognition (OCR) (ocr-with-text-detection) - Model detects text regions in the image, and then performs OCR on each detected region

• Captioning (short) (caption) - Model provides a short description of the image

• Captioning (detailed-caption) - Model provides a long description of the image

• Captioning (long) (more-detailed-caption) - Model provides a very long description of the image

• Unprompted Object Detection (object-detection) - Model detects and returns the bounding boxes for prominent objects in the image

• Object Detection (open-vocabulary-object-detection) - Model detects and returns the bounding boxes for the provided classes

• Detection & Captioning (object-detection-and-caption) - Model detects prominent objects and captions them

• Prompted Object Detection (phrase-grounded-object-detection) - Based on the textual prompt, model detects objects matching the descriptions

• Prompted Instance Segmentation (phrase-grounded-instance-segmentation) - Based on the textual prompt, model segments objects matching the descriptions

• Segment Bounding Box (detection-grounded-instance-segmentation) - Model segments the object in the provided bounding box into a polygon

• Classification of Bounding Box (detection-grounded-classification) - Model classifies the object inside the provided bounding box

• Captioning of Bounding Box (detection-grounded-caption) - Model captions the object in the provided bounding box

• Text Recognition (OCR) for Bounding Box (detection-grounded-ocr) - Model performs OCR on the text inside the provided bounding box

• Regions of Interest proposal (region-proposal) - Model proposes Regions of Interest (Bounding Boxes) in the image

Type identifier

Use the following identifier in step "type" field: roboflow_core/florence_2@v2to add the block as as step in your workflow.

Properties

Name	Type	Description	Refs
name	str	Enter a unique identifier for this step..	❌
task_type	str	Task type to be performed by model. Value determines required parameters and output response..	❌
prompt	str	Text prompt to the Florence-2 model.	✅
classes	List[str]	List of classes to be used.	✅
grounding_detection	Optional[List[float], List[int]]	Detection to ground Florence-2 model. May be statically provided bounding box [left_top_x, left_top_y, right_bottom_x, right_bottom_y] or result of object-detection model. If the latter is true, one box will be selected based on grounding_selection_mode..	✅
grounding_selection_mode	str	.	❌
model_id	str	Model to be used.	✅

The Refs column marks possibility to parametrise the property with dynamic values available in workflow runtime. See Bindings for more info.

Available Connections

Compatible Blocks

Check what blocks you can connect to Florence-2 Model in version v2.

• inputs: LMM, Background Color Visualization, Stitch Images, Image Slicer, Size Measurement, Dynamic Zone, Model Monitoring Inference Aggregator, Corner Visualization, Detections Transformation, Detections Classes Replacement, Camera Calibration, Mask Visualization, Object Detection Model, Local File Sink, Line Counter, Model Comparison Visualization, Email Notification, Keypoint Detection Model, Pixelate Visualization, Anthropic Claude, Relative Static Crop, Time in Zone, Google Gemini, Florence-2 Model, Multi-Label Classification Model, Ellipse Visualization, Triangle Visualization, Segment Anything 2 Model, Camera Focus, OCR Model, QR Code Generator, Byte Tracker, Label Visualization, Time in Zone, Roboflow Custom Metadata, Florence-2 Model, LMM For Classification, Blur Visualization, Dot Visualization, Overlap Filter, Stability AI Image Generation, Detection Offset, Perspective Correction, Google Vision OCR, Llama 3.2 Vision, EasyOCR, Detections Stabilizer, Absolute Static Crop, Slack Notification, SAM 3, Morphological Transformation, Velocity, Image Blur, Image Threshold, Clip Comparison, Stitch OCR Detections, Depth Estimation, Stability AI Outpainting, Halo Visualization, Byte Tracker, Stability AI Inpainting, Polygon Visualization, OpenAI, Grid Visualization, Roboflow Dataset Upload, Path Deviation, Template Matching, CogVLM, Classification Label Visualization, Email Notification, VLM as Detector, Instance Segmentation Model, Bounding Box Visualization, Byte Tracker, Image Convert Grayscale, Polygon Zone Visualization, OpenAI, Clip Comparison, Detections Stitch, Keypoint Detection Model, Crop Visualization, Object Detection Model, Image Slicer, YOLO-World Model, Detections Merge, Gaze Detection, Moondream2, Icon Visualization, Seg Preview, Color Visualization, Roboflow Dataset Upload, Dimension Collapse, Path Deviation, Keypoint Visualization, Contrast Equalization, Buffer, Image Contours, Instance Segmentation Model, Detections Filter, Circle Visualization, OpenAI, VLM as Classifier, Bounding Rectangle, CSV Formatter, Time in Zone, Detections Combine, Reference Path Visualization, Twilio SMS Notification, VLM as Detector, Dynamic Crop, Webhook Sink, Detections Consensus, Single-Label Classification Model, SIFT, Line Counter Visualization, PTZ Tracking (ONVIF).md), Image Preprocessing, Trace Visualization, SIFT Comparison
• outputs: LMM, Background Color Visualization, Size Measurement, VLM as Classifier, Model Monitoring Inference Aggregator, Corner Visualization, Detections Classes Replacement, Mask Visualization, CLIP Embedding Model, Line Counter, Local File Sink, Model Comparison Visualization, Email Notification, Object Detection Model, Keypoint Detection Model, Time in Zone, Anthropic Claude, Google Gemini, Florence-2 Model, Ellipse Visualization, Triangle Visualization, Segment Anything 2 Model, QR Code Generator, Label Visualization, Time in Zone, Pixel Color Count, Roboflow Custom Metadata, Cache Set, Florence-2 Model, LMM For Classification, Dot Visualization, Stability AI Image Generation, Perspective Correction, Google Vision OCR, Llama 3.2 Vision, Line Counter, Slack Notification, SAM 3, Morphological Transformation, Image Blur, Clip Comparison, Image Threshold, Stitch OCR Detections, Stability AI Outpainting, Halo Visualization, Stability AI Inpainting, Polygon Visualization, OpenAI, Grid Visualization, Roboflow Dataset Upload, Path Deviation, Distance Measurement, CogVLM, Classification Label Visualization, Trace Visualization, Email Notification, VLM as Detector, Instance Segmentation Model, Bounding Box Visualization, Polygon Zone Visualization, Perception Encoder Embedding Model, OpenAI, Clip Comparison, Detections Stitch, Keypoint Detection Model, Crop Visualization, Object Detection Model, Cache Get, YOLO-World Model, Moondream2, Icon Visualization, Seg Preview, Color Visualization, Path Deviation, Roboflow Dataset Upload, JSON Parser, Keypoint Visualization, Contrast Equalization, Buffer, Instance Segmentation Model, Circle Visualization, OpenAI, VLM as Classifier, Time in Zone, Reference Path Visualization, Twilio SMS Notification, VLM as Detector, Webhook Sink, Detections Consensus, Line Counter Visualization, PTZ Tracking (ONVIF).md), Image Preprocessing, Dynamic Crop, SIFT Comparison

Input and Output Bindings

The available connections depend on its binding kinds. Check what binding kinds Florence-2 Model in version v2 has.

Bindings

• input

  • images (image): The image to infer on..
  • prompt (string): Text prompt to the Florence-2 model.
  • classes (list_of_values): List of classes to be used.
  • grounding_detection (Union[instance_segmentation_prediction, keypoint_detection_prediction, object_detection_prediction, list_of_values]): Detection to ground Florence-2 model. May be statically provided bounding box [left_top_x, left_top_y, right_bottom_x, right_bottom_y] or result of object-detection model. If the latter is true, one box will be selected based on grounding_selection_mode..
  • model_id (roboflow_model_id): Model to be used.

• output

  • raw_output (Union[string, language_model_output]): String value if string or LLM / VLM output if language_model_output.
  • parsed_output (dictionary): Dictionary.
  • classes (list_of_values): List of values of any type.

Example JSON definition of step Florence-2 Model in version v2

{
    "name": "<your_step_name_here>",
    "type": "roboflow_core/florence_2@v2",
    "images": "$inputs.image",
    "task_type": "<block_does_not_provide_example>",
    "prompt": "my prompt",
    "classes": [
        "class-a",
        "class-b"
    ],
    "grounding_detection": "$steps.detection.predictions",
    "grounding_selection_mode": "first",
    "model_id": "florence-2-base"
}

v1

Class: Florence2BlockV1 (there are multiple versions of this block)

Source: inference.core.workflows.core_steps.models.foundation.florence2.v1.Florence2BlockV1

Warning: This block has multiple versions. Please refer to the specific version for details. You can learn more about how versions work here: Versioning

Dedicated inference server required (GPU recommended) - you may want to use dedicated deployment

This Workflow block introduces Florence 2, a Visual Language Model (VLM) capable of performing a wide range of tasks, including:

• Object Detection

• Instance Segmentation

• Image Captioning

• Optical Character Recognition (OCR)

• and more...

Below is a comprehensive list of tasks supported by the model, along with descriptions on how to utilize their outputs within the Workflows ecosystem:

Task Descriptions:

• Custom Prompt (custom) - Use free-form prompt to generate a response. Useful with finetuned models.

• Text Recognition (OCR) (ocr) - Model recognizes text in the image

• Text Detection & Recognition (OCR) (ocr-with-text-detection) - Model detects text regions in the image, and then performs OCR on each detected region

• Captioning (short) (caption) - Model provides a short description of the image

• Captioning (detailed-caption) - Model provides a long description of the image

• Captioning (long) (more-detailed-caption) - Model provides a very long description of the image

• Unprompted Object Detection (object-detection) - Model detects and returns the bounding boxes for prominent objects in the image

• Object Detection (open-vocabulary-object-detection) - Model detects and returns the bounding boxes for the provided classes

• Detection & Captioning (object-detection-and-caption) - Model detects prominent objects and captions them

• Prompted Object Detection (phrase-grounded-object-detection) - Based on the textual prompt, model detects objects matching the descriptions

• Prompted Instance Segmentation (phrase-grounded-instance-segmentation) - Based on the textual prompt, model segments objects matching the descriptions

• Segment Bounding Box (detection-grounded-instance-segmentation) - Model segments the object in the provided bounding box into a polygon

• Classification of Bounding Box (detection-grounded-classification) - Model classifies the object inside the provided bounding box

• Captioning of Bounding Box (detection-grounded-caption) - Model captions the object in the provided bounding box

• Text Recognition (OCR) for Bounding Box (detection-grounded-ocr) - Model performs OCR on the text inside the provided bounding box

• Regions of Interest proposal (region-proposal) - Model proposes Regions of Interest (Bounding Boxes) in the image

Type identifier

Use the following identifier in step "type" field: roboflow_core/florence_2@v1to add the block as as step in your workflow.

Properties

Name	Type	Description	Refs
name	str	Enter a unique identifier for this step..	❌
task_type	str	Task type to be performed by model. Value determines required parameters and output response..	❌
prompt	str	Text prompt to the Florence-2 model.	✅
classes	List[str]	List of classes to be used.	✅
grounding_detection	Optional[List[float], List[int]]	Detection to ground Florence-2 model. May be statically provided bounding box [left_top_x, left_top_y, right_bottom_x, right_bottom_y] or result of object-detection model. If the latter is true, one box will be selected based on grounding_selection_mode..	✅
grounding_selection_mode	str	.	❌
model_version	str	Model to be used.	✅

The Refs column marks possibility to parametrise the property with dynamic values available in workflow runtime. See Bindings for more info.

Available Connections

Compatible Blocks

Check what blocks you can connect to Florence-2 Model in version v1.

• inputs: Background Color Visualization, Stitch Images, Image Slicer, Size Measurement, Detections Transformation, Corner Visualization, Mask Visualization, Local File Sink, Line Counter, Model Comparison Visualization, Email Notification, Time in Zone, Florence-2 Model, Multi-Label Classification Model, Ellipse Visualization, Camera Focus, OCR Model, Label Visualization, LMM For Classification, Blur Visualization, Dot Visualization, Google Vision OCR, Llama 3.2 Vision, Detections Stabilizer, Slack Notification, Image Blur, Stitch OCR Detections, Depth Estimation, Stability AI Outpainting, Halo Visualization, Stability AI Inpainting, CogVLM, Classification Label Visualization, VLM as Detector, Instance Segmentation Model, Byte Tracker, Image Convert Grayscale, Polygon Zone Visualization, Clip Comparison, Detections Stitch, Crop Visualization, Image Slicer, YOLO-World Model, Detections Merge, Icon Visualization, Seg Preview, Color Visualization, Path Deviation, Buffer, Circle Visualization, Bounding Rectangle, CSV Formatter, Time in Zone, Dynamic Crop, Single-Label Classification Model, SIFT, Line Counter Visualization, PTZ Tracking (ONVIF).md), Image Preprocessing, Trace Visualization, SIFT Comparison, LMM, Dynamic Zone, Model Monitoring Inference Aggregator, Detections Classes Replacement, Camera Calibration, Object Detection Model, Keypoint Detection Model, Pixelate Visualization, Anthropic Claude, Relative Static Crop, Google Gemini, Triangle Visualization, Segment Anything 2 Model, QR Code Generator, Byte Tracker, Time in Zone, Roboflow Custom Metadata, Florence-2 Model, Overlap Filter, Stability AI Image Generation, Detection Offset, EasyOCR, Absolute Static Crop, SAM 3, Morphological Transformation, Velocity, Clip Comparison, Image Threshold, Byte Tracker, Polygon Visualization, OpenAI, Grid Visualization, Roboflow Dataset Upload, Path Deviation, Template Matching, Email Notification, Bounding Box Visualization, OpenAI, Keypoint Detection Model, Object Detection Model, Gaze Detection, Moondream2, Roboflow Dataset Upload, Dimension Collapse, Keypoint Visualization, Contrast Equalization, Image Contours, Instance Segmentation Model, Detections Filter, OpenAI, VLM as Classifier, Detections Combine, Reference Path Visualization, Twilio SMS Notification, VLM as Detector, Webhook Sink, Detections Consensus, Perspective Correction
• outputs: LMM, Background Color Visualization, Size Measurement, VLM as Classifier, Model Monitoring Inference Aggregator, Corner Visualization, Detections Classes Replacement, Mask Visualization, CLIP Embedding Model, Line Counter, Local File Sink, Model Comparison Visualization, Email Notification, Object Detection Model, Keypoint Detection Model, Time in Zone, Anthropic Claude, Google Gemini, Florence-2 Model, Ellipse Visualization, Triangle Visualization, Segment Anything 2 Model, QR Code Generator, Label Visualization, Time in Zone, Pixel Color Count, Roboflow Custom Metadata, Cache Set, Florence-2 Model, LMM For Classification, Dot Visualization, Stability AI Image Generation, Perspective Correction, Google Vision OCR, Llama 3.2 Vision, Line Counter, Slack Notification, SAM 3, Morphological Transformation, Image Blur, Clip Comparison, Image Threshold, Stitch OCR Detections, Stability AI Outpainting, Halo Visualization, Stability AI Inpainting, Polygon Visualization, OpenAI, Grid Visualization, Roboflow Dataset Upload, Path Deviation, Distance Measurement, CogVLM, Classification Label Visualization, Trace Visualization, Email Notification, VLM as Detector, Instance Segmentation Model, Bounding Box Visualization, Polygon Zone Visualization, Perception Encoder Embedding Model, OpenAI, Clip Comparison, Detections Stitch, Keypoint Detection Model, Crop Visualization, Object Detection Model, Cache Get, YOLO-World Model, Moondream2, Icon Visualization, Seg Preview, Color Visualization, Path Deviation, Roboflow Dataset Upload, JSON Parser, Keypoint Visualization, Contrast Equalization, Buffer, Instance Segmentation Model, Circle Visualization, OpenAI, VLM as Classifier, Time in Zone, Reference Path Visualization, Twilio SMS Notification, VLM as Detector, Webhook Sink, Detections Consensus, Line Counter Visualization, PTZ Tracking (ONVIF).md), Image Preprocessing, Dynamic Crop, SIFT Comparison

Input and Output Bindings

The available connections depend on its binding kinds. Check what binding kinds Florence-2 Model in version v1 has.

Bindings

• input

  • images (image): The image to infer on..
  • prompt (string): Text prompt to the Florence-2 model.
  • classes (list_of_values): List of classes to be used.
  • grounding_detection (Union[instance_segmentation_prediction, keypoint_detection_prediction, object_detection_prediction, list_of_values]): Detection to ground Florence-2 model. May be statically provided bounding box [left_top_x, left_top_y, right_bottom_x, right_bottom_y] or result of object-detection model. If the latter is true, one box will be selected based on grounding_selection_mode..
  • model_version (string): Model to be used.

• output

  • raw_output (Union[string, language_model_output]): String value if string or LLM / VLM output if language_model_output.
  • parsed_output (dictionary): Dictionary.
  • classes (list_of_values): List of values of any type.

Example JSON definition of step Florence-2 Model in version v1

{
    "name": "<your_step_name_here>",
    "type": "roboflow_core/florence_2@v1",
    "images": "$inputs.image",
    "task_type": "<block_does_not_provide_example>",
    "prompt": "my prompt",
    "classes": [
        "class-a",
        "class-b"
    ],
    "grounding_detection": "$steps.detection.predictions",
    "grounding_selection_mode": "first",
    "model_version": "florence-2-base"
}