Google Gemini

Class: GoogleGeminiBlockV1

Source: inference.core.workflows.core_steps.models.foundation.google_gemini.v1.GoogleGeminiBlockV1

Ask a question to Google's Gemini model with vision capabilities.

You can specify arbitrary text prompts or predefined ones, the block supports the following types of prompt:

• Open Prompt (unconstrained) - Use any prompt to generate a raw response

• Text Recognition (OCR) (ocr) - Model recognizes text in the image

• Visual Question Answering (visual-question-answering) - Model answers the question you submit in the prompt

• Captioning (short) (caption) - Model provides a short description of the image

• Captioning (detailed-caption) - Model provides a long description of the image

• Single-Label Classification (classification) - Model classifies the image content as one of the provided classes

• Multi-Label Classification (multi-label-classification) - Model classifies the image content as one or more of the provided classes

• Unprompted Object Detection (object-detection) - Model detects and returns the bounding boxes for prominent objects in the image

• Structured Output Generation (structured-answering) - Model returns a JSON response with the specified fields

You need to provide your Google AI API key to use the Gemini model.

WARNING!

This block makes use of /v1beta API of Google Gemini model - the implementation may change in the future, without guarantee of backward compatibility.

Type identifier

Use the following identifier in step "type" field: roboflow_core/google_gemini@v1to add the block as as step in your workflow.

Properties

Name	Type	Description	Refs
name	str	Enter a unique identifier for this step..	❌
task_type	str	Task type to be performed by model. Value determines required parameters and output response..	❌
prompt	str	Text prompt to the Gemini model.	✅
output_structure	Dict[str, str]	Dictionary with structure of expected JSON response.	❌
classes	List[str]	List of classes to be used.	✅
api_key	str	Your Google AI API key.	✅
model_version	str	Model to be used.	✅
max_tokens	int	Maximum number of tokens the model can generate in it's response..	❌
temperature	float	Temperature to sample from the model - value in range 0.0-2.0, the higher - the more random / "creative" the generations are..	✅
max_concurrent_requests	int	Number of concurrent requests that can be executed by block when batch of input images provided. If not given - block defaults to value configured globally in Workflows Execution Engine. Please restrict if you hit Google Gemini API limits..	❌

The Refs column marks possibility to parametrise the property with dynamic values available in workflow runtime. See Bindings for more info.

Available Connections

Compatible Blocks

Check what blocks you can connect to Google Gemini in version v1.

• inputs: Size Measurement, Absolute Static Crop, Relative Static Crop, Keypoint Visualization, Object Detection Model, Clip Comparison, LMM For Classification, Google Vision OCR, Slack Notification, Dimension Collapse, Trace Visualization, Color Visualization, Instance Segmentation Model, Polygon Zone Visualization, Camera Focus, Halo Visualization, CSV Formatter, OCR Model, Identify Changes, Camera Calibration, VLM as Classifier, Dynamic Zone, Triangle Visualization, Stability AI Inpainting, Image Threshold, Reference Path Visualization, Corner Visualization, Ellipse Visualization, Gaze Detection, OpenAI, Single-Label Classification Model, Morphological Transformation, Roboflow Custom Metadata, Grid Visualization, Image Preprocessing, CogVLM, Line Counter Visualization, OpenAI, Florence-2 Model, Roboflow Dataset Upload, Stitch OCR Detections, Label Visualization, Model Comparison Visualization, Multi-Label Classification Model, Roboflow Dataset Upload, OpenAI, Model Monitoring Inference Aggregator, Polygon Visualization, Llama 3.2 Vision, Icon Visualization, Local File Sink, Blur Visualization, Image Contours, Clip Comparison, VLM as Detector, Twilio SMS Notification, Bounding Box Visualization, SIFT, Cosine Similarity, Classification Label Visualization, Background Color Visualization, Webhook Sink, Dynamic Crop, Dot Visualization, Pixelate Visualization, Email Notification, Buffer, Image Slicer, Stitch Images, Crop Visualization, Mask Visualization, SIFT Comparison, QR Code Generator, Depth Estimation, Image Slicer, Google Gemini, Perspective Correction, Image Convert Grayscale, Stability AI Image Generation, Keypoint Detection Model, EasyOCR, Contrast Equalization, Anthropic Claude, Image Blur, Circle Visualization, Stability AI Outpainting, LMM, Florence-2 Model
• outputs: Size Measurement, VLM as Detector, JSON Parser, Keypoint Visualization, Object Detection Model, Clip Comparison, LMM For Classification, VLM as Classifier, Google Vision OCR, Slack Notification, Trace Visualization, Color Visualization, Instance Segmentation Model, Seg Preview, Polygon Zone Visualization, Halo Visualization, VLM as Classifier, Triangle Visualization, Segment Anything 2 Model, Stability AI Inpainting, Image Threshold, Reference Path Visualization, Corner Visualization, Detections Classes Replacement, Ellipse Visualization, OpenAI, Time in Zone, Morphological Transformation, Roboflow Custom Metadata, Image Preprocessing, Grid Visualization, Cache Set, CogVLM, Line Counter Visualization, OpenAI, YOLO-World Model, Florence-2 Model, Roboflow Dataset Upload, Stitch OCR Detections, Path Deviation, Label Visualization, PTZ Tracking (ONVIF).md), Model Comparison Visualization, Roboflow Dataset Upload, OpenAI, Line Counter, Line Counter, Path Deviation, Polygon Visualization, Model Monitoring Inference Aggregator, Time in Zone, Instance Segmentation Model, Llama 3.2 Vision, Icon Visualization, Local File Sink, Time in Zone, Clip Comparison, VLM as Detector, Object Detection Model, Twilio SMS Notification, Moondream2, Bounding Box Visualization, Classification Label Visualization, Background Color Visualization, Webhook Sink, Dynamic Crop, Dot Visualization, Detections Consensus, Email Notification, Buffer, CLIP Embedding Model, Crop Visualization, Keypoint Detection Model, Perception Encoder Embedding Model, Mask Visualization, SIFT Comparison, Pixel Color Count, QR Code Generator, Google Gemini, Perspective Correction, Stability AI Image Generation, Keypoint Detection Model, Distance Measurement, Contrast Equalization, Anthropic Claude, Image Blur, Cache Get, Circle Visualization, Stability AI Outpainting, LMM, Detections Stitch, Florence-2 Model

Input and Output Bindings

The available connections depend on its binding kinds. Check what binding kinds Google Gemini in version v1 has.

Bindings

• input

  • images (image): The image to infer on..
  • prompt (string): Text prompt to the Gemini model.
  • classes (list_of_values): List of classes to be used.
  • api_key (Union[secret, string]): Your Google AI API key.
  • model_version (string): Model to be used.
  • temperature (float): Temperature to sample from the model - value in range 0.0-2.0, the higher - the more random / "creative" the generations are..

• output

  • output (Union[string, language_model_output]): String value if string or LLM / VLM output if language_model_output.
  • classes (list_of_values): List of values of any type.

Example JSON definition of step Google Gemini in version v1

{
    "name": "<your_step_name_here>",
    "type": "roboflow_core/google_gemini@v1",
    "images": "$inputs.image",
    "task_type": "<block_does_not_provide_example>",
    "prompt": "my prompt",
    "output_structure": {
        "my_key": "description"
    },
    "classes": [
        "class-a",
        "class-b"
    ],
    "api_key": "xxx-xxx",
    "model_version": "gemini-2.5-pro",
    "max_tokens": "<block_does_not_provide_example>",
    "temperature": "<block_does_not_provide_example>",
    "max_concurrent_requests": "<block_does_not_provide_example>"
}