Google Gemini¶
Class: GoogleGeminiBlockV1
Source: inference.core.workflows.core_steps.models.foundation.google_gemini.v1.GoogleGeminiBlockV1
Ask a question to Google's Gemini model with vision capabilities.
You can specify arbitrary text prompts or predefined ones, the block supports the following types of prompt:
-
Open Prompt (
unconstrained) - Use any prompt to generate a raw response -
Text Recognition (OCR) (
ocr) - Model recognizes text in the image -
Visual Question Answering (
visual-question-answering) - Model answers the question you submit in the prompt -
Captioning (short) (
caption) - Model provides a short description of the image -
Captioning (
detailed-caption) - Model provides a long description of the image -
Single-Label Classification (
classification) - Model classifies the image content as one of the provided classes -
Multi-Label Classification (
multi-label-classification) - Model classifies the image content as one or more of the provided classes -
Unprompted Object Detection (
object-detection) - Model detects and returns the bounding boxes for prominent objects in the image -
Structured Output Generation (
structured-answering) - Model returns a JSON response with the specified fields
You need to provide your Google AI API key to use the Gemini model.
WARNING!
This block makes use of /v1beta API of Google Gemini model - the implementation may change
in the future, without guarantee of backward compatibility.
Type identifier¶
Use the following identifier in step "type" field: roboflow_core/google_gemini@v1to add the block as
as step in your workflow.
Properties¶
| Name | Type | Description | Refs |
|---|---|---|---|
name |
str |
Enter a unique identifier for this step.. | ❌ |
task_type |
str |
Task type to be performed by model. Value determines required parameters and output response.. | ❌ |
prompt |
str |
Text prompt to the Gemini model. | ✅ |
output_structure |
Dict[str, str] |
Dictionary with structure of expected JSON response. | ❌ |
classes |
List[str] |
List of classes to be used. | ✅ |
api_key |
str |
Your Google AI API key. | ✅ |
model_version |
str |
Model to be used. | ✅ |
max_tokens |
int |
Maximum number of tokens the model can generate in it's response.. | ❌ |
temperature |
float |
Temperature to sample from the model - value in range 0.0-2.0, the higher - the more random / "creative" the generations are.. | ✅ |
max_concurrent_requests |
int |
Number of concurrent requests that can be executed by block when batch of input images provided. If not given - block defaults to value configured globally in Workflows Execution Engine. Please restrict if you hit Google Gemini API limits.. | ❌ |
The Refs column marks possibility to parametrise the property with dynamic values available
in workflow runtime. See Bindings for more info.
Available Connections¶
Compatible Blocks
Check what blocks you can connect to Google Gemini in version v1.
- inputs:
Circle Visualization,Background Color Visualization,Corner Visualization,Twilio SMS Notification,Slack Notification,VLM as Detector,LMM,Polygon Zone Visualization,Camera Focus,Image Slicer,Image Blur,Dot Visualization,Google Gemini,Roboflow Dataset Upload,Stability AI Inpainting,Pixelate Visualization,OpenAI,Gaze Detection,Image Convert Grayscale,Absolute Static Crop,Stability AI Image Generation,Webhook Sink,Color Visualization,Image Threshold,Halo Visualization,Polygon Visualization,VLM as Classifier,Dynamic Zone,CogVLM,Camera Calibration,Email Notification,Classification Label Visualization,Cosine Similarity,Single-Label Classification Model,Llama 3.2 Vision,Google Vision OCR,Roboflow Dataset Upload,Ellipse Visualization,Size Measurement,Bounding Box Visualization,Object Detection Model,Line Counter Visualization,Image Preprocessing,Trace Visualization,Label Visualization,Clip Comparison,Local File Sink,Image Slicer,Anthropic Claude,Crop Visualization,OCR Model,Relative Static Crop,Model Comparison Visualization,Stitch OCR Detections,Perspective Correction,OpenAI,Mask Visualization,Clip Comparison,Dynamic Crop,CSV Formatter,Florence-2 Model,Instance Segmentation Model,Keypoint Detection Model,Image Contours,Buffer,SIFT,Reference Path Visualization,Florence-2 Model,Triangle Visualization,Model Monitoring Inference Aggregator,SIFT Comparison,Keypoint Visualization,Identify Changes,Multi-Label Classification Model,LMM For Classification,Roboflow Custom Metadata,Grid Visualization,Stitch Images,Dimension Collapse,Blur Visualization - outputs:
Circle Visualization,Background Color Visualization,Corner Visualization,Twilio SMS Notification,Slack Notification,VLM as Detector,VLM as Classifier,LMM,Polygon Zone Visualization,Image Blur,Cache Set,Dot Visualization,Path Deviation,Google Gemini,Roboflow Dataset Upload,Stability AI Inpainting,Line Counter,OpenAI,Detections Consensus,Distance Measurement,Stability AI Image Generation,Webhook Sink,Color Visualization,Image Threshold,Halo Visualization,Polygon Visualization,Grid Visualization,VLM as Classifier,CogVLM,Instance Segmentation Model,Email Notification,Object Detection Model,Classification Label Visualization,Llama 3.2 Vision,Google Vision OCR,Roboflow Dataset Upload,Ellipse Visualization,Size Measurement,Pixel Color Count,Cache Get,Bounding Box Visualization,JSON Parser,Object Detection Model,Line Counter Visualization,Image Preprocessing,Keypoint Detection Model,Trace Visualization,Label Visualization,Clip Comparison,Local File Sink,Anthropic Claude,Crop Visualization,Detections Stitch,YOLO-World Model,Model Comparison Visualization,Perspective Correction,OpenAI,Path Deviation,Mask Visualization,Time in Zone,Clip Comparison,Time in Zone,Dynamic Crop,Florence-2 Model,Instance Segmentation Model,Keypoint Detection Model,Buffer,Florence-2 Model,Reference Path Visualization,CLIP Embedding Model,Triangle Visualization,Model Monitoring Inference Aggregator,SIFT Comparison,VLM as Detector,Keypoint Visualization,LMM For Classification,Roboflow Custom Metadata,Line Counter,Segment Anything 2 Model
Input and Output Bindings¶
The available connections depend on its binding kinds. Check what binding kinds
Google Gemini in version v1 has.
Bindings
-
input
images(image): The image to infer on..prompt(string): Text prompt to the Gemini model.classes(list_of_values): List of classes to be used.api_key(Union[string,secret]): Your Google AI API key.model_version(string): Model to be used.temperature(float): Temperature to sample from the model - value in range 0.0-2.0, the higher - the more random / "creative" the generations are..
-
output
output(Union[string,language_model_output]): String value ifstringor LLM / VLM output iflanguage_model_output.classes(list_of_values): List of values of any type.
Example JSON definition of step Google Gemini in version v1
{
"name": "<your_step_name_here>",
"type": "roboflow_core/google_gemini@v1",
"images": "$inputs.image",
"task_type": "<block_does_not_provide_example>",
"prompt": "my prompt",
"output_structure": {
"my_key": "description"
},
"classes": [
"class-a",
"class-b"
],
"api_key": "xxx-xxx",
"model_version": "gemini-2.0-flash-exp",
"max_tokens": "<block_does_not_provide_example>",
"temperature": "<block_does_not_provide_example>",
"max_concurrent_requests": "<block_does_not_provide_example>"
}