Skip to content

Roboflow Dataset Upload

Version v2

Block let users save their images and predictions into Roboflow Dataset. Persisting data from production environments helps iteratively building more robust models.

Block provides configuration options to decide how data should be stored and what are the limits to be applied. We advice using this block in combination with rate limiter blocks to effectively collect data that the model struggle with.

Type identifier

Use the following identifier in step "type" field: roboflow_core/roboflow_dataset_upload@v2to add the block as as step in your workflow.

Properties

Name Type Description Refs
name str The unique name of this step.. ❌
target_project str name of Roboflow dataset / project to be used as target for collected data. ✅
usage_quota_name str Unique name for Roboflow project pointed by target_project parameter, that identifies usage quota applied for this block.. ❌
data_percentage float Percent of data that will be saved (in range [0.0, 100.0]). ✅
persist_predictions bool Boolean flag to decide if predictions should be registered along with images. ✅
minutely_usage_limit int Maximum number of data registration requests per minute accounted in scope of single server or whole Roboflow platform, depending on context of usage.. ❌
hourly_usage_limit int Maximum number of data registration requests per hour accounted in scope of single server or whole Roboflow platform, depending on context of usage.. ❌
daily_usage_limit int Maximum number of data registration requests per day accounted in scope of single server or whole Roboflow platform, depending on context of usage.. ❌
max_image_size Tuple[int, int] Maximum size of the image to be registered - bigger images will be downsized preserving aspect ratio. Format of data: (width, height). ❌
compression_level int Compression level for images registered. ❌
registration_tags List[str] Tags to be attached to registered datapoints. ✅
disable_sink bool boolean flag that can be also reference to input - to arbitrarily disable data collection for specific request. ✅
fire_and_forget bool Boolean flag dictating if sink is supposed to be executed in the background, not waiting on status of registration before end of workflow run. Use True if best-effort registration is needed, use False while debugging and if error handling is needed. ✅
labeling_batch_prefix str Prefix of the name for labeling batches that will be registered in Roboflow app. ✅
labeling_batches_recreation_frequency str Frequency in which new labeling batches are created in Roboflow app. New batches are created with name prefix provided in labeling_batch_prefix in given time intervals.Useful in organising labeling flow.. ❌

The Refs column marks possibility to parametrise the property with dynamic values available in workflow runtime. See Bindings for more info.

Available Connections

Check what blocks you can connect to Roboflow Dataset Upload in version v2.

The available connections depend on its binding kinds. Check what binding kinds Roboflow Dataset Upload in version v2 has.

Bindings

  • input

    • images (image): The image to infer on.
    • target_project (roboflow_project): name of Roboflow dataset / project to be used as target for collected data.
    • predictions (Union[instance_segmentation_prediction, classification_prediction, keypoint_detection_prediction, object_detection_prediction]): Model predictions to be saved.
    • data_percentage (float): Percent of data that will be saved (in range [0.0, 100.0]).
    • persist_predictions (boolean): Boolean flag to decide if predictions should be registered along with images.
    • registration_tags (string): Tags to be attached to registered datapoints.
    • disable_sink (boolean): boolean flag that can be also reference to input - to arbitrarily disable data collection for specific request.
    • fire_and_forget (boolean): Boolean flag dictating if sink is supposed to be executed in the background, not waiting on status of registration before end of workflow run. Use True if best-effort registration is needed, use False while debugging and if error handling is needed.
    • labeling_batch_prefix (string): Prefix of the name for labeling batches that will be registered in Roboflow app.

  • output

    • error_status (boolean): Boolean flag.
    • message (string): String value.
Example JSON definition of step Roboflow Dataset Upload in version v2 
{
    "name": "<your_step_name_here>",
    "type": "roboflow_core/roboflow_dataset_upload@v2",
    "images": "$inputs.image",
    "target_project": "my_dataset",
    "usage_quota_name": "quota-for-data-sampling-1",
    "predictions": "$steps.object_detection_model.predictions",
    "data_percentage": true,
    "persist_predictions": true,
    "minutely_usage_limit": 10,
    "hourly_usage_limit": 10,
    "daily_usage_limit": 10,
    "max_image_size": [
        1920,
        1080
    ],
    "compression_level": 95,
    "registration_tags": [
        "location-florida",
        "factory-name",
        "$inputs.dynamic_tag"
    ],
    "disable_sink": true,
    "fire_and_forget": "<block_does_not_provide_example>",
    "labeling_batch_prefix": "my_labeling_batch_name",
    "labeling_batches_recreation_frequency": "never"
}

Version v1

Block let users save their images and predictions into Roboflow Dataset. Persisting data from production environments helps iteratively building more robust models.

Block provides configuration options to decide how data should be stored and what are the limits to be applied. We advice using this block in combination with rate limiter blocks to effectively collect data that the model struggle with.

Type identifier

Use the following identifier in step "type" field: roboflow_core/roboflow_dataset_upload@v1to add the block as as step in your workflow.

Properties

Name Type Description Refs
name str The unique name of this step.. ❌
target_project str name of Roboflow dataset / project to be used as target for collected data. ✅
usage_quota_name str Unique name for Roboflow project pointed by target_project parameter, that identifies usage quota applied for this block.. ❌
persist_predictions bool Boolean flag to decide if predictions should be registered along with images. ❌
minutely_usage_limit int Maximum number of data registration requests per minute accounted in scope of single server or whole Roboflow platform, depending on context of usage.. ❌
hourly_usage_limit int Maximum number of data registration requests per hour accounted in scope of single server or whole Roboflow platform, depending on context of usage.. ❌
daily_usage_limit int Maximum number of data registration requests per day accounted in scope of single server or whole Roboflow platform, depending on context of usage.. ❌
max_image_size Tuple[int, int] Maximum size of the image to be registered - bigger images will be downsized preserving aspect ratio. Format of data: (width, height). ❌
compression_level int Compression level for images registered. ❌
registration_tags List[str] Tags to be attached to registered datapoints. ✅
disable_sink bool boolean flag that can be also reference to input - to arbitrarily disable data collection for specific request. ✅
fire_and_forget bool Boolean flag dictating if sink is supposed to be executed in the background, not waiting on status of registration before end of workflow run. Use True if best-effort registration is needed, use False while debugging and if error handling is needed. ✅
labeling_batch_prefix str Prefix of the name for labeling batches that will be registered in Roboflow app. ✅
labeling_batches_recreation_frequency str Frequency in which new labeling batches are created in Roboflow app. New batches are created with name prefix provided in labeling_batch_prefix in given time intervals.Useful in organising labeling flow.. ❌

The Refs column marks possibility to parametrise the property with dynamic values available in workflow runtime. See Bindings for more info.

Available Connections

Check what blocks you can connect to Roboflow Dataset Upload in version v1.

The available connections depend on its binding kinds. Check what binding kinds Roboflow Dataset Upload in version v1 has.

Bindings

  • input

    • images (image): The image to infer on.
    • predictions (Union[instance_segmentation_prediction, classification_prediction, keypoint_detection_prediction, object_detection_prediction]): Reference q detection-like predictions.
    • target_project (roboflow_project): name of Roboflow dataset / project to be used as target for collected data.
    • registration_tags (string): Tags to be attached to registered datapoints.
    • disable_sink (boolean): boolean flag that can be also reference to input - to arbitrarily disable data collection for specific request.
    • fire_and_forget (boolean): Boolean flag dictating if sink is supposed to be executed in the background, not waiting on status of registration before end of workflow run. Use True if best-effort registration is needed, use False while debugging and if error handling is needed.
    • labeling_batch_prefix (string): Prefix of the name for labeling batches that will be registered in Roboflow app.

  • output

    • error_status (boolean): Boolean flag.
    • message (string): String value.
Example JSON definition of step Roboflow Dataset Upload in version v1 
{
    "name": "<your_step_name_here>",
    "type": "roboflow_core/roboflow_dataset_upload@v1",
    "images": "$inputs.image",
    "predictions": "$steps.object_detection_model.predictions",
    "target_project": "my_dataset",
    "usage_quota_name": "quota-for-data-sampling-1",
    "persist_predictions": true,
    "minutely_usage_limit": 10,
    "hourly_usage_limit": 10,
    "daily_usage_limit": 10,
    "max_image_size": [
        512,
        512
    ],
    "compression_level": 75,
    "registration_tags": [
        "location-florida",
        "factory-name",
        "$inputs.dynamic_tag"
    ],
    "disable_sink": true,
    "fire_and_forget": true,
    "labeling_batch_prefix": "my_labeling_batch_name",
    "labeling_batches_recreation_frequency": "never"
}