Workflows kinds¶

In Workflows, some values cannot be defined when the Workflow Definition is created. To address this, the Execution Engine supports selectors, which are references to step outputs or workflow inputs. To help the Execution Engine understand what type of data will be provided once a reference is resolved, we use a simple type system known as kinds.

Kinds are used to represent the semantic meaning of the underlying data. When a step outputs data of a specific kind and another step requires input of that same kind, the system assumes that the data will be compatible. This reduces the need for extensive type-compatibility checks.

For example, we have different kinds to distinguish between predictions from object detection and instance segmentation models, even though representation of those kinds is sv.Detections(...). This distinction ensures that each block that needs a segmentation mask clearly indicates this requirement, avoiding the need to repeatedly check for the presence of a mask in the input.

Note

As for now, kinds are such simplistic that do not support types polymorphism - and developers are asked to use unions of kinds to solve that problem. As defining extensive unions of kinds may be problematic, this problem will probably be addressed in Execution Engine v2.

Warning

In inference release 0.18.0 we decided to make drastic move to heal the ecosystem from the problem with ambiguous kinds names (Batch[X] vs X - see more here).

The change is breaking only if there is remote Workflow plugin depending on imports from inference.core.workflows.execution_engine.entities.types module, which is not the case to the best of our knowledge. We removed problematic kinds as if they never existed in the ecosystem and fixed all blocks from roboflow_core plugin. If there is anyone impacted by the change - here is the migration guide.

Kinds declared in Roboflow plugins¶

language_model_output: LLM / VLM output
image_keypoints: Image keypoints detected by classical Computer Vision method
qr_code_detection: Prediction with QR code detection
string: String value
*: Equivalent of any element
list_of_values: List of values of any type
rgb_color: RGB color
image_metadata: Dictionary with image metadata required by supervision
image: Image in workflows
classification_prediction: Predictions from classifier
float_zero_to_one: float value in range [0.0, 1.0]
roboflow_project: Roboflow project name
roboflow_api_key: Roboflow API key
parent_id: Identifier of parent for step output
roboflow_model_id: Roboflow model id
numpy_array: Numpy array
prediction_type: String value with type of prediction
video_metadata: Video image metadata
detection: Single element of detections-based prediction (like object_detection_prediction)
boolean: Boolean flag
top_class: String value representing top class predicted by classification model
dictionary: Dictionary
contours: List of numpy arrays where each array represents contour points
float: Float value
serialised_payloads: Serialised element that is usually accepted by sink
bar_code_detection: Prediction with barcode detection
integer: Integer value
instance_segmentation_prediction: Prediction with detected bounding boxes and segmentation masks in form of sv.Detections(...) object
zone: Definition of polygon zone
object_detection_prediction: Prediction with detected bounding boxes in form of sv.Detections(...) object
keypoint_detection_prediction: Prediction with detected bounding boxes and detected keypoints in form of sv.Detections(...) object
point: Single point in 2D