Skip to content

Inference

ClassificationInferenceResponse

Bases: CvInferenceResponse, WithVisualizationResponse

Classification inference response.

Attributes:

Name Type Description
predictions List[ClassificationPrediction]

List of classification predictions.
top str

The top predicted class label.
confidence float

The confidence of the top predicted class label.
Source code in inference/core/entities/responses/inference.py 
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
class ClassificationInferenceResponse(CvInferenceResponse, WithVisualizationResponse):
    """Classification inference response.

    Attributes:
        predictions (List[inference.core.entities.responses.inference.ClassificationPrediction]): List of classification predictions.
        top (str): The top predicted class label.
        confidence (float): The confidence of the top predicted class label.
    """

    predictions: List[ClassificationPrediction]
    top: str = Field(
        description="The top predicted class label", default=""
    )  # Not making this field optional to avoid breaking change - in other parts of the codebase `model_dump` is called with `exclude_none=True`
    confidence: float = Field(
        description="The confidence of the top predicted class label",
        default=0.0,
    )
    parent_id: Optional[str] = Field(
        description="Identifier of parent image region. Useful when stack of detection-models is in use to refer the RoI being the input to inference",
        default=None,
    )

ClassificationPrediction

Bases: BaseModel

Classification prediction.

Attributes:

Name Type Description
class_name str

The predicted class label.
class_id int

Numeric ID associated with the class label.
confidence float

The class label confidence as a fraction between 0 and 1.
Source code in inference/core/entities/responses/inference.py 
120
121
122
123
124
125
126
127
128
129
130
131
132
133
class ClassificationPrediction(BaseModel):
    """Classification prediction.

    Attributes:
        class_name (str): The predicted class label.
        class_id (int): Numeric ID associated with the class label.
        confidence (float): The class label confidence as a fraction between 0 and 1.
    """

    class_name: str = Field(alias="class", description="The predicted class label")
    class_id: int = Field(description="Numeric ID associated with the class label")
    confidence: float = Field(
        description="The class label confidence as a fraction between 0 and 1"
    )

CvInferenceResponse

Bases: InferenceResponse

Computer Vision inference response.

Attributes:

Name Type Description
image Union[List[InferenceResponseImage], InferenceResponseImage]

Image(s) used in inference.
Source code in inference/core/entities/responses/inference.py 
186
187
188
189
190
191
192
193
class CvInferenceResponse(InferenceResponse):
    """Computer Vision inference response.

    Attributes:
        image (Union[List[inference.core.entities.responses.inference.InferenceResponseImage], inference.core.entities.responses.inference.InferenceResponseImage]): Image(s) used in inference.
    """

    image: Union[List[InferenceResponseImage], InferenceResponseImage]

DepthEstimationResponse

Bases: BaseModel

Response for depth estimation inference.

Attributes:

Name Type Description
normalized_depth List[List[float]]

The normalized depth map as a 2D array of floats between 0 and 1.
image Optional[str]

Base64 encoded visualization of the depth map if visualize_predictions is True.
time float

The processing time in seconds.
visualization Optional[str]

Base64 encoded visualization of the depth map if visualize_predictions is True.
Source code in inference/core/entities/responses/inference.py 
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
class DepthEstimationResponse(BaseModel):
    """Response for depth estimation inference.

    Attributes:
        normalized_depth (List[List[float]]): The normalized depth map as a 2D array of floats between 0 and 1.
        image (Optional[str]): Base64 encoded visualization of the depth map if visualize_predictions is True.
        time (float): The processing time in seconds.
        visualization (Optional[str]): Base64 encoded visualization of the depth map if visualize_predictions is True.
    """

    normalized_depth: List[List[float]] = Field(
        description="The normalized depth map as a 2D array of floats between 0 and 1"
    )
    image: Optional[str] = Field(
        None,
        description="Base64 encoded visualization of the depth map if visualize_predictions is True",
    )

FaceDetectionPrediction

Bases: ObjectDetectionPrediction

Face Detection prediction.

Attributes:

Name Type Description
class_name str

fixed value "face".
landmarks Union[List[Point], List[Point3D]]

The detected face landmarks.
Source code in inference/core/entities/responses/inference.py 
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
class FaceDetectionPrediction(ObjectDetectionPrediction):
    """Face Detection prediction.

    Attributes:
        class_name (str): fixed value "face".
        landmarks (Union[List[inference.core.entities.responses.inference.Point], List[inference.core.entities.responses.inference.Point3D]]): The detected face landmarks.
    """

    class_id: Optional[int] = Field(
        description="The class id of the prediction", default=0
    )
    class_name: str = Field(
        alias="class", default="face", description="The predicted class label"
    )
    landmarks: Union[List[Point], List[Point3D]]

InferenceResponse

Bases: BaseModel

Base inference response.

Attributes:

Name Type Description
inference_id Optional[str]

Unique identifier of inference
frame_id Optional[int]

The frame id of the image used in inference if the input was a video.
time Optional[float]

The time in seconds it took to produce the predictions including image preprocessing.
Source code in inference/core/entities/responses/inference.py 
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
class InferenceResponse(BaseModel):
    """Base inference response.

    Attributes:
        inference_id (Optional[str]): Unique identifier of inference
        frame_id (Optional[int]): The frame id of the image used in inference if the input was a video.
        time (Optional[float]): The time in seconds it took to produce the predictions including image preprocessing.
    """

    model_config = ConfigDict(protected_namespaces=())
    inference_id: Optional[str] = Field(
        description="Unique identifier of inference", default=None
    )
    frame_id: Optional[int] = Field(
        default=None,
        description="The frame id of the image used in inference if the input was a video",
    )
    time: Optional[float] = Field(
        default=None,
        description="The time in seconds it took to produce the predictions including image preprocessing",
    )

InferenceResponseImage

Bases: BaseModel

Inference response image information.

Attributes:

Name Type Description
width int

The original width of the image used in inference.
height int

The original height of the image used in inference.
Source code in inference/core/entities/responses/inference.py 
149
150
151
152
153
154
155
156
157
158
159
160
class InferenceResponseImage(BaseModel):
    """Inference response image information.

    Attributes:
        width (int): The original width of the image used in inference.
        height (int): The original height of the image used in inference.
    """

    width: int = Field(description="The original width of the image used in inference")
    height: int = Field(
        description="The original height of the image used in inference"
    )

InstanceSegmentationInferenceResponse

Bases: CvInferenceResponse, WithVisualizationResponse

Instance Segmentation inference response.

Attributes:

Name Type Description
predictions List[InstanceSegmentationPrediction]

List of instance segmentation predictions.
Source code in inference/core/entities/responses/inference.py 
243
244
245
246
247
248
249
250
251
252
class InstanceSegmentationInferenceResponse(
    CvInferenceResponse, WithVisualizationResponse
):
    """Instance Segmentation inference response.

    Attributes:
        predictions (List[inference.core.entities.responses.inference.InstanceSegmentationPrediction]): List of instance segmentation predictions.
    """

    predictions: List[InstanceSegmentationPrediction]

InstanceSegmentationPrediction

Bases: BaseModel

Instance Segmentation prediction.

Attributes:

Name Type Description
x float

The center x-axis pixel coordinate of the prediction.
y float

The center y-axis pixel coordinate of the prediction.
width float

The width of the prediction bounding box in number of pixels.
height float

The height of the prediction bounding box in number of pixels.
confidence float

The detection confidence as a fraction between 0 and 1.
class_name str

The predicted class label.
class_confidence Union[float, None]

The class label confidence as a fraction between 0 and 1.
points List[Point]

The list of points that make up the instance polygon.
class_id int

int = Field(description="The class id of the prediction")
Source code in inference/core/entities/responses/inference.py 
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
class InstanceSegmentationPrediction(BaseModel):
    """Instance Segmentation prediction.

    Attributes:
        x (float): The center x-axis pixel coordinate of the prediction.
        y (float): The center y-axis pixel coordinate of the prediction.
        width (float): The width of the prediction bounding box in number of pixels.
        height (float): The height of the prediction bounding box in number of pixels.
        confidence (float): The detection confidence as a fraction between 0 and 1.
        class_name (str): The predicted class label.
        class_confidence (Union[float, None]): The class label confidence as a fraction between 0 and 1.
        points (List[Point]): The list of points that make up the instance polygon.
        class_id: int = Field(description="The class id of the prediction")
    """

    x: float = Field(description="The center x-axis pixel coordinate of the prediction")
    y: float = Field(description="The center y-axis pixel coordinate of the prediction")
    width: float = Field(
        description="The width of the prediction bounding box in number of pixels"
    )
    height: float = Field(
        description="The height of the prediction bounding box in number of pixels"
    )
    confidence: float = Field(
        description="The detection confidence as a fraction between 0 and 1"
    )
    class_name: str = Field(alias="class", description="The predicted class label")

    class_confidence: Union[float, None] = Field(
        None, description="The class label confidence as a fraction between 0 and 1"
    )
    points: List[Point] = Field(
        description="The list of points that make up the instance polygon"
    )
    class_id: int = Field(description="The class id of the prediction")
    detection_id: str = Field(
        description="Unique identifier of detection",
        default_factory=lambda: str(uuid4()),
    )
    parent_id: Optional[str] = Field(
        description="Identifier of parent image region. Useful when stack of detection-models is in use to refer the RoI being the input to inference",
        default=None,
    )

MultiLabelClassificationInferenceResponse

Bases: CvInferenceResponse, WithVisualizationResponse

Multi-label Classification inference response.

Attributes:

Name Type Description
predictions Dict[str, MultiLabelClassificationPrediction]

Dictionary of multi-label classification predictions.
predicted_classes List[str]

The list of predicted classes.
Source code in inference/core/entities/responses/inference.py 
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
class MultiLabelClassificationInferenceResponse(
    CvInferenceResponse, WithVisualizationResponse
):
    """Multi-label Classification inference response.

    Attributes:
        predictions (Dict[str, inference.core.entities.responses.inference.MultiLabelClassificationPrediction]): Dictionary of multi-label classification predictions.
        predicted_classes (List[str]): The list of predicted classes.
    """

    predictions: Dict[str, MultiLabelClassificationPrediction]
    predicted_classes: List[str] = Field(description="The list of predicted classes")
    parent_id: Optional[str] = Field(
        description="Identifier of parent image region. Useful when stack of detection-models is in use to refer the RoI being the input to inference",
        default=None,
    )

MultiLabelClassificationPrediction

Bases: BaseModel

Multi-label Classification prediction.

Attributes:

Name Type Description
confidence float

The class label confidence as a fraction between 0 and 1.
Source code in inference/core/entities/responses/inference.py 
136
137
138
139
140
141
142
143
144
145
146
class MultiLabelClassificationPrediction(BaseModel):
    """Multi-label Classification prediction.

    Attributes:
        confidence (float): The class label confidence as a fraction between 0 and 1.
    """

    confidence: float = Field(
        description="The class label confidence as a fraction between 0 and 1"
    )
    class_id: int = Field(description="Numeric ID associated with the class label")

ObjectDetectionInferenceResponse

Bases: CvInferenceResponse, WithVisualizationResponse

Object Detection inference response.

Attributes:

Name Type Description
predictions List[ObjectDetectionPrediction]

List of object detection predictions.
Source code in inference/core/entities/responses/inference.py 
215
216
217
218
219
220
221
222
class ObjectDetectionInferenceResponse(CvInferenceResponse, WithVisualizationResponse):
    """Object Detection inference response.

    Attributes:
        predictions (List[inference.core.entities.responses.inference.ObjectDetectionPrediction]): List of object detection predictions.
    """

    predictions: List[ObjectDetectionPrediction]

ObjectDetectionPrediction

Bases: BaseModel

Object Detection prediction.

Attributes:

Name Type Description
x float

The center x-axis pixel coordinate of the prediction.
y float

The center y-axis pixel coordinate of the prediction.
width float

The width of the prediction bounding box in number of pixels.
height float

The height of the prediction bounding box in number of pixels.
confidence float

The detection confidence as a fraction between 0 and 1.
class_name str

The predicted class label.
class_confidence Union[float, None]

The class label confidence as a fraction between 0 and 1.
class_id int

The class id of the prediction
Source code in inference/core/entities/responses/inference.py 
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
class ObjectDetectionPrediction(BaseModel):
    """Object Detection prediction.

    Attributes:
        x (float): The center x-axis pixel coordinate of the prediction.
        y (float): The center y-axis pixel coordinate of the prediction.
        width (float): The width of the prediction bounding box in number of pixels.
        height (float): The height of the prediction bounding box in number of pixels.
        confidence (float): The detection confidence as a fraction between 0 and 1.
        class_name (str): The predicted class label.
        class_confidence (Union[float, None]): The class label confidence as a fraction between 0 and 1.
        class_id (int): The class id of the prediction
    """

    x: float = Field(description="The center x-axis pixel coordinate of the prediction")
    y: float = Field(description="The center y-axis pixel coordinate of the prediction")
    width: float = Field(
        description="The width of the prediction bounding box in number of pixels"
    )
    height: float = Field(
        description="The height of the prediction bounding box in number of pixels"
    )
    confidence: float = Field(
        description="The detection confidence as a fraction between 0 and 1"
    )
    class_name: str = Field(alias="class", description="The predicted class label")

    class_confidence: Union[float, None] = Field(
        None, description="The class label confidence as a fraction between 0 and 1"
    )
    class_id: int = Field(description="The class id of the prediction")
    tracker_id: Optional[int] = Field(
        description="The tracker id of the prediction if tracking is enabled",
        default=None,
    )
    detection_id: str = Field(
        description="Unique identifier of detection",
        default_factory=lambda: str(uuid4()),
    )
    parent_id: Optional[str] = Field(
        description="Identifier of parent image region. Useful when stack of detection-models is in use to refer the RoI being the input to inference",
        default=None,
    )

Point

Bases: BaseModel

Point coordinates.

Attributes:

Name Type Description
x float

The x-axis pixel coordinate of the point.
y float

The y-axis pixel coordinate of the point.
Source code in inference/core/entities/responses/inference.py 
53
54
55
56
57
58
59
60
61
62
class Point(BaseModel):
    """Point coordinates.

    Attributes:
        x (float): The x-axis pixel coordinate of the point.
        y (float): The y-axis pixel coordinate of the point.
    """

    x: float = Field(description="The x-axis pixel coordinate of the point")
    y: float = Field(description="The y-axis pixel coordinate of the point")

Point3D

Bases: Point

3D Point coordinates.

Attributes:

Name Type Description
z float

The z-axis pixel coordinate of the point.
Source code in inference/core/entities/responses/inference.py 
65
66
67
68
69
70
71
72
class Point3D(Point):
    """3D Point coordinates.

    Attributes:
        z (float): The z-axis pixel coordinate of the point.
    """

    z: float = Field(description="The z-axis pixel coordinate of the point")

WithVisualizationResponse

Bases: BaseModel

Response with visualization.

Attributes:

Name Type Description
visualization Optional[Any]

Base64 encoded string containing prediction visualization image data.
Source code in inference/core/entities/responses/inference.py 
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
class WithVisualizationResponse(BaseModel):
    """Response with visualization.

    Attributes:
        visualization (Optional[Any]): Base64 encoded string containing prediction visualization image data.
    """

    visualization: Optional[Any] = Field(
        default=None,
        description="Base64 encoded string containing prediction visualization image data",
    )

    @field_serializer("visualization", when_used="json")
    def serialize_visualisation(self, visualization: Optional[Any]) -> Optional[str]:
        if visualization is None:
            return None
        return base64.b64encode(visualization).decode("utf-8")