Environmental vaiables¶

Inference behavior can be controlled by set of environmental variables. All environmental variables are listed in inference/core/env.py

Below is a list of some environmental values that require more in-depth explanation.

Environmental variable	Description	Default
`ONNXRUNTIME_EXECUTION_PROVIDERS`	List of execution providers in priority order, warning message will be displayed if provider is not supported on user platform	See here
`SAM2_MAX_EMBEDDING_CACHE_SIZE`	The number of sam2 embeddings that will be held in memory. The embeddings will be held in gpu memory. Each embedding takes 16777216 bytes.	100
`SAM2_MAX_LOGITS_CACHE_SIZE`	The number of sam2 logits that will be held in memory. The the logits will be in cpu memory. Each logit takes 262144 bytes.	1000
`DISABLE_SAM2_LOGITS_CACHE`	If set to True, disables the caching of SAM2 logits. This can be useful for debugging or in scenarios where memory usage needs to be minimized, but may result in slower performance for repeated similar requests.	False
`ENABLE_WORKFLOWS_PROFILING`	If set to True, in `inference` server allows the server to output Workflows profiler traces the client, running in Python package with `InferencePipeline` it enables profiling.	False
`WORKFLOWS_PROFILER_BUFFER_SIZE`	Size of profiler buffer (number of consecutive Wrofklows Execution Engine `run(...)` invocations to trace in buffer.	64
`ENABLE_STREAM_API`	Flag to enable Stream Management API in `inference` server - see more.	False
`RUNS_ON_JETSON`	Boolean flag to tell if `inference` runs on Jetson device - set to `True` in all docker builds for Jetson architecture.	False
`WORKFLOWS_DEFINITION_CACHE_EXPIRY`	Number of seconds to cache Workflows definitions as a result of `get_workflow_specification(...)` function call	`15 * 60` - 15 minutes
`DOCKER_SOCKET_PATH`	Path to the local socket mounted to the container - by default empty, if provided - enables pooling docker container stats from the docker deamon socket. See more here	Not Set
`ENABLE_PROMETHEUS`	Boolean flag to enable Prometeus `/metrics` enpoint.	True for docker images in dockerhub