Over UDP

You can run Inference directly on frames using UDP.

This is ideal for real-time use cases where reducing latency is essential (i.e. sports broadcasting).

This feature only works on devices with a CUDA-enabled GPU.

Inference has been used at sports broadcasting events around the world for real-time object detection.

Run a Vision Model on a UDP Stream¶

To run inference on frames from a UDP stream, you will need to:

1. Set up a listening server to receive predictions from Inference, and;
2. Run Inference, connected directly to your stream.

Authenticate with Roboflow¶

To use Inference with a UDP stream, you will need a Roboflow API key. If you don't already have a Roboflow account, sign up for a free Roboflow account. Then, retrieve your API key from the Roboflow dashboard. Run the following command to set your API key in your coding environment:

export API_KEY=<your api key>

Configure a Listening Server¶

You need a server to receive predictions from Inference. This server is where you can write custom logic to process predictions.

Create a new Python file and add the following code:

import socket
import json
import time

    fps_array = []

    # Create a datagram (UDP) socket
    UDPClientSocket = socket.socket(family=socket.AF_INET, type=socket.SOCK_DGRAM)

    # Bind to the given IP address and port
    UDPClientSocket.bind((ip, port))

    print(f"UDP server up and listening on http://{ip}:{port}")

    # Listen for incoming datagrams
    while True:
        t0 = time.time()

        bytesAddressPair = UDPClientSocket.recvfrom(1024)
        message = bytesAddressPair[0]
        address = bytesAddressPair[1]

        clientMsg = json.loads(message)
        clientIP = "Client IP Address:{}".format(address)

        print(clientMsg)
        print(clientIP)

        t = time.time() - t0
        fps_array.append(1 / t)
        fps_array[-150:]
        fps_average = sum(fps_array) / len(fps_array)
        print("AVERAGE FPS: " + str(fps_average))

start_udp_server("localhost", arguments.port)

Above, replace PORT with the port on which you want to run your server.

Run a Broadcasting Server¶

• set up socket
• render will broadcast
• https://hub.docker.com/repository/docker/roboflow/roboflow-inference-server-udp-gpu/general