OpenMind self-hosted VILA VLM API Reference. To streamline integration with the VILA VLM API, we offer a simple package for seamless interaction within OM1.

To minimize latency, the API endpoint utilizes WebSockets for efficient real-time communication.

wss://api-vila.openmind.org?api_key=<YOUR_API_KEY>

​
Installation

Install the OM1 package:

install OM1
uv pip3 install git+https://github.com/OpenmindAGI/OM1.git

If you don’t have uv installed, you can install it using the following command:

install OM1
pip3 install git+https://github.com/OpenmindAGI/OM1.git

​
Usage

The following example demonstrates how to interact with the VILA VLM API using the OM1 package:

usage
import time
from om1_utils import ws
from om1_vlm import VideoStream

# Initialize the VILA VLM API
ws_client = ws.Client(url="wss://api-vila.openmind.org?api_key=<YOUR_API_KEY>")
vlm = VideoStream(ws_client.send_message, fps=30)

# Start the VILA VLM API
ws_client.start()
vlm.start()

# Retrieve the VILA VLM API response
ws_client.register_message_callback(lambda msg: print(msg))

while True:
  time.sleep(1)

The expected response from the VILA VLM API will be in the following format:

response
{
  "vlm_reply": "The most interesting aspect in this series of images is the man's constant motion of speaking and looking in different directions while sitting in front of a laptop."
}

Was this page helpful?