Skip to main content

​
Configuration

Agents are configured via JSON5 files in the /config directory. The configuration file is used to define the LLM system prompt, agent’s inputs, LLM configuration, and actions etc. Here is an example of the configuration file: 
{
  "hertz": 0.5,
  "name": "agent_name",
  "api_key": "openmind_free",
  "URID": "default",
  "system_prompt_base": "...",
  "system_governance": "...",
  "system_prompt_examples": "...",
  "agent_inputs": [
    {
      "type": "GovernanceEthereum"
    },
    {
      "type": "VLM_COCO_Local",
      "config": {
        "camera_index": 0
      }
    }
  ],
  "cortex_llm": {
    "type": "OpenAILLM",
    "config": {
      "base_url": "",       // Optional: URL of the LLM endpoint
      "agent_name": "Iris", // Optional: Name of the agent
      "history_length": 10
    }
  },
  "simulators": [
    {
      "type": "WebSim",
      "config": {
        "host": "0.0.0.0",
        "port": 8000,
        "tick_rate": 100,
        "auto_reconnect": true,
        "debug_mode": false
      }
    }
  ],
  "agent_actions": [
    {
      "name": "move",
      "llm_label": "move",
      "implementation": "passthrough",
      "connector": "ros2"
    },
    {
      "name": "speak",
      "llm_label": "speak",
      "implementation": "passthrough",
      "connector": "ros2"
    }
  ]
}

​
Common Configuration Elements

  • hertz Defines the base tick rate of the agent. This rate can be adjusted to allow the agent to respond quickly to changing environments, but comes at the expense of reducing the time available for LLLms to finish generating tokens. Note: time critical tasks such as collision avoidance should be handled through low level control loops operating in parallel to the LLM-based logic, using event-triggered callbacks through real-time middleware.
  • name A unique identifier for the agent.
  • api_key The API key for the agent. You can get your API key from the OpenMind Portal.
  • URID The Universal Robot ID for the robot. Used to join a decentralized machine-to-machine coordination and communication system (FABRIC).
  • system_prompt_base Defines the agent’s personality and behavior.
  • system_governance The agent’s laws and constitution.
  • system_prompt_examples The agent’s example inputs/actions.

​
Agent Inputs (agent_inputs)

Example configuration for the agent_inputs section: 
  "agent_inputs": [
    {
      "type": "GovernanceEthereum"
    },
    {
      "type": "VLM_COCO_Local",
      "config": {
        "camera_index": 0
      }
    }
  ]
The agent_inputs section defines the inputs for the agent. Inputs might include a camera, a LiDAR, a microphone, or governance information. OM1 implements the following input types:
  • GoogleASRInput
  • VLMVila
  • VLM_COCO_Local
  • RPLidar
  • TurtleBot4Batt
  • UnitreeG1Basic
  • UnitreeGo2Lowstate
  • GovernanceEthereum
  • more being added continuously…
You can implement your own inputs by following the Input Plugin Guide. The agent_inputs config section is specific to each input type. For example, the VLM_COCO_Local input accepts a camera_index parameter.

​
Cortex LLM (cortex_llm)

The cortex_llm field allow you to configure the Large Language Model (LLM) used by the agent. In a typical deployment, data will flow to at least three different LLMs, hosted in the cloud, that work together to provide actions to your robot.

​
Robot Control by a Single LLM

Here is an example configuration of the cortex_llm showing use of a single LLM to generate decisions: 
  "cortex_llm": {
    "type": "OpenAILLM",
    "config": {
      "base_url": "",       // Optional: URL of the LLM endpoint
      "api_key": "...",     // Optional: Override the default API key
      "agent_name": "Iris", // Optional: Name of the agent
      "history_length": 10
    }
  }
  • type: Specifies the LLM plugin.
  • config: LLM configuration, including the API endpoint (base_url), agent_name, and history_length.
You can directly access other OpenAI style endpoints by specifying a custom API endpoint in your configuration file. To do this, provide a suitable base_url and the api_key for OpenAI, DeepSeek, or other providers. Possible base_url choices include: You can implement your own LLM endpoints or use more sophisticated approaches such as multiLLM robotics-focused endpoints by following the LLM Guide.

​
Simulators (simulators)

Lists the simulation modules used by the agent. Here is an example configuration for the simulators section: 
  "simulators": [
    {
      "type": "WebSim",
      "config": {
        "host": "0.0.0.0",
        "port": 8000,
        "tick_rate": 100,
        "auto_reconnect": true,
        "debug_mode": false
      }
    }
  ]

​
Agent Actions (agent_actions)

Defines the agent’s available capabilities, including action names, their implementation, and the connector used to execute them. Here is an example configuration for the agent_actions section: 
  "agent_actions": [
    {
      "name": "move",
      "llm_label": "move",
      "implementation": "passthrough",
      "connector": "ros2"
    },
    {
      "name": "speak",
      "llm_label": "speak",
      "implementation": "passthrough",
      "connector": "ros2"
    }
  ]
You can customize the actions following the Action Plugin Guide
⌘I