Skip to main content
This guide walks you through creating a new mode for your robot system.

​
Project Structure

src/runtime/multi_mode/
├── config.py          # ModeConfig and ModeSystemConfig classes
├── manager.py         # ModeManager for transitions
├── cortex.py          # ModeCortexRuntime for execution
└── hook.py            # Lifecycle hooks

src/runtime/single_mode/
├── config.py          # ModeConfig and ModeSystemConfig classes
├── cortex.py           # ModeCortexRuntime for Execution

config/
└── your_robot_modes.json5    # Mode configuration file

​
Configuration

​
Step 1: Create Configuration File

Create or modify a configuration file (e.g., your_robot_modes.json5) in the /config/ directory.

​
Step 2: Add Mode Definition

Add your new mode to the modes section of your configuration file.
FieldTypeRequiredDescription
display_namestringYesThe human-readable name shown in the UI for this mode. Example: "Your New Mode"
descriptionstringYesBrief description explaining what this mode does and its purpose.
system_prompt_basestringYesThe foundational system prompt that defines the agent’s behavior and purpose in this mode.
hertzfloatYesThe frequency (in Hz) at which the agent operates or processes information. Example: 1.0
timeout_secondsintegerYesMaximum duration (in seconds) before the agent times out during execution. Example: 300
remember_locationsbooleanYesWhether the agent should persist and recall location data across interactions. Example: false
save_interactionsbooleanYesWhether to save conversation history and interactions for this mode. Example: true
agent_inputsarrayYesList of input sources or data types the agent can accept in this mode.
agent_actionsarrayYesList of actions or capabilities the agent can perform in this mode.
lifecycle_hooksarrayYesEvent handlers triggered at specific points in the agent’s lifecycle (startup, shutdown, etc.).
simulatorsarrayYesList of simulation environments or tools available to the agent in this mode.
cortex_llmobjectYesConfiguration object for the language model powering the agent’s cortex.

​
Step 3: Configure Input Plugins

Specify which inputs your mode needs:
FieldTypeRequiredDescription
typestringYesThe input type identifier. Example: "AudioInput"
configobjectNoConfiguration options specific to this input type.

​
Step 4: Configure LLM (Optional - Can be overwritten for each mode)

Define which LLM needs to be configured:
FieldTypeRequiredDescription
typestringYesThe LLM provider name. Example: "OpenAILLM"
configobjectNoConfiguration options specific to this LLM type.
agent_namestringNoAgent name used in metadata. Example: "Spot"
history_lengthintegerNoNumber of past messages to remember in the conversation. Example: 10

​
Step 5: Configure actions

Actions define what your agent can do. You can define movement, TTS or any other actions here.
FieldTypeRequiredDescription
namestringYesHuman-readable identifier for the action. Example: "speak"
llm_labelstringYesLabel the model uses to refer to this action. Example: "speak"
implementationstringNoDefines the business logic. If none defined, defaults to "passthrough". Example: "passthrough"
connectorstringYesName of the connector. This is the Python file name defined under actions/action_name/connector. Example: "elevenlabs_tts"
configobjectNoConfiguration options specific to this action.
exclude_from_promptbooleanNoWhether to exclude this action from the LLM prompt. Default: false

​
Step 6: Add Lifecycle hooks (Only required for multi-mode)

A hook is a programmable event point that executes specific actions at key stages. To define lifecycle hooks, you can add the following to your config.
FieldTypeRequiredDescription
hook_typestringYesThe lifecycle event type. Allowed values: "on_startup", "on_shutdown", "on_entry", "on_exit", "on_timeout"
handler_typestringYesThe type of handler to execute. Allowed values: "message", "command", "function", "action"
handler_configobjectYesConfiguration for the handler containing one of: message, command, function, or action as a string property
priorityintegerNoExecution priority when multiple hooks exist for the same event.
async_executionbooleanNoWhether to execute the handler asynchronously.
timeout_secondsnumberNoMaximum duration for handler execution.
on_failurestringNoBehavior when handler fails. Allowed values: "log", "ignore", "abort"

​
Step 7: Add Transition Rules (Only required for multi-mode)

Add to the transition_rules section:
FieldTypeRequiredDescription
from_modestringYesThe source mode from which the transition originates.
to_modestringYesThe target mode to which the agent will transition.
transition_typestringYesThe type of transition mechanism.
trigger_keywordsarrayYesList of keywords that trigger this transition. Each item is a string.
priorityintegerYesPriority level for this transition rule when multiple rules match.
cooldown_secondsnumberNoMinimum time (in seconds) before this transition can be triggered again.

​
Step 8: Update Default Mode (Optional)

If “new_mode” should be the starting mode, define 
"default_mode": "new_mode"
Now, your new mode is ready to be tested. Deploy it directly on your robot or configure it through the docker_compose file!