These examples are similar to the Hello World (Spot) example, except they use DeepSeek, Gemini or Grok rather than OpenAI 4o.

LLM Models
uv run src/run.py deepseek
uv run src/run.py gemini
uv run src/run.py grok

​
Emotion Detection with DeepSeek

This example process emotion detection and a simulated environment. It will request you to accept a prompt to on your camera, this is what it uses to capture your emotions.

​
Command

uv run src/run.py deepseek

​
Code Explanation

​
Initialization Issues

WARNING:root: Unitree SDK not found. Please install the Unitree SDK to use this plugin.
  • Issue: The Unitree SDK (likely for robot control) is missing.
  • Impact: If using a robotic or hardware-based system, some features may not work.

​
Camera Initialization Warning

2025-02-22 09:04:11.416 Python[4112:79434] WARNING: AVCaptureDeviceTypeExternal is deprecated for Continuity Cameras. 
Please use AVCaptureDeviceTypeContinuityCamera and add NSCameraUseContinuityCameraDeviceType to your Info.plist.
  • Issue: A deprecated camera type is being used.
  • Solution: Update the configuration to use AVCaptureDeviceTypeContinuityCamera instead.

​
WebSim (Simulation Environment) Setup

INFO:root:Found cam(0)
INFO:root:Initializing WebSim...
INFO:root:Starting WebSim server thread...
INFO:root:WebSim server started successfully - Open http://localhost:8000 in your browser
  • A web-based simulation environment (WebSim) is successfully launched.
  • Can be accessed at http://localhost:8000 for further interaction.

​
Emotion Detection System Running

INFO:root:EmotionCapture: I do not see anyone, so I can't estimate their emotion.
INFO:root:EmotionCapture: I see a person. Their emotion is angry.
INFO:root:EmotionCapture: I see a person. Their emotion is happy.
INFO:root:EmotionCapture: I see a person. Their emotion is neutral.
  • The system is detecting people’s emotions using computer vision.
  • Initially, no person was detected, then it recognized emotions like angry, happy, and neutral.

​
API Authentication Error (DeepSeek LLM Failing)

INFO:httpx:HTTP Request: POST https://api.openmind.org/api/core/deepseek/chat/completions "HTTP/1.1 401 Unauthorized"
ERROR:root:Error asking LLM: Error code: 401 - {'error': 'API key not found'}
WARNING:root:No output from LLM
  • Issue: The system is trying to send a request to DeepSeek AI but gets a 401 Unauthorized error.
  • Cause:
  • API key is missing or incorrect.
  • Authentication headers not set properly.
  • Impact: No response from the AI, so no chatbot functionality.
  • Fix: Add a valid API key to the system’s configuration.
  • Summary of Key Issues & Fixes

​
Emotion Detection with Gemini

This example process emotion detection and a simulated environment. It will request you to accept a prompt to on your camera, this is what it uses to capture your emotions.

​
Command

uv run src/run.py gemini

​
Response

WARNING:root:Unitree SDK not found. Please install the Unitree SDK to use this plugin.
WARNING:root:Unitree SDK not found. Please install the Unitree SDK to use this plugin.
INFO:root:VLM Gemini Provider initialized
INFO:om1_vlm:Started video processing thread
INFO:root:VLM Gemini Provider started
2025-02-22 09:08:06.524 Python[4215:83654] WARNING: AVCaptureDeviceTypeExternal is deprecated for Continuity Cameras. Please use AVCaptureDeviceTypeContinuityCamera and add NSCameraUseContinuityCameraDeviceType to your Info.plist.
INFO:root:Found cam(0)
INFO:root:Initializing Gemini OpenAI client with {'base_url': 'https://api.openmind.org/api/core/gemini', 'api_key': 'openmind_free'}
INFO:root:Initializing WebSim...
INFO:root:Starting WebSim server thread...
OpenCV: out device of bound (0-1): 2
OpenCV: camera failed to properly initialize!
INFO:om1_vlm:Using camera: 0
INFO:root:WebSim server started successfully - Open http://localhost:8000 in your browser
INFO:root:EmotionCapture: I do not see anyone, so I can't estimate their emotion.
INFO:httpx:HTTP Request: POST https://api.openmind.org/api/core/gemini/chat/completions "HTTP/1.1 401 Unauthorized"
ERROR:root:Error processing frame: Error code: 401 - {'error': 'API key not found'}
INFO:root:EmotionCapture: I see a person. Their emotion is happy.
INFO:httpx:HTTP Request: POST https://api.openmind.org/api/core/gemini/chat/completions "HTTP/1.1 401 Unauthorized"
ERROR:root:Error processing frame: Error code: 401 - {'error': 'API key not found'}
INFO:root:EmotionCapture: I see a person. Their emotion is happy.
INFO:httpx:HTTP Request: POST https://api.openmind.org/api/core/gemini/chat/completions "HTTP/1.1 401 Unauthorized"
ERROR:root:Gemini API error: Error code: 401 - {'error': 'API key not found'}
WARNING:root:No output from LLM
INFO:httpx:HTTP Request: POST https://api.openmind.org/api/core/gemini/chat/completions "HTTP/1.1 401 Unauthorized"
ERROR:root:Error processing frame: Error code: 401 - {'error': 'API key not found'}
INFO:root:EmotionCapture: I see a person. Their emotion is happy.
INFO:root:EmotionCapture: I see a person. Their emotion is happy.
INFO:httpx:HTTP Request: POST https://api.openmind.org/api/core/gemini/chat/completions "HTTP/1.1 401 Unauthorized"
ERROR:root:Gemini API error: Error code: 401 - {'error': 'API key not found'}
WARNING:root:No output from LLM
INFO:httpx:HTTP Request: POST https://api.openmind.org/api/core/gemini/chat/completions "HTTP/1.1 401 Unauthorized"
ERROR:root:Error processing frame: Error code: 401 - {'error': 'API key not found'}
INFO:root:EmotionCapture: I see a person. Their emotion is angry.
INFO:root:EmotionCapture: I see a person. Their emotion is neutral.
INFO:httpx:HTTP Request: POST https://api.openmind.org/api/core/gemini/chat/completions "HTTP/1.1 401 Unauthorized"
ERROR:root:Error processing frame: Error code: 401 - {'error': 'API key not found'}
INFO:httpx:HTTP Request: POST https://api.openmind.org/api/core/gemini/chat/completions "HTTP/1.1 401 Unauthorized"
ERROR:root:Gemini API error: Error code: 401 - {'error': 'API key not found'}
WARNING:root:No output from LLM
INFO:root:EmotionCapture: I see a person. Their emotion is neutral.
INFO:root:EmotionCapture: I see a person. Their emotion is neutral.
INFO:httpx:HTTP Request: POST https://api.openmind.org/api/core/gemini/chat/completions "HTTP/1.1 401 Unauthorized"
ERROR:root:Error processing frame: Error code: 401 - {'error': 'API key not found'}
INFO:httpx:HTTP Request: POST https://api.openmind.org/api/core/gemini/chat/completions "HTTP/1.1 401 Unauthorized"
ERROR:root:Gemini API error: Error code: 401 - {'error': 'API key not found'}
WARNING:root:No output from LLM
INFO:root:EmotionCapture: I see a person. Their emotion is neutral.
INFO:httpx:HTTP Request: POST https://api.openmind.org/api/core/gemini/chat/completions "HTTP/1.1 401 Unauthorized"
ERROR:root:Error processing frame: Error code: 401 - {'error': 'API key not found'}
INFO:root:EmotionCapture: I see a person. Their emotion is sad.
INFO:root:EmotionCapture: I see a person. Their emotion is sad.
INFO:httpx:HTTP Request: POST https://api.openmind.org/api/core/gemini/chat/completions "HTTP/1.1 401 Unauthorized"
ERROR:root:Gemini API error: Error code: 401 - {'error': 'API key not found'}
WARNING:root:No output from LLM
INFO:httpx:HTTP Request: POST https://api.openmind.org/api/core/gemini/chat/completions "HTTP/1.1 401 Unauthorized"
ERROR:root:Error processing frame: Error code: 401 - {'error': 'API key not found'}
INFO:root:EmotionCapture: I see a person. Their emotion is sad.
INFO:root:EmotionCapture: I see a person. Their emotion is sad.

You can directly access other OpenAI style endpoints by specifying a custom API endpoint in your configuration file. To do this, provide an suitable base_url and the api_key for OpenAI, DeepSeek, or other providers. Possible base_url choices are:

  • and many others…