To test this feature, visit your live site.

Grove Vision AI V2 + WiFi + ROS Node

I recently got my hands on the Grove Vision AI V2 module, and I’d like to share some ideas with the community to get your feedback:

While the native OpenCat control via the camera module seems straightforward, I’m exploring ways to integrate this module into a larger framework. Specifically, I’d like to create a ROS node for the video stream from the sensor, enabling real-time observation on a local PC.

From what I’ve seen online, tutorials for streaming video over Wi-Fi with the module typically use an additional ESP32S3 alongside the Grove Vision AI V2 to enable Wi-Fi capabilities. However, I believe this extra module might be unnecessary since the BiBoard already has built-in Wi-Fi. It seems feasible to connect to Wi-Fi directly through the BiBoard, eliminating the need for the ESP32S3.

Furthermore, if the BiBoard is connected to Wi-Fi, it should be possible to run a ROS node directly on it. Right now, I’m using a Raspberry Pi to create a ROS node on the robot, but leveraging the BiBoard for this purpose would be a much more streamlined and elegant solution.

I’d love to hear your thoughts and suggestions on this approach!

4 comments

4 Comments

Rongzhong Li

I don't remember if I sent you guys the newest BiBoard V1: https://docs.petoi.com/biboard/biboard-v1-guide

It has a dedicated Pi socket as featured on the original NyBoard. It uses UART2 and can be opened by OpenCatEsp32's firmware via serial command "XS", which means eXtension Serial.

este este

Thanks for those additional details @Reid Graves! Sounds like a great project - I look forward to hearing more! 😎

este este

13h

I like this topic a lot!

In my limited past experience with ROS on a Raspberry Pi controlled robot, the challenge was getting all of the software parts to work properly together. In that case, ROS was added to an existing robot. This gave subtle incompatibilities between ROS, Linux and the underlying software that actually controlled the robot, which were all running on the Pi. It seems to me that those complexities can be minimized with Bittle as long as ROS and Linux reside on the Raspberry Pi and OpenCatEsp32 alone lives on the BiBoard.

That said, putting some form of ROS on the BiBoard itself is a very interesting idea. Micro-ROS is known to run on an ESP32-WROOM-32E chip (the BiBoard currently has a ESP32-WROOM-32D chip). However, it is unclear to me how one could put both Micro-ROS and OpenCatEsp32 both on the BiBoard since this introduces the same sort of complexities that, personally, I am now trying to avoid.

Your idea of introducing only a ROS node on the BiBoard then seems like the way to go. This has been done for Arduinos so a small modification to the OpenCatEsp32 source code should make this possible with minimal complexity. I agree that using the built in BiBoard WiFi to stream the camera should be possible.

BTW, I looked at your earlier posts but it wasn't clear to me how your Raspberry Pi is connected to the BiBoard of your Bittle. In one of your videos, it looked like it may be attached somehow above the BiBoard. Could you elaborate on the physical and communication connections between your Pi and the BiBoard on your robots?

Edited

Reid Graves

12h

Replying to

The way I worked around the OpenCat code on the microcontroller with ROS was by creating a ROS node on a raspberry pi that communicated through serial protocol to the microcontroller. I use either port ttyS0 or ttyAMA0. The node just sends commands such as krest, kwk, etc, just like you would do with the serialmaster.py script.

The way I connected it on the NyBoard was by soldiering a female header on the upper left corner of the board, and then plugging the raspberry pi in directly on top of it. I'd say the way to go is a Pi Zero 2W, as it is powerful enough to stream video, but doesn't add very much weight or require a separate voltage regulator hard wired into the power terminals (this is required for a Pi 4 or Pi 5).

For the Biboard, you can connect a raspberry pi with a grove connector plugged into the plug that tx2 and rx2 are located at, and then plugging the opposite ends into the tx and rx pins on the raspberry pi header, along with the ground and +5v wires. Then hook up your BiBoard to your computer, open the arduino ide serial monitor, and enter XS, and it should print out "Start Serial 2". To deactivate, type Xs.

Running ROS on the raspberry pi was achieved by creating a docker container with the same Ubuntu version as your ROS distribution requires (I've used Foxy on Ubuntu 20 as well as Humble on Ubuntu 22), then just running the ROS node from their, then it is in the same environment as your PC, where you can do more heavy AI algorithms on streamed video, visualize your video feed, and control the robot with your keyboard or USB video game controller.

But removing the raspberry pi completely would be the ideal scenario if video streaming from the smaller grove vision camera is possible, and I think it should be only a few lines of code to get a microROS node that receives commands from the ROS nodes on the PC and executes them. This would facilitate having path planning and decision making computations run on a more powerful computer, with the low level control remaining on the BiBoard.

Grove Vision AI V2 + WiFi + ROS Node

Win 11 & Mobile App BLE problems

Can't get UI.exe to run in Windows 11

Grove Vision AI V2 + WiFi + ROS Node