I have face tracking working for the Y directions but X is proving infintely more difficult.
I have tried a lot of different setups. The best I have got working is my current setup. I have the esp32 cam stream images to my laptop which uses cv2 and python. Then I wrote a custom script within this script to connect to bluetooth serial at the same time. In this way the cam will send images to my laptop which recognizes faces and then sends movement commands to nybble.
The MU sensor didn't do faces and the esp32 cam was so bad at detecting faces it was never going to function well, and the raspberry pi integration i tried for this same type of thing also was just too weak to detect faces well. So this is the solution I have and I'm happy with.
The issue now is that the up/down tracking works great but nybble has no head joint for moving left/right. This means I have to move the entire cats body. This is a difficult problem because I want the cat to remain sitting while doing the face tracking. Doing a full walk mode is just too hard and beyond the scope for me right now.
A big issue I'm facing is when I try to manually manipulate the joints with over the bluetooth serial, the auto balance will kick in and then move all the joints. I looked through the serial documentation a lot and I couldn't find any way to turn it off. the i command mentions something about restoring head control if it was done manually but I don't know what that refers to. Can you manually control the head somehow while it walks?
In any case I'm looking into gestures now and maybe that would let me do the final piece of this project? Not going to lie this project has required me to learn so many skills I never had before that I'm kinda feeling ran over. So any help would be great. The ideal would be the cat somehow spins left/right while seated and doesn't mess with the up/down very much for the camera while doing so. But at this point even just being able to move the cat left/right slowly is enough. I could do standing but I find when the cat stands the balancing takes over the head a lot which again I don't see any way to turn off. I did find the serial command to make the cat spin and it works but again it takes control of the head so I can't do the face tracking.
Thanks for any and all help. You pointed me in the proper direction several times and it alway saved me a lot of trial and error.
So an update I finally got time to finish the head tracking and it works soooo good. As long as I prop nybble in a way so that the neck rotation acts as an X axis. It's enough to show off the project so thanks for the help. I'll add some video and probably Tuesday to show it off!
I'm confused by "Nybble has no head joint for moving left/right." Nybble has both pan and tilt joints on the neck.
Could you post a picture of your Nybble?
If you wan to turn off the gyro, you can comment the definition sentence in the OpenCat.h as follows:
If you want to control the Head joint while walking, you can send a serial command 'i' first, then send the contro comand 'm 0 -30' ... etc. BTW, you must use the latest code on the GitHub.