Please help me: Sending String from PyCharm via usb to NXT, which is programmed by Bricx CC

[BlackWhiteWallpaper]

Body

Hello guys,I am new here and hope for some help.

So, I am doing a school project right now and we are working with Lego Mindstorms NXT and write the program with the Bricx Command Center. I know that this is pretty old but maybe some of you have tips or ideas.
So my idea was to create the game rock paper scissors but in a way the player never wins. To do that I worked with the Mediapipe Hand gesture recognition I am running in PyCharm which works great. My webcam analyses the hand gesture. But we have also to work with the Bricx Command center which works like this: i built a lego hand and the thumb, index finger and the rest of the fingers are connected to 3 motors. With those I can make the opposite gesture to the gesture the player made. So when the programm analyses the players gesture I want it to send a string via usb to the NXT. The nxt knows the string and makes the corresponding gesture. But thats exactly my problem because I also need a program written in the Bricx Command Center which can receive the string I send from PyCharm via usb. Are there any ideas or solutions how i could do that? It would be very helpful. I tried many things also asking ChatGPT but as we all know this isnt often that helpful.

Guidelines

• I have read and understood this category's guidelines before making this post.

[DevJSter]

I think this might help you

1. Set Up Serial Communication in PyCharm

First, you'll need to establish a serial communication link between your computer and the NXT. For this, you can use the pyserial library in Python.

Install pyserial

You can install pyserial using pip:

pip install pyserial

Python Code to Send Data

Here's an example of how you can send a string from Python to the NXT:

import serial
import time

# Replace 'COM3' with the port your NXT is connected to
ser = serial.Serial('COM3', 9600)  # Adjust the COM port and baud rate as needed
time.sleep(2)  # Give some time for the connection to establish

def send_command(command):
    ser.write(command.encode())  # Encode the string to bytes and send
    time.sleep(1)  # Give some time for the NXT to process

# Example usage
gesture = "rock"  # This should be the gesture recognized by Mediapipe
send_command(gesture)

ser.close()

2. Receive Data on NXT using Bricx Command Center

Next, you need to write a program in NXC (Not eXactly C) that runs on the NXT and receives the string via USB.

NXC Code to Receive Data

Here's an example of how to receive and act on the string in NXC:

task main()
{
    string command;
    SetSerialPort(S1);  // Set serial port to port 1

    while(true)
    {
        if (SerialDataAvailable())  // Check if data is available
        {
            ReceiveSerialString(command);  // Receive the data as a string

            // Act on the received command
            if (command == "rock")
            {
                // Code to make the corresponding gesture for "rock"
                TextOut(0, LCD_LINE1, "Received: rock");
                // Control motors here
            }
            else if (command == "paper")
            {
                TextOut(0, LCD_LINE1, "Received: paper");
                // Control motors here
            }
            else if (command == "scissors")
            {
                TextOut(0, LCD_LINE1, "Received: scissors");
                // Control motors here
            }
            else
            {
                TextOut(0, LCD_LINE1, "Unknown command");
            }
        }

        Wait(100);  // Add a small delay to prevent overloading the CPU
    }
}

3. Combine and Test

1. Connect the NXT to your computer via USB.
2. Run your Python script from PyCharm to send the gesture command.
3. Ensure the NXT program is running and is ready to receive the command.

Tips

• Ensure the COM port and baud rate match between the Python script and the NXT settings.
• Test with simple commands first to ensure communication is established.
• Debug using print statements or the NXT's display to verify that commands are received correctly.

If you encounter any specific issues, feel free to ask for more detailed troubleshooting!

Hope this answeres your question #128675 then Mark it as answered

[BlackWhiteWallpaper]

Thank you, but sadly it doesnt work...I get many compile errors... @DevJSter

[DevJSter]

For your project, you'll need to set up USB communication between your Python program (which uses Mediapipe for gesture recognition) and your NXT brick (which will be controlled using the Bricx Command Center). Since direct USB communication with NXT using Bricx Command Center is challenging, we can use a workaround with Python to control the NXT motors via USB.

import nxt.locator
from nxt.motor import Motor, PORT_A, PORT_B, PORT_C
import time

def send_command(gesture):
    # Connect to NXT brick
    brick = nxt.locator.find_one_brick()
    
    m_thumb = Motor(brick, PORT_A)
    m_index = Motor(brick, PORT_B)
    m_others = Motor(brick, PORT_C)

    if gesture == "rock":
        # Make the opposite gesture (paper)
        m_thumb.turn(100, 180)  # Adjust rotation angle as needed
        m_index.turn(100, 180)
        m_others.turn(100, 180)
    elif gesture == "paper":
        # Make the opposite gesture (scissors)
        m_thumb.turn(100, 90)   # Adjust rotation angle as needed
        m_index.turn(100, 90)
        m_others.turn(100, 90)
    elif gesture == "scissors":
        # Make the opposite gesture (rock)
        m_thumb.turn(100, 360)  # Adjust rotation angle as needed
        m_index.turn(100, 360)
        m_others.turn(100, 360)
    else:
        print("Unknown gesture")

# Example usage
gesture = "rock"  # Replace this with the gesture recognized by Mediapipe
send_command(gesture)

Integrate the Mediapipe hand gesture recognition code with the send_command function. This example assumes you have a function get_gesture() that returns the recognized gesture as a string ("rock", "paper", or "scissors").

import mediapipe as mp
import cv2

# Initialize Mediapipe Hand
mp_hands = mp.solutions.hands
hands = mp_hands.Hands()
mp_drawing = mp.solutions.drawing_utils

def get_gesture():
    cap = cv2.VideoCapture(0)

    while cap.isOpened():
        ret, frame = cap.read()
        if not ret:
            break
        
        # Convert the BGR image to RGB.
        image = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
        image.flags.writeable = False
        results = hands.process(image)
        image.flags.writeable = True
        image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)

        if results.multi_hand_landmarks:
            for hand_landmarks in results.multi_hand_landmarks:
                mp_drawing.draw_landmarks(image, hand_landmarks, mp_hands.HAND_CONNECTIONS)
                # Analyze hand_landmarks to determine the gesture
                gesture = "rock"  # Placeholder: replace with actual gesture detection logic
                
                return gesture
        
        cv2.imshow('Hand Tracking', image)
        if cv2.waitKey(5) & 0xFF == 27:
            break

    cap.release()
    return "none"

# Main loop
while True:
    gesture = get_gesture()
    if gesture != "none":
        send_command(gesture)

1. Run the Mediapipe gesture recognition: This will recognize the gesture and send the corresponding command to the NXT brick.
2. Observe the NXT's response: The NXT brick should make the opposite gesture using its motors.

Tell me if there's any other queries

[samannaz001]

Hi

Your project sounds really interesting! For receiving a string from PyCharm to NXT using Bricx Command Center, you'll need to use the Serial commands in NXC (Not eXactly C) to handle the communication over USB.

Here's a basic idea of what you could do:

Set up the serial connection in PyCharm:

Use the pyserial library to send data via USB.
Write a program in the Bricx Command Center to receive the data:

Use NXC's Serial commands to read the string and control the motors accordingly.
Here’s a simple example of what the NXC code might look like:

c
Copy code
task main() {
// Initialize the serial connection
SerialPortInit(0, BAUD_9600);

string receivedGesture;

while (true) {
    // Read the incoming data
    SerialPortReceiveString(0, receivedGesture, 10);

    // Check the received gesture and control the motors
    if (receivedGesture == "rock") {
        // Make the corresponding gesture
        // Control motors for rock gesture
    } else if (receivedGesture == "paper") {
        // Control motors for paper gesture
    } else if (receivedGesture == "scissors") {
        // Control motors for scissors gesture
    }
}

}
And in PyCharm, you might have something like this:

python
Copy code
import serial

Set up the serial connection

ser = serial.Serial('COM3', 9600) # Adjust COM port as needed

gesture = "rock" # Example gesture
ser.write(gesture.encode()) # Send the gesture
This is a simplified version, but it should give you a starting point. Make sure the COM port and baud rate match in both the PyCharm script and the NXT program.

[BlackWhiteWallpaper]

Thank you @samannaz001 I will try it tomorrow. I still have a question left: what do you mean by "python copy code and import serial"? How would my python code look like to send the string via usb?