Lego Mindstorms with Java

Lego Mindstorms NXT (2006, or successor EV3 2013) is a robotics [ros] package with a controller that can be flashed with a Java firmware called LeJOS, and quite comfortably called from the Eclipse IDE by USB cable or by Bluetooth.

We aim for Eclipse-based Java programming deployed over Bluetooth.

Linux Install

There is no apt package for LeJOS [docs]. The NXJ install package (<20MB) [dl] is a simple tar file:

cd /opt
tar -xvzpf leJOS_NXJ_0.9.1beta-3.tar.gz
ln -s leJOS_NXJ_0.9.1beta-3 lejos

# also add to your .bashrc
export NXJ_HOME=/opt/lejos
export LEJOS_NXT_JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
export JAVA_HOME=$LEJOS_NXT_JAVA_HOME

You might also download an OpenJDK 8 when the system-wide JDK is unsuitable for some reason. Then we build LeJOS using ant, here on Ubuntu 18.04 (LTS). Building with OpenJDK 8 succeeded but failed with OpenDK 11 ("cannot find jni.h"). Not setting the additional JAVA_HOME also lead to failure.

apt-get install ant libusb-dev libbluetooth-dev

cd $NXJ_HOME/build
ant
find .. -name 'lib*.so'

... and that should give us a new file /opt/lejos/lib/pc/native/linux/x86_64/libjlibnxt.so to work USB connections with, needed for flashing the firmware (no BT at this stage yet).

Firmware Flashing

First we add an access group lego and make udev recognize our NXT brick.

nano -w /etc/udev/rules.d/70-lego.rules
    # Lego NXT brick in normal mode
    SUBSYSTEM=="usb", DRIVER=="usb", ATTRS{idVendor}=="0694", ATTRS{idProduct}=="0002", GROUP="lego", MODE="0660"
    # Lego NXT brick in firmware update mode (Atmel SAM-BA mode)
    SUBSYSTEM=="usb", DRIVER=="usb", ATTRS{idVendor}=="03eb", ATTRS{idProduct}=="6124", GROUP="lego", MODE="0660"

groupadd lego
gpasswd -a myuser lego
udevadm control --reload-rules

Now connect the NXT brick by USB cable and tail -f /var/log/syslog where it should say something like Found NXT: NXT 001653146B3E or so. Next comes the scary part, the firmware flashing. To put the NXT brick into flash mode, press the hidden button near the USB port [blog] for 4 seconds until the brick starts ticking.

Then use the GUI flasher to get things going:

$NXJ_HOME/bin/nxjflashg

There is also a console version nxjflash without the g in case you dont like seeing Java GUIs. Once flashing is finished the NXT brick should reboot automatically and we are good to go.

Bluetooth Coupling

Using the buttons on the NXT brick, you should be able to get to the BT menu [docs]. Disconnect the USB cable before you start. In the BT menu, do Power on and Visibility on. Change pin to a 4-digit number you like.

On the Linux machine, go to your standard BT pairing dialog and make sure to choose Custom PIN before clicking on NXT to pair. So the NXT brick is discoverable and your Linux machine does not need to be.

Java per Command Line

Our first test of the BT connection is on the console:

nano -w HelloWorld.java
    import lejos.nxt.Button;
    public class HelloWorld {
      public static void main(String[] args) {
        System.out.println("Hello World");
        Button.waitForAnyPress();
      }
    }

$NXJ_HOME/bin/nxjc HelloWorld.java
$NXJ_HOME/bin/nxjlink -o HelloWorld.nxj HelloWorld
$NXJ_HOME/bin/nxjupload -r HelloWorld.nxj

... and after the BT upload taking a good 5 seconds, it should automatically start. You can always abort by pressing the orange button. And if you get stuck with autostart, hold the left gray arrow button before pushing the orange button [docs].

Eclipse

LeJOS supports Eclipse [docs] and Netbeans [docs]. When going with the former, open Help->Install New Software with the update site http://lejos.sourceforge.net/tools/eclipse/plugin/nxj/ to install the NXJ plugin, which probably requires an Eclipse reboot.

Next, go to Window->Preferences and leJOS NXJ to enter your path for NXJ_HOME and that we want bluetooth. I dont really know anymore whether the "named brick" thing was necessary.

And with that, create a File->New->LeJOS NXT Project, write a LegoTest.java file [docs], and deploy with Run->Run As->LeJOS NXT Program, which uses BT as specified in our preferences. It may also be necessary to right-click the project, open the Properties, and adjust the Java Compiler options.

Motors

Using the Motor class is the most fun [docs]. Ordinarily a motor is "regulated" meaning you just specify some degrees forward or backwards to move, and apart from some minor drift the motor will do so. There is also a non-braking mode but this is rather unpractical. Go to the main() method and here we go:

int headAngle = Motor.A.getPosition();
System.out.println("head=" + headAngle);
Button.RIGHT.waitForPress();

Current angle is important to keep track of where we are. The RIGHT button is the grey triangle to the right of the orange square ENTER; if we didnt wait for it, the printout on the NXT brick display would instantly vanish.

For the next upload, you can use Ctrl+F11 in Eclipse to run the same project upload again.

Motor.A.rotate(-15); // in degrees, not radians

That is more like it, isnt it? Practical for single limbs. But generally, we will have combined two motors to a tank configuration:

DifferentialPilot pilot = new DifferentialPilot(
    /*wheel diameter*/ 3.0, /*track width*/ 15.0,
    Motor.B, Motor.C, /*reverse*/ false);
pilot.rotate(/*deg*/ -15.0, /*return immediately*/ false);
pilot.travel(/*cm */ +10.0, /*return immediately*/ false);

So from the three motors A, B and C, we combined the latter two to a "pilot" with a certain wheel and track size, from which the DifferentialPilot class [docs] can determine what "15°" means in terms of motor rotation.

Note that we said "no" to immediate return, i.e. our calls are blocking. If we had said "yes", our program could to other stuff while the robot moves, but we would need an event loop where we check for pilot.isMoving() to see whether we can start the next move.

It can also make sense to check for pilot.isStalled() and do a pilot.stop() to prevent damage to the motors when grinding against an obstacle.

Sensors

Motors are fine for doing something, but we need sensors [docs] to have a basis for decisions. Accessing them depends on the port we have connected them to.

UltrasonicSensor ultra1 = new UltrasonicSensor(SensorPort.S2);
ColorSensor      color1 = new ColorSensor(SensorPort.S3);
TouchSensor      touch1 = new TouchSensor(SensorPort.S4);

The touch sensor is the easiest, with isPressed() the information that you would normally like to know. Ultrasonic [docs] has a distance version of that, with getDistance() returning 255 if there is too much noise for a reliable answer; there is also some object detection available [docs].

Color is a bit more complicated: getLightValue() and getColor() get you a reading but nothing to compare it against, so you should probably calibrate it with calibrateLow() and calibrateHigh(); I havent figured it out yet though.

Programming

Having an event loop is one easy way to go about programming the NXT brick, but apparently the LeJOS makers have also thought about it and created a "behavior API" [docs]. Could be fun looking at it!

Conclusion

Programming the NXT robot with Java is a lot more flexible and easier than having to resort to a visual language. And with the NXJ tutorial, there is lots more to discover. I hope you have fun with Lego Mindstorms, and have a good day!