RobotArm

1. Introduction

"NXT Robotic Arm" application is based on a Lego Mindstorms Robotic Arm controlled by an Android device. This robotic arm imitates the movement of your real arm using three different interfaces. It provides you button control (for fine adjustments) and, on the other hand,  sensor control (imitates the arm movements using the phone's gyroscope) and also an interface that allows you to write. We can control the movements on three axis (X,Y,Z). The writing tool can be used to draw numbers from 0 to 9 (letters will be drawn in our next version).

2. State of the Art

Searching the market we found a few applications that are similar to ours (but they control different types of Mindstorms robots): NXT Remote Control https://play.google.com/store/apps/details?id=org.jfedor.nxtremotecontrol&feature=search_result (for a vehicle based robot, controlling only the wheel motors), Stream-O-Bot https://play.google.com/store/apps/details?id=edu.snc.android.mindstorm&feature=search_result (still a vehicle based robot, but it allows you to put your phone on it, and stream video to anther phone,but it doesn't use the gyroscope and it will no longer be updated), NXT Simple Remote https://play.google.com/store/apps/details?id=NXT.Simple&feature=search_result#?t=W251bGwsMSwxLDEsIk5YVC5TaW1wbGUiXQ.. (this app uses only 2  motors at once and doesn't have a gyroscope controlling option). There are several applications on the market, but neither of them specifically built for controlling a robotic arm which follows the movement of your human arm and tries to write numbers.

3. Software Requirements

"NXT Robotic Arm" is an application designed to control a Lego Mindstorms Robotic Arm as it follows:
- it connects your phone through Bluetooth with the NXT Brick
- gives you direct control to all three motors, with the possibility to adjust the power (independent for each engine), using Button Mode
- it has a Sensor Mode in order to follow the movement of your arm, while keeping the phone in your hand
- it has a predefined Drawing Mode for all the digits (0 to 9)
- Voice Control for Drawing Mode

4. System Design

"NXT Robotic Arm" uses Bluetooth connection to communicate with Lego Mindstorms NXT which is a programmable robotics kit with: 3 servo motors, 4 sensors (ultrasonic, sound, touch, and light), 7 connection cables, a USB interface cable, and the NXT Intelligent Brick. The Intelligent Brick is the "brain" of a Mindstorms machine. It lets the robot autonomously perform different operations.

The application provides you three different interfaces :
       1.The button control interface has six buttons which control all three engines. The power of each engine can be changed using the three seek bars on the bottom of the screen. You have to be careful with the power (100% may be too much ).
       2.The second tab is the sensor interface which is based on the Phone's gyroscope and accelerometer. It is easy to use because you have some hints in the application. Basically it uses up, down, left or right movements. 
       3.Draw mode is the third tab. This part of the application allows you to select the number that you want to be drawn by the robotic arm. It also offers you voice control for selecting the digit that you want. 

If the Bluetooth connection is not active ( you will see a red message with the text :"Not connected ") you can try to reconnect the Android device to the NXT using the Connect option from the Setting Menu ( you can also search for devices using the Scan button). 

5. Implementation

We use Lego NXT 2.0 that provides three servomotors, four sensors (ultrasonic, sound, touch, and light), 7 connection cables, a USB interface cable, and the NXT Intelligent Brick. Our application uses Bluetooth connection to communicate with the Android device. 

For best user experience we advise you to build your personal Robotic Arm as follows, but the most important thing is to place the engines in the right way: one for horizontal rotation - base rotation - and the other two for vertical movements : 

Robot Robot  Robot Robot

 In terms of implementation we use Android SDK. This tool provides you the API libraries and developer tools necessary to build, test, and debug apps for Android. We recommend you to get the ADT Bundle if you want to start developing apps. It includes the essential Android SDK components and a version of the Eclipse IDE with built-in ADT. The minimum version required for our application to run is version 8, but we recommend you install the application on a device with Android 4.0 or higher. 

Basically the project has two activities : one for device detection  and another one for control. The first activity gives you a list with detected devices and the possibility to scan again for new devices . From this list you have to select NXT. 

DeviceChooser 

The second activity gives you three different types of control using a Tab Widget if the device is connected. 

The first Tab is the Button Mode tab. This interface lets you control the power of all three engines easily because it has six custom Buttons - Image Buttons - (two for each engine) and three Seek Bars for power control (also, one for each engine).  All those components use events behind the interface (for the six buttons we use onTouchListener event and for the three seek bars onProgressChangedListener event).  This has also three TextViews  which show the current seek bar percentage. 

Button Mode  

The second tab is Sensor Mode. Using a Dialog Fragment we are able to display a pop-up which shows the user how to control the robotic arm through the sensors. This fragment uses a check box which updates an instance of shared preferences that tells the application if it should automatically show the fragment if tab is open, or not (a "Don't show this again" check box). 

In order to control the arm we use input from the accelerometer and orientation sensor: when the tab is open there is an orientation dead zone set around the current orientation of the phone. Any orientation change of the phone to the left or right outside of the dead zone changes the arm's base to move left or right. The accelerometer is used to measure the gravitational force and through that to define active zones where the phone is tilted up or down (to move the middle arm up or down) or tiled left or right (to move the upper part of the arm up or down - left is for down and right is for up -). To provide the user with feedback when the phone is inside an active sensor zone we have implemented a fragmented circle which light up different parts depending  on which sensor zones are currently active and moving one of the robot's motors.

Sensor Mode  Second Tab  Second Tab  Second Tab  Second tab  Second tab  Second tab  Second tab

 

The third tab is Draw Mode. It is also using a Dialog Fragment to display a pop-up which tells the user how to use the drawing mode and it has the check box button too (with the "Don't show this again"). In this mode you can draw numbers from 0 to 9 by cycling through available numbers using the two buttons or using the Voice Control. 

Third tab  Third tab

 

6. Testing

In the first stage we tested the Bluetooth connection with one button to move one motor. After that we added buttons for all the three motors.
Next we added sensor mode, and we tested how the accelerometer and orientation sensor works, and we adjusted the active zone limits to not be overly sensitive but still only require small movements to enter them.
Most of the time we spent trying to draw the numbers because the arm is not so precise and it has only circular movement and we needed to draw straight lines.

Testing   Testing

If you want to see final results you have to get the application. 


7. Conclusion

 The "NXT Robotic ARM" is the application designed to control the Lego Mindstors Robotic Arm. Using Bluetooth connection, it gives you direct control to all three motors in two ways: button control and sensor control. 
It provides a tool for drawing digits (0 to 9) and has also a voice control for this tool. 

For the next version we will try to make the robotic arm to draw the alphabet.