Final Project (Fall 2016)

For the final project in this year's course, we are going to split up into 6 teams of 4 students, and have a Baxter part-manipulation competition. The guidelines for what you must accomplish are intentionally only loosely specified. Judges will be judging creativity, complexity, and reliability.

Below are some ideas that you could possibly incorporate into your projects to serve as inspiration.

GUI

It would be great to build a nice GUI to interact with your project. This GUI could allow changing of settings, control of the system state, or user interactivity. This could be done with something like rqt or interactive markers in rviz, or you could write a custom GUI using something like Tkinter or PyQt.

Web Interface

You could use something like Robot Web Tools and rosbridge_suite to build a web-based interface for communicating with your Baxter. You could even set this up to work over the internet (not just local network).

Simulation

You could base some of the technical content of the project largely around building a complex simulation environment in Gazebo or V-REP. You could even have the outputs of the simulation feeding into the decisions that the real Baxter makes.

IK

You could solve IK by implementing your own algorithms, you could even compare the IK solutions from pre-configured IK tools like IKFast, Baxter's IK service, trac_ik, and KDL.

Force/Torque Control/Sensing

You could do some very interesting things using Baxter's torque control modes, or you could use the estimate of end-effector force to try and control the arm to apply specific forces to an object or to estimate the inertia of an object. This could be used ensure the arm is compliant in one or more directions, for keeping pressure on some object constant, or for implementing advanced object placing strategies (e.g. when inserting a peg into a hole you could measure when forces applied to the peg).

Responsive Grasping

You could try and estimate whether grasps have been successful by utilizing forces measured by the grippers.

Joint Velocity Control

You could have the arm stabilize to target EE poses or trajectories by writing a joint velocity controller.

Visual Servoing

You could implement a controller that continuously tracks a target object and adjusts the control the estimate of the object's relative pose evolves. This is different from simply measuring where an object is, planning a path to get there, and then blindly executing that path.

Automatic Calibration

You could implement a routine that allows Baxter to automatically calibrate to his environment. For example, imagine that you have an external camera, and that you are relying on knowing the pose of the camera relative to a fixed frame on Baxter. Your calibration could involve taking pictures of a tag with the external camera and one of Baxter's cameras. Then, since both cameras have an estimate of the tag location in their respective camera frames, you could calibrate the pose from Baxter to the camera.

Face Communication

You could use Baxter's face display to animate/illustrate/indicate the how your demo is functioning. For example, you could display the output of motion planning schemes or image processing algorithms directly on the face, or you could use the display to print out the status/operation mode of your demo.

Baxter Button Interfaces

You could use Baxter's built-in arm buttons for implementing some sort interactivity.

Voice Command

Tools such as pocketsphinx or gspeech could be used to implement voice-based control (would likely want an external, directional microphone).

Face Recognition

You could recognize/track users with tools such as face_recognition, face_detector, or OpenCV's face recognition module.

Dynamic Reconfigure

You could use dynamic_reconfigure to create parameters that are easy to modify during a running demo.

For the final project, each student will receive four separate grades.

1. Each member of a team will receive a proposal grade. Theses points will be completion points. Details of the proposal will be in Canvas.
2. All students will receive a group grade. This grade will be assigned based on the effort put forth in the competition and final submission. All members receive the same grade.
3. Each student will receive a group assessment grade. You will be asked to assess your own group. This grade will depend on the quality of your assessment.
4. Each student will receive individual points for their effort on the final project. Ideally every student will receive full marks for this individual grade. However, if I hear reports of group members not fully contributing, they may not receive full credit.

Tags

There are a variety of tag tracking packages in ROS. These packages are usually designed to provide a pose estimate from a camera frame to the real-world location of some sort of visual tag (AR tags, QR codes, etc.). Some of the packages can also read metadata off of the tags. These packages may be a great way to calibrate your system, identify graspable objects, etc. They are also, generally, pretty easy to use. Typically all you need is a calibrated camera, some topic remapping, and a few configuration files.

• Mini Project: During this course in 2014, two students did a fantastic mini project where they provided tutorials on how to use several different tag tracking packages, and they compared and contrasted the pros and cons of each.
• Project from Last Year: This project last year used a couple of different tag tracking packages with Baxter. This may be a good resource to see how to use ar_track_alvar and visp_auto_tracker with Baxter.

Perception

For many of your demonstrations, you be interested in using some form of external world perception. Below are several obvious tools for perception.

• OpenCV: This is by far the most widely-used computer vision library. This package integrates easily with ROS via the cv_bridge package. I will provide one or two introductory lectures on this in the coming weeks.
• PCL: The Point Cloud Library "is a standalone, large scale, open project for 2D/3D image and point cloud processing." It is nicely integrated with ROS via the perception_pcl and the pcl_ros packages. If you are interested in using a 3D camera such as a Kinect, this is how to do it.
• Camera Calibration: The lenses on cameras tend to distort images. In order to accurately use a camera for perception, it is often desirable to use a calibration procedure to remove these distortions. Most of the tag tracking tools require calibrated cameras. Camera calibration is easily done in ROS using the camera_calibration package. The YAML files produced by this calibration are then fully compatible with the ROS image_pipeline.

Motion Planning

Motion planning for a robot arm in ROS is accomplished almost exclusively with MoveIt!. Many of your demonstrations can likely be accomplished with quite simple motion planning strategies. However, if your team chooses to leverage MoveIt!, you will certainly have significantly enhanced capabilites (at the expense of higher complexity). I will lecture on this in the coming weeks.

Examples

Here are a few good pick-and-place Baxter demos that you may find useful. These may be good starting places, but they are far from perfect. I strongly encourage you to use these for inspiration not solutions. If you use code from any examples online, you must cite your sources.

• Jon Rovira Pick-Place Demo: A few years ago, an undergraduate working with me put together a quick demo that allowed Baxter to find and grasp a red cube on a table (he was using a Rubik's cube, red-side-up). It worked reliably, but was sensitive to the height of the table.
• Visual Servoing Baxter Example: This is an example from Active Robots that is quite similar to what we are trying to do. Do not plan on copying this code and having it work.
• Baxter Pick and Learn: This example is from VUB/ULB universities in Belgium. "t uses a shape recognition algorithm to identify similar shapes and place them to a location previously demonstrated by the operator." I have not studied or used this example, but it could be helpful.
• 2014 Final Projects: There were two interesting final projects in 2014. These may give you some inspiration, but they are not necessarily the best code examples to follow :)
  • Baxter part sorting by mass: https://github.com/sherifm/baxter_sort
  • Baxter stocking stuffer: https://github.com/ChuChuIgbokwe/ME495-Final-Project-Baxter-Stocking-Stuffer
• 2015 Final Projects: Here are some of the final projects from the part sorting competition in 2015.
  • Group 1: GitHub repo
  • Group 2: Documentation, GitHub repo, video
  • Group 4: GitHub repo, video
  • Group 5: GitHub repo, video

Simulators

• Gazebo: Rethink has released a baxter_simulator package that could be very useful for developing much of your demonstrations. It is easily installed via apt-get. Check out the main documentation page for more.
• V-REP: If you'd like to investigate using V-REP instead-of or in-addition-to Gazebo, feel free to do so. Check out Jon Rovira's demo for getting started.

State Machines

Likely a good ROS-y way of implementing the behaviors that you want are through actions (you certainly don't need actions, they are somewhat complicated). ROS provides the SMACH package to quickly build a hierarchical state machine that is implemented via actions. Here is a Baxter Demo Manager state machine a student wrote a few years ago using SMACH.

Setup and Calibration

Likely a big concern you should have is how to quickly guarantee that the environment is configured properly for your code. You could certainly rely on manual calibrations (jigs, tape measures, etc.). Alternatively, you could write some calibrations routines that allow your system to adjust based on the current environment setup. Obviously the automatic solution is more advanced, but a manual procedure will likely work fine. Be sure that you know how to get calibrated quickly so that you don't waste your time during the competition.

Trajectory Action Server

The Baxter software provides a Joint Trajectory Action Server that allows Baxter to execute motions through a sequence of joint positions and possibly velocities/accelerations. You may have much better luck achieving high precision motions if you properly use this tool because Baxter can do a much better job of accounting for his own dynamics if the trajectory is known ahead of time.