Catkin, Workspaces, and Environment Variables

Both the devel directory and the install directory contain setup files and compiled targets. In fact, the generic ROS term is that they are both result spaces. So a natural question one might ask is "why do we need both a devel and an install directory"? The easy answer is that catkin allows you to specify different sets of rules to follow when installing targets. These special rules may optionally be used to for developer convenience (examples described below), but they are also an important part of the ROS build farm automatically building packages that can be installed via apt-get. Here are a few examples of how one could effectively use these result spaces:

1. Let's say we are actively developing and debugging complex C++ code that takes a long time to compile. We could configure catkin to build the code in the devel space with no compiler optimizations (to save compilation time), and to include debug flags in the compiled binary (to allow a debugger to be able to trace errors in the compiled binary back to particular lines in the source code). Then, we could configure the catkin_make install targets to add compiler optimizations (to compile more efficient binaries at the expense of greater compilation time), and to remove debug flags (to produce smaller binaries). With this setup, we could do all of our development and debugging in the devel space, and then when we want to actually use the code that we've compiled, we could switch over to the install space.
2. Let's say we have a ROS package that's primary purpose is not to provide ROS nodes, but rather to provide generically useful libraries that other ROS packages would be able to link against. Maybe the libraries are so useful that other non-ROS projects might also want to link against them. Well we could, in principle, configure our install targets to actually move these compiled libraries to somewhere that is natively used by non-ROS projects, e.g., /usr/local/lib/. This is not done regularly in practice, because uninstalling targets that are installed this way is can be difficult if you don't take precautions (this is one of the purposes of dpkg / apt-get).
3. Python packages are handled quite differently when using the devel vs. the install space. In the devel space, there are special "relaying" scripts (e.g. this one) that help Python automatically access the actual copies of the Python scripts, packages, and modules located in the src directory of a workspace. This means that we can edit any Python script in the src directory, and re-run whatever ROS thing we are doing, and we will automatically be using the updated version of the Python code. This is not true in the install space; in the install space, we'd have to re-run catkin_make install every time we edit any Python code. One common use case for the install directory is to build Python C/C++ extensions using a setup.py file.

4. As mentioned above, the install targets are used by the build farm when creating deb packages that can be installed via apt-get. Let's say that you have a package that you'd like to distribute that contained not only nodes, but also a launch file to use those nodes. Then you could create a special install target that caused the launch file to be "installed" when running catkin_make install. Effectively this would likely just copy the launch file to a location that ROS searches for launch files when "sourcing" the setup.bash file from the install space. Then when you submit your package to the build farm for auto-generation of the debian package, the build farm will also call your install target, which will result in your launch file being included in the deb package. For one example of this, check out the CMakeLists.txt file of the turtlebot_bringup package. Note they don't even compile any code, all they do is distribute launch files, configuration files, and images. For another example, check out the last few lines of the turtlesim package's CMakeLists.txt file. Here they ensure that all of the svg and png files that are included in the images directory are added to the package when installed. On my system, I have the turtlesim package installed from apt-get, and I can demonstrate the impact of this install target on the generated deb file with the following snippet:

   jarvis@test2018:~⟫ rospack find turtlesim
   /opt/ros/melodic/share/turtlesim
   jarvis@test2018:~⟫ ls /opt/ros/melodic/share/turtlesim/images/
   box-turtle.png   electric.png  groovy.png  hydro.svg   indigo.svg  kinetic.png  lunar.png  melodic.png  robot-turtle.png  turtle.png
   diamondback.png  fuerte.png    hydro.png   indigo.png  jade.png    kinetic.svg  lunar.svg  palette.png  sea-turtle.png
   

• These are simply variables that applications can use to control their configurations
• In Ubuntu, some are set to default "global" values when you login. There should be very little reason to ever change these.
• Bash is a "shell" for interacting with the operating system. When you are typing in a terminal, you are interacting with bash.
  • There are other common shells people use in Linux/Unix:
    • zsh
    • sh
    • csh and tcsh
• "source" command is used to execute a piece of bash code
  • It is just like "pasting" the contents of a file into the terminal
  • ROS variables are controlled by "sourcing" appropriate setup files
    • source /opt/ros/melodic/setup.bash
    • source ~/catkinws/devel/setup.bash
• User-specific variables are typically set in the ~/.bashrc file. This file contains lines of bash code that are executed every time a new terminal is opened.
  • This file can be re-ran by "sourcing" it i.e. source ~/.bashrc. This can be useful if you are trying to test edits to the file.
  • Every terminal has its own environment. Editing ~/.bashrc will not take effect in your current terminal unless you re-run the file using the above source command
  • the dot at the beginning of the filename denotes this file as a "hidden file"
  • ls won't show this file without an ls -a
  • in Nautilus (the built-in file browser) C-h will to toggle visibility of hidden files
• There are other relevant bash configuration files that will be run in various situations.
  • ~/.profile
  • ~/.inputrc
  • ~/.bash_profile
  • Read about these files by typing man bash and navigating to the INVOCATION section
  • The important point right now is that if you need to edit any environment variables (in Ubuntu at least), you should be editing the ~/.bashrc file
• Every terminal has a different set of environment variables
• "env" command prints all environment variables in current terminal
  • "Pipe" to grep or less to view more conveniently
  • env |grep ROS will show all environment variables with "ROS" in them.
    • Issues with environment variables and workspace setups is one of the most common issues that new ROS users encounter. Commit the above command to memory (or even alias it) and always check the output as one of your initial debugging attempts.
  • env |less will show all environment variables in the program less.
    • less is a "pager" that supports searching
    • press "h" to see all keyboard shorcuts
    • press "q" to quit
• export is used to set environment variables
  • Without export a bash variable is only visible to parent process
  • See this stack overflow post for more information
• unset can be used to remove variables

• See this page for the complete documentation on ROS environment variables
• We generally do not need to use "export" to set these manually as the workspace setup files manage most of these variables. The notable exceptions are ROS_IP/ROS_HOSTNAME and ROS_MASTER_URI
• The following is a list of a few of the most important ROS environment variables.
  • ROS_ROOT This sets where the ROS core packages are located. The "setup.bash" file included with a ROS destroy automatically sets this i.e. source /opt/ros/melodic/setup.bash takes care of this variable for you.
  • ROS_MASTER_URI This variable is required, and it tells nodes where to look for master. The default value is http://localhost:11311; this basically tells all nodes that the master should be running on the same computer. We modify this value for networking ROS computers together.
  • ROS_IP/ROS_HOSTNAME These variables are optional variables that facilitate networking multiple ROS computers. These variables are often misunderstood and used incorrectly. In short, both of these variables are used to tell master where any node needing to communicate with a node running on your machine can find your machine. You could think of these variables as an important part of the address for all nodes on your machine. You should only ever set either ROS_IP or ROS_HOSTNAME – you should never set both. If you are planning to use a name server to be able to refer to the computers on your ROS network using their hostnames, then you would set the ROS_HOSTNAME environment variable to be the hostname of your machine. Note that you often see .local appended to the hostname when the computers are on a local network. This is because adding a .local suffix for local machines is a common setup for many router's DNS services. The advantage of using hostnames is that you don't need to rely on static IP addresses. This is more flexible, and usually requires less up-front configuration. However, you are relying on name resolution to work properly; if that isn't working properly your ROS system won't work. The other option is to use IP addresses instead of hostnames, and in this case you would need to know your IP address and you would set the ROS_IP variable to be equal to your IP address. This is generally a more reliable but less flexible option. You don't require name resolution to work, but you do need to know the IP addresses of all of the machines, and if you want your setup to be persistently reliable, you would want to figure out how to ensure all machines had long-term static IPs.
  • PYTHONPATH This is a standard Python environment variable that tells Python where to search for modules when calling import statements. ROS requires this to be set as many core ROS tools require Python. The "setup.bash" files automatically modify this variable.
  • ROS_PACKAGE_PATH This variable isn't actually used by catkin, but it was used by rosbuild (catkin's predecessor); so it is kept around for legacy purposes. That being said, this is still a convenient variable to look at to verify that your environment is configured properly.
  • CMAKE_PREFIX_PATH This is a standard CMake variable that CMake uses when looking for libraries, executables, and objects that are required for building a CMake project. This is the most important variable for catkin, as it is what catkin uses when looking for ROS packages. It is automatically set by "setup.bash" files, and should never require manual editing.

• ROS allows users to combine workspaces in a very flexible way
• This concept is called "overlaying", and has a decent tutorial here
• Which workspaces are active, and which order they are traversed is controlled by environment variables
• The relevant environment variables for catkin and ROS in general are controlled by setup files
• Rebuilding a workspace with other workspaces active automatically regenerates the setup files for that workspace i.e. the next time you want to use that workspace structure, there is no need to separately "activate" each workspace setup file
  • The setup files override each other. there is never a need to source multiple setup files.
• If you are having issues with your environment setup files, or if your workspace isn't building correctly, it is often a good idea to remove the build/ and devel/ directory