Packages and Workspaces

1. All ROS packages have a base directory containing a manifest file called package.xml.
   • This file is an XML document.
   • The full specification for package.xml is in ROS Rep 149.
   • The XML Schema for package.xml provides a machine-readable method for automatically validating the package.xml
2. An important element of package.xml is the <export><build_type>, which determines the type of package
   • ament python is used for pure python packages.
   • ament_cmake is used for C++ packages and packages that define custom Messages, Services, or Actions (i.e., Interfaces)
   • ament_cmake_python is for packages with mixed python/C++ code.

• Ament Python is the preferred package type for pure python ROS 2 packages
• Boilerplate for this package can be created using ros2 pkg create --build-type ament_python

• The directory structure is:

  pkg_name/
  ├── package.xml           # package manifest
  ├── pkg_name/             # python package
  │   ├── __init__.py       # Marks this directory as a python package
  │   └── module.py         # A python module
  ├── launch/               # Launchfiles go here
  │   ├── file.launch.xml   # An xml launch file
  │   ├── file.launch.py    # A python launch file
  │   └── file.launch.yaml  # A yaml launch file
  ├── config/               # Configuration directory
  │   ├── parameters.yaml   # File storing parameters for some nodes
  │   └── view.rviz         # Saved rviz configuration
  ├── resource/             # Used for registering packages in ament_index
  │   └── package_name      # Empty file used to register package with index
  ├── setup.cfg             # Sets up installation directories for ROS
  ├── setup.py              # Metadata, node entry , other files to install
  └── test/                 # Unit tests
      ├── test_copyright.py # ROS test to ensure proper copyright notice 
      ├── test_flake8.py    # Uses flake8 to enforce code style 
      └── test_pep257.py    # Ensures compliance with PEP 257
  

• Ament Cmake packages are primarily used for C++ ROS projects
• Boilerplate for this package can be created using ros2 pkg create --build-type ament_cmake
• Currently, custom interfaces (e.g., messages, services, or actions) need to be in their own Ament CMake package

• The directory structure is:

  pkg_name/
  ├── package.xml           # The manifest file
  ├── CMakeLists.txt        # Build instructions, uses ament_* functions
  ├── msg/                  # Directory for custom message types
  │   └── MessageType.msg   # A ROS IDL file defining a message
  ├── srv/                  # Directory for custom service types
  │   └── ServiceType.srv   # A ROS IDL file defining a service
  └── action                # Directory for custom action types
      └── ActionType.action # A ROS IDL file defining an action
  

1. Custom interfaces are created in ament_cmake type packages
2. To create a custom interface file, first write an interface file using the ROS IDL
   • Message files end with .msg and go in <package_name>/msg
   • Service files end with .srv and go in <package_name>/srv
   • Action files end with .action and go in <package_name>/action
   • See the Custom ROS Interface Tutorial

3. Edit CMakeLists.txt and add

   # Add in the CMake code that generates ROS interfaces
   find_package(rosidl_default_generators REQUIRED)
   
   # If any of your interfaces depend on another interface file
   # (For example assume you are using a geometry_msgs/msg/Twist)
   # Then you must first find the dependency with find_package (e.g.)
   find_package(geometry_msgs REQUIRED)
   
   # Tell Cmake what files to generate interface definitions for
   rosidl_generate_interfaces(${PROJECT_NAME}
        "msg/MessageType.msg" # Whatever message file you've defined
        "srv/ServiceType.srv" # Whatever service file you've defined
        ...    # and continue for all the interface files
        DEPENDENCIES geometry_msgs # in this example we depend on geometry_msgs
        )
   # Export a dependency on the ros_idl_runtime
   ament_export_dependencies(rosidl_default_runtime)
   

4. Edit package.xml
   • Add a <buildtool_depend> on rosidl_default_generators (contains tools needed to generate the interface code).
   • Add an <exec_depend> on rosidl_default_runtime to allow access to these types at runtime
   • Add the package to the rosidl_interface_packages group with <member_of_group> because this package generates interfaces.
   • You should also <depend> on any packages whose types were used within your custom types (in this example geometry_msgs)
5. All custom interface must start with a capital letter, and the names can contain only letters and numbers
6. More information at Implementing Custom Interfaces Tutorial
7. It is generally good ROS practice to make interfaces is separate packages so that they can be more easily used
   • It is also generally good ROS practice to use existing interfaces when possible, to make your code as compatible as possible with the rest of the ROS ecosystem.
8. An example of custom interfaces is available at https://github.com/m-elwin/me495_demo_interfaces

1. You must add an <exec_depend> on the package that defines the interface
2. In python use from <interface_package_name>.<msg|srv|action> import TypeName
   • <interface_package_name> is the name of the package that defines the interfaces
   • <msg|srv|action> is one of msg, srv, or action, as appropriate
   • TypeName is the name of the Message, Service, or Action Type

A ROS Workspace is a directory (called ws here) containing:

1. The source code to ROS packages: ws/src/<repo>
   • A package is any directory under ws/ containing a package.xml file
   • By convention, packages are placed under ws/src/<repo>, where <repo> is the name of the source-code repository[fn:Technically repositories can be directly under ws but that defies convention!]
   • Each repository may contain multiple packages
2. Log files generated from building the packages go in ws/log. The latest logs are in ws/log/latest
3. A directory where the packages are installed: ws/install
4. ROS workspaces enable you to build collections of related packages
5. Use a new workspace for each project, that way issues in packages you aren't currently developing won't effect you.
6. For a practical guide see the Creating a Workspace Tutorial.

• Workspaces must be built before they can be used.
• Run colcon build from the workspace directory (ws) to build all the packages in the ws/src directory
  • Build artifacts are stored in ws/build
  • The final results are installed to ws/install, which is where your code (even python) actually runs from
  • Thus, you always need to run colcon build at least once before using your workspace
• There are two important options to colcon build:
  • colcon build --symlink-install enables you to change python files without re-running colcon build.
    • This enables faster development but comes with some caveats: for example, changes to launchfiles will require a rebuild.
  • colcon build --merge-install by default each package is installed into its own sub-directory tree. This can make the path very long, so merge install keeps all package files of a given type (e.g., libraries and executables) in common directories.

• After building the ws/install directory (called the install space) is created
  • This contains your python files and other files (as long as you properly installed them in setup.py)
• After building you must source it to make the files in it available to ROS
• To source the workspace run source <path_to_install_space>/setup.bash
  • For example, if the current directory is the base of the workspace (e.g., ws/) run source install/setup.bash
  • Technically you should never run colcon build from a terminal window where you have sourced the associated workspace.
• Sourcing the workspace sets up the environment. See the Colcon Environment Documentation for more details.

• colcon_cd allows you to quickly switch between the workspace directory and that of a package
• Install with sudo apt install python3-colcon-cd
• Then, add source /usr/share/colcon_cd/function/colcon_cd.sh
• From your workspace directory run colcon_cd package to go to that package
• You can then run colcon_cd to return to the workspace directory

• colcon-clean allows you to easily remove the build results from a workspace
• Install with sudo apt install python3-colcon-clean
• colcon clean workspace will clean all generated files
• colcon clean packages allows you to select individual packages to clean

1. The colcon.pkg file, when placed at the base of the package directory enables a package to change colcon settings when it is being built
2. One use of this file is to install environment hooks, which are files that set environment variables when install/setup.bash is sourced
3. To set up an environment hook
   1. Create a dsv file in the env-hooks subdirectory of your package
      • For example: prepend-non-duplicate;VAR_NAME;VAR_VALUE will prepend VAR_VALUE to VAR_NAME, as it is not duplicated

   2. Add the hook in colcon.pkg:

      {
          "hooks": ["share/<PACKAGE_NAME>/env-hooks/<DSV_NAME>.dsv"]
      }
      

   3. The above code adds a hook located at the specified path
   4. In setup.py be sure to install the .dsv file to the path referenced in colcon.pkg

• Technically, you should not have the overlay sourced when using colcon build, which means you need a separate window for building and running ROS commands
  • Having the overlay sourced when building makes the packages built from the previous build available, which can sometimes mask or cause dependency problems
  • In practical day-to-day usage this issue rarely matters, but it is a good habit to be in or you may eventually forget and waste a bunch of time.
• Multiple ROS workspaces can be overlayed on top of each other allowing you to use packages from multiple workspaces or even override specific packages.
  • One common use case is if you are using external packages that are only available as source code (e.g., the interbotix packages).
  • Another use case is to allow you to develop a package that is installed on your system, without needing to uninstall the system version.
• The install/setup.bash file is created when colcon build is run and it captures the current ROS environment and adds your workspace to that.
  • This means that source install/setup.bash will setup your environment to be what it was at the time of build plus the current workspace.
  • install/local_setup.bash will overlay the current workspace onto the existing ROS environment, regardless of the environment at build time
• The ROS 2 environment that exists when colcon build is run for the first time is captured by the colcon build.
  • Thus source install/setup.bash will bring in all the workspaces that were on the path when colcon build ran the first time.
• Each environment also has local_setup.bash files that can be sourced. These bring in only that environment, but not the other environments that were present when colcon build first ran.

Here is my current suggested workaround (based in part off of this issue suggestion) to use pip packages with ROS 2 workspaces

1. Create a new workspace and create a dummy package in it:

   mkdir venv_ws
   cd venv_ws
   ros2 pkg create --build-type ament_python venv_pkg
   source ven_ws/install/setup.bash
   

2. Install the packages you want with pip into the install directory of this workspace by setting the PYTHONUSERBASE variable: PYTHONUSERBASE=$(ros pkg prefix venv_pkg) pip3 install --user --break-system-packages pip_package_to_install
   • By setting PYTHONUSERBASE you are telling pip that your system's root directory is actually the path to the venv_pkg
   • The --user flag tells pip to install to the "python user directory"
   • The --break-system-packages flag overrides pips admonition to only use it in virtual environments. However, because of PYTHONUSERBASE, the packages are only getting installed into the ROS workspace not the sysem, so this should be safe.
3. Now create a new workspace (proj_ws) for your project, and run colcon build
   • The proj_ws overlays the venv_ws and therefore will have venv_pkg on its python path
   • The pip installed packages will therefore be visible in the new workspace.
4. This method mixes existing system packages with new pip installed packages, when the ROS workspace is sourced
   • From a ROS point-of-view this situation is desirable because ROS packages are designed to work with system python packages so using as many of those as possible is beneficial
   • From the point-of-view of the pip package, some system packages may be incompatible: in that case you will need to install the correct version with pip
5. This method should work with -r requirements.txt as well as individual pip packages