tensorflow_ros_train

• 

Inspiration

After my first year of diving into autonomous navigation research, I've realized that this world can be pretty unfriendly to newcomers. The learning curve is steep. It's very easy to get bogged down in the details of complex ROS infrastructure and data synchronization and waste a lot of time just debugging data collection instead of actually analyzing and learning from the data. I hope to help change this.

What it does

tensorflow_ros_train is an unprecedented ROS package that accelerates model development and training using ROS data. The integration of Tensorflow and ROS infrastructure is made seamless through convenient classes and methods. After throroughly searching for a package to help accelerate my recent research project, I could find no existing ROS packages that offer integration of ROS and tensorflow beyond support for dependencies (allowing ROS and tensorflow code in the same file) and tutorial packages that integrate only a specific model into ROS.

With this package, a user can create any model that uses ROS data as inputs, collect data, and fit the model to the data-- all in only several lines of code.

The primary elements are:

ROSModel: A ROS wrapper class for tensorflow models that automates tailored data collection. A csv file is automatically generated based on specified features and targets.

ROSReading: A wrapper class for ROS data, similar to a ROS message type but specialized for being inputted to a model. The class provides built-in topic subscription and message data extraction. Features and targets are specified using ROSReadings.

ROSTask: A class describing a task for a simulated robot to perform. Built-in motion controllers are provided. Tasks can be queued and randomized.

Currently, the package offers the ability to run simulations in Turtlesim. A turtle is spawned in the map and given either a user-specified or randomly-generated goal location. It uses a PID controller to navigate to that goal. Data including turtlebot location and velocity is automatically recorded. A Tensorflow model to output velocity commands could quickly be designed and evaluated locally using this data. (I hoped to do this for fun but ran out of time.)

Check out the github repo README for a more thorough explanation of the package functionality and included example user code files!!

How I built it

This project was built with Tensorflow 2.2, ROS Kinetic, and Python 3.

Challenges I ran into

Tensorflow 2 uses Python 3. ROS Kinetic only officially supports Python 2. This caused a lot of issues and required workarounds. With the release of ROS Noetic (which supports Python 3) only 2 weeks away, I decided to persevere through incompatibility issues to demonstrate the functionality of the idea.

The most significant consequence of this was that I was unable to use ROS Gazebo simulation, my original plan. I instead used the 2D simulator Turtlesim. Also, I had to manually implement data synchronization because built-in functionality for this in ROS didn't work with python3.

I ended up not having enough time to implement some cool features like online training (simultaneous data collection and training). I plan to roll-out these improvements soon.

Accomplishments that I'm proud of

Successfully demonstrating the idea despite Python incompatibility issues

Making my first contribution to the open source world

Helping others dive into this exciting research area

What I learned

What a virtual environment actually is

How cool multithreading is

How cute turtles are

How to design an actual software package

Lots of debugging techniques

What's next for Tensorflow_ros_train

My plan is to optimize the package for ROS Noetic, which will solve all of the Python incompatibility issues and allow use of ROS Gazebo and other 3D simulations. I plan to release the package to the open source community for feedback and collaborate with others to develop more functionality (like various motion planners, environments, pre-trained models, and repositories of community-sourced data). I also hope to create a beginner tutorial to ROS and tensorflow that uses the package.

Built With

• python
• ros
• tensorflow
• turtlesim