We had a blast at ICCV during our tutorial, as we open sourced and demoed Kinect Fusion (more on that soon), as well as had a lot of interesting discussions with our colleagues from all around the world. I’m curious to see how things evolve in the next 2 years and whether ICCV 2013 will be more 3D centric than 2011 was.

If you’re interested in joining our code sprints, we’re currently accepting applications for the PCL Trimble Code Sprint (TRCS).

It is our immense pleasure to announce the beginning of a new PCL Code Sprint sponsored by Toyota: PCL-TOCS!

PCL Code Sprints are intended to rapidly advance the capabilities of the Point Cloud Library in a certain area/subject by offering stipends to talented student developers and pairing them with knowledgeable mentors for several months of accelerated software development. These sprints have been inspired by the Google Summer Of Code (GSOC) initiative, and we will be following the same basic model. Projects will run for an initial period of 3 months with the same structure and performance evaluations as the GSOC program, and all of the code produced will be open source.

For this fall’s PCL-Toyota Code Sprint, we have identified the following important areas for further development in PCL, and we are therefore searching for outstanding candidates (and mentors) to work on the following projects:

1. Surface reconstruction and point cloud smoothing

   Accurate surface reconstruction and point cloud smoothing have been demonstrated in specific test environments with minimal clutter and when the object of interest fills most of the field of view. However, additional algorithms are needed when performing surface reconstruction and/or point cloud smoothing in a cluttered space. In this project, students will work to develop algorithms capable of smoothing noisy point clouds and reconstructing surfaces of objects so that they can be compared against a collection of known 3D objected models. The algorithms developed must be able to operate in a cluttered home or health-care environment, and they must be tolerant of noisy point cloud data (such as those acquired by sensors like the Microsoft Kinect). This project will also investigate the reliability and accuracy of different surface reconstruction techniques when data is acquired from a moving sensor.

2. CPU and GPU optimizations
   • CPU and GPU optimizations for real-time global point cloud localization
     In this project, students will work to develop CPU and GPU optimizations to enable real-time 3D place recognition. Existing algorithms are able to process a point cloud containing a partial view of an environment and subsequently localize it within a previously captured 3D map of the complete environment. However, this task is very computationally expensive, and many improvements are needed to make these approaches practical for real world uses. The goal of this project is to develop highly optimized implementations of a number of general purpose search and feature extraction algorithms that can be used in a variety of applications.
   • CPU and GPU optimizations for real-time 3D segmentation and boundary extraction
     With image segmentation, being able to perform multiple levels of segmentation on a single image is becoming more common to aid in recognition tasks. For instance, it is common to use coarse segmentation initially to identify regions of interest and then use a finer segmentation for boundary detection of individual objects in the scene. In this project, the student will develop real-time algorithms for segmenting 3D images, using CPU and GPU optimizations so that multiple levels of segmentation be computed quickly.

The above projects will run for a period of 3 months divided into two terms. Each student developer will receive a stipend of $5000 for their contributions, subject to satisfactory mid-term and final evaluations.

Potential candidates should submit the following information to pcl-tocs@pointclouds.org:

• a brief resume
• a list of existing PCL contributions (if any)
• a list of projects (emphasis on open source projects please) that they contributed to in the past

Advanced C++ programming skills are required for all projects. Please remember that this is a unique opportunity to work with some of the world’s best 3D perception researchers!

Interested mentors should send a brief e-mail to the address mentioned above pointing out their time commitments and expertise in the field.

/usr/bin/ld: cannot find -lgcc_s

again, do:

sudo ln -s /lib/libgcc_s.so.1 /lib/libgcc_s.so

One thing that sets us apart from other initiatives is our openness to change, and to the whole management process of the library in general. We’re really happy to be part of a very exciting community, which seems to be continuously growing by the minute. So, check PCL out, and please let us know what’s wrong with it, so we can fix it!

Part of a 1-week project, together with a few of my colleagues, we programmed the PR2 robot to:

• navigate and find a refrigerator
• open its door by locating the fridge handle
• position the robot’s manipulators so that the door doesn’t close while grasping things inside the fridge
• automatically identify the types of beer available in the fridge (we trained models on about 9 beers - training involves simply storing a picture of the beer bottle you want to identify in a directory)
• determine the types of beer the user selected from a web page, compute grasping points, move the arms and grasp them, and then move the beers onto a storing rack that we installed on the robot
• close the fridge door
• navigate the the delivery point selected by the user on the web page
• identify people faces and perform delivery/hand-off beer + bottle opener
• wait for bottle opener and resume!

Optionally, the robot can open the beer too!

The entire source code is available at:
https://code.ros.org/svn/wg-ros-pkg/stacks/pr2_drinks/trunk/ .

If you find it useful, PLEASE let us know! Obviously this app is “powered by PCL”

You can read our entire blog entry here: http://www.willowgarage.com/blog/2010/07/06/beer-me-robot. Cheers!

Edit: I just noticed that we got picked by Slashdot. Some of the comments posted are pretty funny.

There are two main differences between the usual traditional workshops and this one:

• in the next few days, we will try to make available training and test datasets for two specific problems that the robotics community is now facing: 1) rigid object recognition and 2) people detection. We encourage submissions that can show results against these datasets. Submissions presenting similar problems using different data sources are obviously welcomed as well.
• validation of results: the organizing committee is eager to accept submissions that can show results that are reproducible. To do that, for the experimental datasets presented above, we will release a test set a few hours before the workshop. Participants are encouraged to try their algorithms/models on the test set (after have trained on the training set provided above). We aim at awarding the participants who manage to show accuracy results > 95% on the test set. We also encourage submissions that are willing to share their results as open source to the robotics community, and are willing to do a live demo on site.

We hope to see you all in Taiwan! With all these exciting workshops, we’re sure IROS will be awesome!

Because not a lot of people know why we are doing this or how this started, here’s a few random bits on PCL…

• PCL is a complete open source software package built as part of ROS, which stands for Robot Operating System/Robot Open Source. If you don’t know what ROS is yet, don’t worry, you will soon. We’re growing bigger everyday, and adding more functionality that will make your robots go “wheeeee” sooner than you think. Check out a few of them today to see them moving around, picking up objects, or simply just acting cool. ROS is a community driven open source effort (supported by Willow Garage) and the best way to think about it is a massive collection of tightly integrated software packages that enable robotics research and development. There’s probably over 100 useful packages in there, including OpenCV and OpenRave so don’t just take my word for it, go check it out yourself!
• PCL was created as a successor to an earlier attempt of designing PointCloud data types, structures and algorithms that proved to be inefficient. Those days are gone now, and we’re proud to announce PointCloud…2. Yes, our new underlying data type that represents point cloud data is _really_ called PointCloud2. Call it an uninspired day to name things if you wish. PointCloud2 was designed with PCL in mind (or vice versa - same thing ;), and supports alignment (e.g. SSE), has a compact data size, and has a lower computational overhead in general (especially when used with nodelets - more on that later).
• Because we want PCL to become for 3D and point cloud data processing what OpenCV is for 2D image processing, we are developing a large number of algorithms for operations such as: filtering, downsampling, 3D feature estimation, registration, surface reconstruction, segmentation, and many more, into PCL. To make things flexible and efficient on the long run, we designed PCL as a fully templated C++ library, with an Eigen backend, and lots of Boost goodies. Yes, porting it to an Android (or whatever) might be a bit of a challenge, but we’ll worry about that later. Right now, we just want robots to perceive and understand their surrounding environment better. The “robot sharing a cluster of cell phones” app will come naturally once we manage to correctly identify all the cell phones in a building
• We try to use explicit parallelization where possible in PCL, via OpenMP and TBB. We try to avoid parallelizing everything implicitly, because saving CPU cycles on those extra cores that your computer might have is a good thing ®. Basically you want to know when something is going all “parallel” and eating up all your resources on a robot. We ran into decent issues in the past, and learned a valuable lesson. I think.
• Because we know that learning something new is hard, we’re trying to help you out. There’s documentation, API documentation, tutorials, videos, and more. We already have examples for you on how to do object recognition, and environment modeling, to name a few. And you need to understand that we just started. PCL is roughly 3-4 months old, give or take.

There’s lots more to be said, but for now, just go check it out! If you’re asking yourselves why bother, well… why not? It’s open source, it’s fast, and it’s easy to use (or so we hope). If you don’t like it, just e-mail us and we’ll fix it for you. Yup, just like that!

Here’s a teaser for one of the many things that can be done as of today with PCL. Though the video was generated before PCL was conceived, all the algorithms that are needed to obtain these results are present in PCL. In fact, we’ll create a tutorial soon to show you how to do it too in case you get stuck.

Finally, don’t forget to check out the slides from our class at Stanford for more information about PCL and point clouds. I’ve also uploaded my ICRA 2010 PCL talk/presentation slides here.