Grasping the future with a robotic arm-hand combo

Reaching into your pocket to retrieve your phone seems like no big deal. For a robot, however, it is. Like many routine gestures that we take for granted, it’s actually very complex. Doing so requires large movements from the arm, followed by the hand’s finer motions to grasp and pull the phone out.  

Robot manipulation today generally focuses on motions either with a robot arm or a dexterous hand, but rarely both. Roboticists Daniel Rakita, who works frequently on robotic arms, and Aaron Dollar, a specialist in robotic grasping, recently teamed up for a project that combines their areas of expertise. The work will be presented at the 2023 International Conference on Robotics and Automation (ICRA). It takes place in London May 29 - June 2 and is one of the biggest robotics conferences in the world. Vatsal Patel, a Ph.D. candidate in Dollar’s lab, is the lead author. 

“Eventually, if you want to see robots in homes, around humans and doing complex tasks, we need robotic arms and hands to work together,” Patel said.

Combining the arm and hand movements has long been a priority for the robotics community, but it’s a goal that comes with some significant challenges. The parts of a robotic system that you can control are known as the degrees of freedom. The greater the degrees of freedom, the more you can do with a robot — however, it also increases the system’s complexity. It is, Rakita says, “kind of a trade-off.” 

For their robotic arm/hand, Rakita and Dollar developed algorithms that can grapple with such a high-dimensional system to get the optimal amount of freedom and control. One of the benefits of having extra degrees of freedom at the hand level as well as at the arm level is that the robot could have smoother motions and can better navigate within its environment.

“For instance, if you imagine a robot being able to reach into a drawer and you don't want its forearm resting on the drawer itself, maybe the robot could use its fingers to just reach for something in the drawer while avoiding collisions,” said Rakita, assistant professor of computer science. 

Intuitively, it makes sense that combining hands and arms allows robots to do more things. But because of the added complexity that comes with combining the two, Patel notes, robot arms and hands were always studied as separate sub-systems. 

“I think this was the first time that anyone has unified them in a single model and quantitatively characterized its benefits,” Patel said. 

The greater degrees of freedom add both manipulability, flexibility, and overall dexterity. It also means that the robotic system has more valid solutions for it to choose from to complete a task. This requires more sophisticated software to identify the best possible movements. That is, you need the right algorithms to pick those movements out of all the others — the “diamonds in the rough.” In the case of the researchers’ unified arm and hand system, as the degrees of freedom increased, so did the quality of the solutions.  

“This suggests that our algorithms are able to identify important criteria, and perhaps more fundamentally, complex arms and hands can indeed coordinate to achieve more effective movements together,” Rakita said. 

Rakita said the results of their work could be a springboard for thinking about what benefits could be gained by searching for other such algorithms. He added that the work has the potential for use in a multiple-robot system to do tasks in human-centered environments. 

“A lot of what motivates me in the robotics work that I do is home care and medical tasks,” he said. “I would love for this to be used in, say, a home care scenario where a robot could help out with chores or cooking tasks for people who want to age in place at home, but just need a little bit of help in the home.” 

Patel stressed that the project is just a starting point.

“We’re showing that unifying manipulation strategy works in this specific setup on a variety of arms and hands, but there’s still lots of potential here for exciting work to be done in this area,” he said.

Overall, Rakita said, the system shows what can be done through collaborations between the different robotics groups that frequently happen at Yale.

“We had this light bulb moment that happens when you put these ideas together, and we got this gestalt effect where the sum of the parts is greater than the whole,” he said. “Then the gears started going.”