Princeton researchers help a bot tidy up using large language model

The team takes AI personalization to a whole new level.
Ameya Paleja
The robot following rules generated by LLM to tidy up a room
The robot following rules generated by LLM to tidy up a room

Princeton University 

Researchers at the School of Engineering at Princeton University have successfully deployed a large language model (LLM) to help a robotic manipulator make sense of instructions to tidy up a room.

Robotic arms, or manipulators, are great at performing assigned tasks. In a factory setup, the manipulator can assemble machine parts, paint cars and even carve sculptures. However, get one at home, and the robot is clueless. It could turn the house upside down for a simple instruction such as "tidy up the room."

How to get a robot to tidy up

Researchers at Princeton University attribute this to an individual's personal preferences. When tidying up, the major challenge is determining the proper place to place an object. In an industrial setup, every item has a defined destination.

At home, however, the place where an item is kept is dependent on various factors ranging from personal taste to cultural backgrounds. For instance, some prefer to put their shirts in a drawer, while others would put them on shelves or hang them up.

While a robot deployed at home can be specifically programmed to carry out these tasks, with the advent of large language models, this step can be conveniently avoided. The researchers at Princeton attempted to train the robot on personal preferences using limited examples of prior interactions and were largely successful.

The team put user preferences through an LLM to generate placement rules for objects using the robot's three primitive functions of picking, placing, and tossing. Based on the rules, the robot was able to separate dark and light clothing into different receptacles and even sort tools, wooden blocks, and fruits into separate shelves.

In a messy room scenario, the robot was successful in sorting plastic utensils from paper bags and cans and even placing them in trashcans or recycling bins as desired. The robotic arm, which is capable of opening and closing drawers, was also successful in placing items inside them, whether they were round or rectangular. It was also able to differentiate storage boxes on the basis of colors and sort items in accordance with the instructions provided.

In real-world scenarios, the robot successfully placed objects with 85 percent accuracy.

The researchers have placed the code for LLM evaluation and real robot implementation on the software repository GitHub. A research paper detailing the work was also put on the pre-print server arxiv. The findings of the experiments have not been peer-reviewed yet.

Add Interesting Engineering to your Google News feed.
Add Interesting Engineering to your Google News feed.
message circleSHOW COMMENT (1)chevron
Job Board