Six applications of vision and AI in warehousing

One thing that still stands in the way of fully automated warehousing is the ability of robots to pick products up. Vision technology that is combined with artificial intelligence (AI) to recognize products and control robots can play a crucial role in this. Dutch vision pioneer Fizyr, together with robot suppliers and other partners, recently showed what else this technology can be used for.

By Marcel te Lindert

1 Picking and packing

This is the area that robot suppliers have been working on for years: developing robots that can replace order pickers. Swedish company Cognibotics has developed a robot arm made not of steel but of carbon fibre. Because all the motors are housed in the base rather than in the arm itself, the 1.8-metre-long robot arm is lighter and faster than traditional ones. Cognibotics claims its robots have a reach of 10 square metres and can pick 2,000 products weighing up to 7.5 kilograms per hour.

The vision technology from Fizyr holds the key to the robot’s ability to pick up products from one bin and deposit them in another. This TU Delft spin-off not only provides technology for analysing and recognizing images, but also for translating them into instructions for the robot. In practice, this technology is often better – and in any case faster – than humans at distinguishing different products in a bin from each other and estimating how best to handle them. Even if those products are partially or completely transparent.

2 Loading and unloading containers

For years, the market has been looking for fully automated or semi-automated solutions for unloading containers. So far, this is often still manual work, meaning that people have to constantly bend or reach to put boxes of various shapes and sizes on a telescopic conveyor in poorly lit containers. Technica, a company with Lebanese roots, has developed a mobile robot that takes just half an hour to do what would normally take two people three hours.

The robot has a 1.2-metre-wide vacuum gripper including forks. As the gripper contains multiple pistons, the robot is able to pull multiple boxes onto the forks at the same time and place them on a discharge conveyor. According to Technica, 240 rows of boxes can be unloaded per hour. In an initial pilot, this resulted in a speed of 2,200 boxes per hour. Fizyr’s vision technology is used to scan the container contents, recognize the different box sizes and translate them into instructions for the robot. Fizyr also registers when the boxes or container walls are not completely straight, and instructs the robot accordingly in order to handle the boxes without product damage. The robot can also be used in reverse to load containers.

3 A different approach to unloading containers

Robot supplier Yaskawa has developed a different approach to unloading containers. This solution consists of not one, but two robot arms with vacuum grippers suspended from a gantry structure. This allows the robots to move past each other, taking turns to pick up a box and deposit it on the roller conveyor. Together, the robots are capable of unloading 1,000 boxes per hour. The fact that the robots are not on the floor has another practical reason besides flexibility: if a box falls, the operator can enter the container more easily to retrieve it.

What makes Yaskawa’s solution special is that both robot arms are able to work together with the help of Fizyr’s vision technology. For example, if a box is too heavy for one robot arm, the second jumps in so that the box can still be placed on the discharge conveyor. If the first box of a new row is slightly jammed, the second robot arm holds the boxes next to it in place. And if a box is too elongated to be picked up from the front only, one robot arm pulls the box forward a little so that the other robot arm can grab it from the side.

4 Sorting pallets

Everyone is familiar with the euro pallet and perhaps also the block pallet, but there are many more different types and sizes of pallets. Because they are hard to distinguish from each other with the naked eye, Ai4Pallets (opening photo) has developed a pallet sorting system using Fizyr’s solution. Each pallet is imaged from all sides by six 2D cameras and one 3D camera. The characteristics are then compared against the 150 types of industrial pallets in the database. The pallets are subsequently sorted by type for reuse.

If the system sees that a pallet is damaged, it goes to the repair department. If the system sees that a pallet is not yet in the database, its vision-captured characteristics are used to add a new pallet type to the database. Currently, Ai4Pallets is expanding the database to include another 200 types of US pallets. Eventually, the company expects to be able to recognize and sort more than a thousand types of pallets.

5 Placing on a sorter

A sorting system saves a lot of work, but has one drawback: parcels often have to be placed on the sorting system manually. AWL has therefore developed a robot called Rosi that takes care of this singulation task. According to AWL, Rosi can pick up 1,700 parcels per hour and place them with 99.9% accuracy. And not just boxes, but also bags and envelopes.

6 Stock counting

Vision technology does not necessarily have to be used in combination with robots, as illustrated by a visitor’s reaction. He works at one of the largest fulfilment centres in the Netherlands, which has a huge shuttle system with thousands of storage bins. One of the problems is that the contents of the stock bins regularly turn out to be incorrect, leading to disruptions at the order picking stations and delays in order processing. ‘It would save us a lot of time if we could use vision technology to check the contents of each stock bin,’ he says.