Amazon Building Computer Vision Tech To Check Defects In Fruits, Vegetables

E-commerce giant Amazon is building a computer vision-based grading solution for food products such as onions and tomatoes. The machine learning-based approach analyses produce images to detect defects such as cuts, cracks and pressure damage. It can carry out millions of assessments per day at a cost that is far below compared to any other method, said a top Amazon scientist.

“Quality is one of the key drivers of fruit and vegetable purchasing decisions and a critical factor in achieving customer satisfaction,” said Rajeev Rastogi, vice president, machine learning, Amazon India at the company’s Smbhav event. “Having humans grade the quality of fruits and vegetables by manually examining each individual piece of produce like tomato or onion is not scalable to millions of quality assessments per day.”

Amazon plans to develop a conveyor belt based automatic grading and packing machine. It would leverage hardware and machine learning to pack produce into predetermined quality grades such as premium-grade A. The gradient pack machine will reduce grading cost by 78 per cent compared to manual grading.

Amazon also plans to use near-infrared sensors to detect attributes such as sweetness and ripeness. These cannot be detected in RGB (red, green, blue) images captured by traditional computer vision algorithms and require destructive methods such as eating the fruit.

“That can't obviously scale,” said Rastogi who began his career at Bell Labs. He also served as the vice president of Yahoo Labs, where his team developed data-extraction algorithms to pull structured information from billions of web pages, and then present them to users in easily digestible ways.

He joined Amazon in 2012. His first Amazon project involved the development of algorithms to classify products into Amazon’s large and complex taxonomical structure. For example, to classify a Samsonite luggage set in ‘carry-on luggage,’ ‘suitcases’ and ‘luggage sets.’ Since then, Rastogi has been involved in utilizing science to make an impact in a number of areas that have resulted in faster, more seamless and sustainable, shopping experiences.

As vice president of machine learning at Amazon India, Rastogi is now helping his team drive innovations that have a profound impact not only on shoppers in India but also on the company’s customers around the world.

For instance, the Amazon India team has also developed CRISP mobile app to tackle the spread of the Covid-19 pandemic and provide a safe work environment for our fulfilment centre employees. The CRISP app uses Bluetooth signals on mobile phones to track social contacts between Amazon associates. This social contact data is used to alert associates when they breach social distancing norms, for example, when they come too close to other associates. It is also used to identify users with a high risk of getting infected with Covid-19 since they have directly or indirectly come in contact with associates, who have tested positive for Covid-19.

“Now the associates with high infection risk scores can be prioritized for testing and quarantine actions,” said Rastogi. “We have launched CRISP across our last mile nodes to help associates maintain appropriate social distancing.”

Business Standard has always strived hard to provide up-to-date information and commentary on developments that are of interest to you and have wider political and economic implications for the country and the world. Your encouragement and constant feedback on how to improve our offering have only made our resolve and commitment to these ideals stronger. Even during these difficult times arising out of Covid-19, we continue to remain committed to keeping you informed and updated with credible news, authoritative views and incisive commentary on topical issues of relevance.

We, however, have a request.

As we battle the economic impact of the pandemic, we need your support even more, so that we can continue to offer you more quality content. Our subscription model has seen an encouraging response from many of you, who have subscribed to our online content. More subscription to our online content can only help us achieve the goals of offering you even better and more relevant content. We believe in free, fair and credible journalism. Your support through more subscriptions can help us practise the journalism to which we are committed.

Support quality journalism and subscribe to Business Standard.

Digital Editor