Apple Harvesters Use Vision To “See” Fruit

Editor’s Note: This past October, Tony Koselka, Jillian Cannons, and Ryan Carlon of Vision Robotics Corporation (VRC) returned to Washington for further field trials of the Newton Scout, a vision-based system for automatic crop load assessment. The following report highlights their results.

Developed in part through the USDA-funded project Comprehensive Automation for Specialty Crops (CASC), the Newton Scout will revolutionize orchard operations by providing growers with a map that accurately maps fruit placement, fruit size, and number throughout the growing season. Fruit maps will be used to determine crop load, which in turn will give growers the information to develop harvest and marketing strategies. And this is just the beginning. Developing reliable vision systems that are robust enough for use in orchards opens up all kinds of possibilities in the area of pest and disease detection, plant health assessment, and the use of robotics at harvest.

The Newton Scout integrates stereo cameras with a custom image processing algorithm. The algorithms use classifiers to identify the apples, where the simplest classifier uses color and shape. For red apples, color is the primary classifier. For detecting green fruit, the Scout detects using statistical classifiers generated by boosting algorithms. The most challenging aspects of detecting fruit are occlusions, shadows, shape and color coincidences, and ambient lighting conditions such as bright sunlight. Once an apple is identified, the stereo cameras enable the system to determine its location and size.

The Vision Robotics field trial was conducted at Allan Brothers Othello Orchard in Washington. Prior to VRC’s arrival, Washington State University (WSU) Extension Educator Karen Lewis, and interns with the Washington Tree Fruit Research Commission, hand tagged, mapped, and sized fruit on several rows of fruit. This data allows for the groundtruthing or validation component of the test. Koselka took time out to speak about the development of a similar unit that counts oranges with the interns.

On one of the field trials for that unit, Koselka asked the grower to estimate his crop. The farm manager responded with 1,200 bushels. For this test, Vision Robotics scouted and counted 1,900 — a sizable difference and one that would have a large financial impact.

As Koselka stated, “not only will it [Scout] be able to accurately count a crop and locate each piece of fruit, it will ultimately feed into a robot harvester that will pick all the apples.” And if that was not enough, Koselka added, “Oh, and a pruner is not far off either!”

Even More Potential

The Newton Scout utility will be enhanced in the future with the addition of other possible technologies, including near infrared (NIR). As Newton evolves, it will have multiple uses throughout the fruit growing cycle — for example, assessment of sugar and acid content and internal damage. CASC is also making progress in other automation fronts that bridge with Vision Robotics’ work.

Carnegie Mellon University (CMU) researchers, led by Sanjiv Singh, are developing an Autonomous Prime Mover (APM) capable of towing the Newton Scout safely through entire orchard blocks. In the future, they will also automate an agricultural harvesting platform so that workers no longer need to climb on ladders to pick fruit. The APM’s multifunctionality, autonomous drive, and wireless networking could move it into the center of precision management of specialty crops.

The CASC team, led by CMU, includes engineers, scientists, Extension educators, growers, equipment manufacturers, and industry representatives. This plurality of expertise has led to synergistic endeavors that push the envelope in both the engineering and the science components of the project. For example, Ben Grocholsky at CMU is developing a geospatial system to map and locate each tree in an orchard; in cooperation with the plant science group, he was able to augment this information with local temperature and NDVI data that has the future possibility of being incorporated to detect plant stress and disease.

In another example of the synergy of the CASC team, during a field trial of an automated caliper measurement device, growers told the CASC team that a simpler tree counter would also be very useful. Singh’s group returned to CMU and produced a prototype tree counter ready to be tested in weeks; they will return to nurseries and test the counter this fall. The total time from conception to development will be less than four months, and if successful, this device could go into production within a year.

It will only be a matter of time before specialty crop growers also start using vehicles, sensors, devices, and software that automate farm operations, increase farm efficiency, and meet the ultimate goal of delivering to the consumer a consistently high quality affordable eating experience while returning money to the farming operation.

For more information on Comprehensive Automation for Specialty Crops, please visit www.cascrop.com.

Leave a Reply

Featured Stories
Citrus
July 26, 2017
Despite Controversy, Most Major Ag Groups Support Sam Clovis’ USDA Nomination
Most of the major agricultural associations are coming out in support of President Donald Trump’s nomination of Sam Clovis to Read More
Citrus
July 26, 2017
If We Want American Farm Workers, We Have to Offer Even More [Opinion]
Now is the time to lobby for ways to attract American labor. One idea? Offer college tuition and student loan forgiveness in exchange for farm work. Read More
Apples & Pears
July 25, 2017
Tips to Transition to an Apple Fruiting Wall System
Follow these suggestions to make the move from a 3-D spindle-shaped canopy to a 2-D system using mechanical pruning. Read More
Diamondback-Moth-adult-Stormy-Sparks
Crop Protection
July 25, 2017
Vegetable Field Scouting Guide: Diamondback Moth
Due diligence is needed to help take down this pest of biblical proportions. Read More
Disease Control
July 25, 2017
Brown Rot Sinks its Teeth into Michigan Cherries
Unseasonably wet weather causes outbreak, and growers are warned it can spread to peaches. Read More
Fruits
July 25, 2017
Michigan’s a Hit with Tree Fruit Growers
International Fruit Tree Association’s annual summer tour draws nearly 250 attendees. Read More
Farm Management
July 25, 2017
Grants Allocated to Tackle Changing Climate in Fruit, Vegetable Production
Projects designed to develop new approaches to mitigate effects of changing growing environment. Read More
Farm Management
July 24, 2017
American Farm Bureau Weighs in for Grower in Clean Water Act Suit
John Duarte, the owner of Duarte Nursery, one of the largest grapevine and nut tree suppliers on the West Coast, recently gained Read More
Disease Control
July 24, 2017
Researchers Find Detection Method for Crown Gall Disease
Oregon State University researchers developed molecular tools to work with commercially available kits that allow the user to quickly and effectively test plants for the disease, using a dipstick that reveals the presence of the pathogen within minutes. Read More
Vegetables
July 24, 2017
23 Of The Latest Pumpkin Varieties
Browse this slideshow for more details on 23 pumpkin varieties you need to know about from the nation’s leading seed breeders and distributors. Read More
Citrus
July 23, 2017
USDA Invests $7.6 Million toward Beneficial Insect Research
Projects to promote beneficial organisms as part of a pest control strategy. Read More
Citrus
July 22, 2017
Representative from Washington Proposes Amendment to H-2A Program
Move broadens use of H-2A to all of agriculture to include those with multiple crops and harvests. Read More
Insect & Disease Update
July 21, 2017
University of California Launches Website to Update Growers on Citrus Research
Easy-to-read format designed to give growers up-to-date information on huanglongbing and Asian citrus psyllid research. Read More
Hurricane Matthew satellite image as it brushed past Florida
Citrus
July 20, 2017
Atlantic Hurricane Forecast Taken Up a Notch
Current conditions in the tropics warrant marked revision in potential storm season scenarios. Read More
Sunset on Florida potato field day
Citrus
July 19, 2017
Researchers On a Mission to Find More Places for Growing Produce
Federal grant to aid exploration of food security solutions for the future. Read More
The Latest
Citrus
July 26, 2017
Despite Controversy, Most Major Ag Group…
Most of the major agricultural associations are coming out in support of President Donald Trump’s nomination of Sam Clovis to Read More
Citrus
July 26, 2017
If We Want American Farm Workers, We Hav…
Now is the time to lobby for ways to attract American labor. One idea? Offer college tuition and student loan forgiveness in exchange for farm work. Read More
Apples & Pears
July 25, 2017
Tips to Transition to an Apple Fruiting …
Follow these suggestions to make the move from a 3-D spindle-shaped canopy to a 2-D system using mechanical pruning. Read More
Crop Protection
July 25, 2017
Vegetable Field Scouting Guide: Diamondb…
Due diligence is needed to help take down this pest of biblical proportions. Read More
Disease Control
July 25, 2017
Brown Rot Sinks its Teeth into Michigan …
Unseasonably wet weather causes outbreak, and growers are warned it can spread to peaches. Read More
Fruits
July 25, 2017
Michigan’s a Hit with Tree Fruit Growers
International Fruit Tree Association’s annual summer tour draws nearly 250 attendees. Read More
Farm Management
July 25, 2017
Grants Allocated to Tackle Changing Clim…
Projects designed to develop new approaches to mitigate effects of changing growing environment. Read More
Farm Management
July 24, 2017
American Farm Bureau Weighs in for Growe…
John Duarte, the owner of Duarte Nursery, one of the largest grapevine and nut tree suppliers on the West Coast, recently gained Read More
Disease Control
July 24, 2017
Researchers Find Detection Method for Cr…
Oregon State University researchers developed molecular tools to work with commercially available kits that allow the user to quickly and effectively test plants for the disease, using a dipstick that reveals the presence of the pathogen within minutes. Read More
Vegetables
July 24, 2017
23 Of The Latest Pumpkin Varieties
Browse this slideshow for more details on 23 pumpkin varieties you need to know about from the nation’s leading seed breeders and distributors. Read More
Citrus
July 23, 2017
USDA Invests $7.6 Million toward Benefic…
Projects to promote beneficial organisms as part of a pest control strategy. Read More
Citrus
July 22, 2017
Representative from Washington Proposes …
Move broadens use of H-2A to all of agriculture to include those with multiple crops and harvests. Read More
Insect & Disease Update
July 21, 2017
University of California Launches Websit…
Easy-to-read format designed to give growers up-to-date information on huanglongbing and Asian citrus psyllid research. Read More
Citrus
July 20, 2017
Atlantic Hurricane Forecast Taken Up a N…
Current conditions in the tropics warrant marked revision in potential storm season scenarios. Read More
Citrus
July 19, 2017
Researchers On a Mission to Find More Pl…
Federal grant to aid exploration of food security solutions for the future. Read More
Citrus
July 19, 2017
Farm Labor Stories Making the News This …
The agricultural labor shortage is strong enough that the consumer press is beginning to report on it regularly. Here are the stories making headlines this month. Read More
Sweet Corn
July 19, 2017
Variety Specs | Production Tips: Corn &#…
Each month, American Vegetable Grower® will offer growing tips on specific varieties, supplied by the breeder or distributor. This month, we’re offering Read More
Citrus
July 19, 2017
Florida Grower Magazine is Seeking Your …
Life hacks are common in social media threads these days. They are those clever ideas or tricks aimed at making Read More