Apple Harvesters Use Vision To “See” Fruit

Editor’s Note: This past October, Tony Koselka, Jillian Cannons, and Ryan Carlon of Vision Robotics Corporation (VRC) returned to Washington for further field trials of the Newton Scout, a vision-based system for automatic crop load assessment. The following report highlights their results.

Developed in part through the USDA-funded project Comprehensive Automation for Specialty Crops (CASC), the Newton Scout will revolutionize orchard operations by providing growers with a map that accurately maps fruit placement, fruit size, and number throughout the growing season. Fruit maps will be used to determine crop load, which in turn will give growers the information to develop harvest and marketing strategies. And this is just the beginning. Developing reliable vision systems that are robust enough for use in orchards opens up all kinds of possibilities in the area of pest and disease detection, plant health assessment, and the use of robotics at harvest.

The Newton Scout integrates stereo cameras with a custom image processing algorithm. The algorithms use classifiers to identify the apples, where the simplest classifier uses color and shape. For red apples, color is the primary classifier. For detecting green fruit, the Scout detects using statistical classifiers generated by boosting algorithms. The most challenging aspects of detecting fruit are occlusions, shadows, shape and color coincidences, and ambient lighting conditions such as bright sunlight. Once an apple is identified, the stereo cameras enable the system to determine its location and size.

The Vision Robotics field trial was conducted at Allan Brothers Othello Orchard in Washington. Prior to VRC’s arrival, Washington State University (WSU) Extension Educator Karen Lewis, and interns with the Washington Tree Fruit Research Commission, hand tagged, mapped, and sized fruit on several rows of fruit. This data allows for the groundtruthing or validation component of the test. Koselka took time out to speak about the development of a similar unit that counts oranges with the interns.

On one of the field trials for that unit, Koselka asked the grower to estimate his crop. The farm manager responded with 1,200 bushels. For this test, Vision Robotics scouted and counted 1,900 — a sizable difference and one that would have a large financial impact.

As Koselka stated, “not only will it [Scout] be able to accurately count a crop and locate each piece of fruit, it will ultimately feed into a robot harvester that will pick all the apples.” And if that was not enough, Koselka added, “Oh, and a pruner is not far off either!”

Even More Potential

The Newton Scout utility will be enhanced in the future with the addition of other possible technologies, including near infrared (NIR). As Newton evolves, it will have multiple uses throughout the fruit growing cycle — for example, assessment of sugar and acid content and internal damage. CASC is also making progress in other automation fronts that bridge with Vision Robotics’ work.

Carnegie Mellon University (CMU) researchers, led by Sanjiv Singh, are developing an Autonomous Prime Mover (APM) capable of towing the Newton Scout safely through entire orchard blocks. In the future, they will also automate an agricultural harvesting platform so that workers no longer need to climb on ladders to pick fruit. The APM’s multifunctionality, autonomous drive, and wireless networking could move it into the center of precision management of specialty crops.

The CASC team, led by CMU, includes engineers, scientists, Extension educators, growers, equipment manufacturers, and industry representatives. This plurality of expertise has led to synergistic endeavors that push the envelope in both the engineering and the science components of the project. For example, Ben Grocholsky at CMU is developing a geospatial system to map and locate each tree in an orchard; in cooperation with the plant science group, he was able to augment this information with local temperature and NDVI data that has the future possibility of being incorporated to detect plant stress and disease.

In another example of the synergy of the CASC team, during a field trial of an automated caliper measurement device, growers told the CASC team that a simpler tree counter would also be very useful. Singh’s group returned to CMU and produced a prototype tree counter ready to be tested in weeks; they will return to nurseries and test the counter this fall. The total time from conception to development will be less than four months, and if successful, this device could go into production within a year.

It will only be a matter of time before specialty crop growers also start using vehicles, sensors, devices, and software that automate farm operations, increase farm efficiency, and meet the ultimate goal of delivering to the consumer a consistently high quality affordable eating experience while returning money to the farming operation.

For more information on Comprehensive Automation for Specialty Crops, please visit www.cascrop.com.

Leave a Reply

Featured Stories

CEU SeriesCEU Series: Pesticide Labels Are A Must-Read
November 1, 2014
Be cued into what's on the container as well as safety data sheets. Read More
CitrusForecast: South Florida Dry Season May See Fair Share Of Damp Days
October 31, 2014
Outlook calls for possible above-normal precipitation around region for fall and winter months. Read More
ProductionDrought Forcing California Growers To Rely On Groundwater
October 31, 2014
While the state issues curtailments to those with junior rights, growers are increasingly relying on groundwater to irrigate their crop. Read More
Crop ProtectionThe Unlikely Parallel Between Human And Plant Health
October 31, 2014
One of the notable national developments in the summer of 2014 was the unexpected occurrence of serious respiratory infections in Read More
VegetablesWhy You Should Be Trialing New Vegetable Varieties
October 31, 2014
Selecting high-performing selections is no easy task. Read More
FruitsCalifornia Growers Sue State Over Drought
October 30, 2014
They claim the State Water Resources Control Board illegally denied water deliveries. Read More
CucurbitsNew Watermelon Planting Program For The Southeast U.S.
October 30, 2014
Sakata seeking to make regional impact by providing in-demand material through several local producers. Read More
Leafy Vegetables3 Can’t-Miss Cabbage Varieties For Florida Growers
October 30, 2014
Reed's Seeds highlights a trio of high-performance selections for Sunshine State producers. Read More
Insect ControlTake Control Of Thrips In Your Greenhouse
October 30, 2014
Why such a foreboding title? Thrips, one of the tiniest pests of greenhouse crops, are the carrier of one of Read More
BerriesManage Black Root Rot Of Strawberry
October 30, 2014
Controls include preplant nematode analysis, three-to-five year crop rotations, and more. Read More

The Latest

CEU SeriesCEU Series: Pesticide Labels Are A Must-Read
November 1, 2014
Be cued into what's on the container as well as safety data sheets. Read More
CitrusForecast: South Florida Dry Season May See Fair Share O…
October 31, 2014
Outlook calls for possible above-normal precipitation around region for fall and winter months. Read More
ProductionDrought Forcing California Growers To Rely On Groundwat…
October 31, 2014
While the state issues curtailments to those with junior rights, growers are increasingly relying on groundwater to irrigate their crop. Read More
Crop ProtectionThe Unlikely Parallel Between Human And Plant Health
October 31, 2014
One of the notable national developments in the summer of 2014 was the unexpected occurrence of serious respiratory infections in Read More
VegetablesWhy You Should Be Trialing New Vegetable Varieties
October 31, 2014
Selecting high-performing selections is no easy task. Read More
FruitsCalifornia Growers Sue State Over Drought
October 30, 2014
They claim the State Water Resources Control Board illegally denied water deliveries. Read More
CucurbitsNew Watermelon Planting Program For The Southeast U.S.
October 30, 2014
Sakata seeking to make regional impact by providing in-demand material through several local producers. Read More
Leafy Vegetables3 Can’t-Miss Cabbage Varieties For Florida Grower…
October 30, 2014
Reed's Seeds highlights a trio of high-performance selections for Sunshine State producers. Read More
Insect ControlTake Control Of Thrips In Your Greenhouse
October 30, 2014
Why such a foreboding title? Thrips, one of the tiniest pests of greenhouse crops, are the carrier of one of Read More
BerriesManage Black Root Rot Of Strawberry
October 30, 2014
Controls include preplant nematode analysis, three-to-five year crop rotations, and more. Read More
FruitsSexual Evolution May Bear More Fruit In The Orchard
October 30, 2014
Scientists take a closer look at genes and find new plant breeding possibilities. Read More
Apples & PearsWashington Apples To Restart Shipments To China
October 29, 2014
Agreement reached between U.S. and Chinese governments to reinstate market access after two-year absence. Read More
EquipmentEnter To Win A Kubota At The Washington State Horticult…
October 29, 2014
Attendees have a chance to take home a new utility vehicle courtesy of Wilbur-Ellis and Western Fruit Grower. Read More
CitrusUSDA To Provide $4 Million For Honey Bee Habitat
October 29, 2014
The funds will go toward helping growers implement conservation practices to improve honey bee health. Read More
FruitsHuge California Drought Meeting Planned
October 29, 2014
State, federal officials to meet as forecasts for the coming year look bleak. Read More
PotatoesNew CEO At Black Gold Farms, A Top 100 Grower
October 28, 2014
Eric Halverson takes the reins at the potato operation, while his father Gregg will remain president of the farm’s board of directors. Read More
FruitsValent U.S.A., MGK Enter Into Marketing Agreement
October 28, 2014
Valent U.S.A. to manage the marketing and sales of MGK’s crop protection line of insect control products beginning in April 2015. Read More
VegetablesMultistate Land-Grant University Research Project Contr…
October 28, 2014
The project began in the 1970s as a Regional Initiative and has since expanded Internationally. Read More