Apple Harvesters Use Vision To “See” Fruit

Editor’s Note: This past October, Tony Koselka, Jillian Cannons, and Ryan Carlon of Vision Robotics Corporation (VRC) returned to Washington for further field trials of the Newton Scout, a vision-based system for automatic crop load assessment. The following report highlights their results.

Developed in part through the USDA-funded project Comprehensive Automation for Specialty Crops (CASC), the Newton Scout will revolutionize orchard operations by providing growers with a map that accurately maps fruit placement, fruit size, and number throughout the growing season. Fruit maps will be used to determine crop load, which in turn will give growers the information to develop harvest and marketing strategies. And this is just the beginning. Developing reliable vision systems that are robust enough for use in orchards opens up all kinds of possibilities in the area of pest and disease detection, plant health assessment, and the use of robotics at harvest.

The Newton Scout integrates stereo cameras with a custom image processing algorithm. The algorithms use classifiers to identify the apples, where the simplest classifier uses color and shape. For red apples, color is the primary classifier. For detecting green fruit, the Scout detects using statistical classifiers generated by boosting algorithms. The most challenging aspects of detecting fruit are occlusions, shadows, shape and color coincidences, and ambient lighting conditions such as bright sunlight. Once an apple is identified, the stereo cameras enable the system to determine its location and size.

The Vision Robotics field trial was conducted at Allan Brothers Othello Orchard in Washington. Prior to VRC’s arrival, Washington State University (WSU) Extension Educator Karen Lewis, and interns with the Washington Tree Fruit Research Commission, hand tagged, mapped, and sized fruit on several rows of fruit. This data allows for the groundtruthing or validation component of the test. Koselka took time out to speak about the development of a similar unit that counts oranges with the interns.

On one of the field trials for that unit, Koselka asked the grower to estimate his crop. The farm manager responded with 1,200 bushels. For this test, Vision Robotics scouted and counted 1,900 — a sizable difference and one that would have a large financial impact.

As Koselka stated, “not only will it [Scout] be able to accurately count a crop and locate each piece of fruit, it will ultimately feed into a robot harvester that will pick all the apples.” And if that was not enough, Koselka added, “Oh, and a pruner is not far off either!”

Even More Potential

The Newton Scout utility will be enhanced in the future with the addition of other possible technologies, including near infrared (NIR). As Newton evolves, it will have multiple uses throughout the fruit growing cycle — for example, assessment of sugar and acid content and internal damage. CASC is also making progress in other automation fronts that bridge with Vision Robotics’ work.

Carnegie Mellon University (CMU) researchers, led by Sanjiv Singh, are developing an Autonomous Prime Mover (APM) capable of towing the Newton Scout safely through entire orchard blocks. In the future, they will also automate an agricultural harvesting platform so that workers no longer need to climb on ladders to pick fruit. The APM’s multifunctionality, autonomous drive, and wireless networking could move it into the center of precision management of specialty crops.

The CASC team, led by CMU, includes engineers, scientists, Extension educators, growers, equipment manufacturers, and industry representatives. This plurality of expertise has led to synergistic endeavors that push the envelope in both the engineering and the science components of the project. For example, Ben Grocholsky at CMU is developing a geospatial system to map and locate each tree in an orchard; in cooperation with the plant science group, he was able to augment this information with local temperature and NDVI data that has the future possibility of being incorporated to detect plant stress and disease.

In another example of the synergy of the CASC team, during a field trial of an automated caliper measurement device, growers told the CASC team that a simpler tree counter would also be very useful. Singh’s group returned to CMU and produced a prototype tree counter ready to be tested in weeks; they will return to nurseries and test the counter this fall. The total time from conception to development will be less than four months, and if successful, this device could go into production within a year.

It will only be a matter of time before specialty crop growers also start using vehicles, sensors, devices, and software that automate farm operations, increase farm efficiency, and meet the ultimate goal of delivering to the consumer a consistently high quality affordable eating experience while returning money to the farming operation.

For more information on Comprehensive Automation for Specialty Crops, please visit www.cascrop.com.

Leave a Reply

Featured Stories
Apple Grower of the YearTerhune Orchards Celebrates 40th Anniversary This Weekend
April 24, 2015
It’s a Mount-u-mental celebration for the 2005 Apple Grower Of The Year and family. Read More
OrganicWebcast: Weed Control Options For Organic Tomato Production
April 24, 2015
As part of a series of free educational webcasts on The Plant Management Network, a presentation on weed control considerations Read More
ProductionpH Reducer Certified For Use In Organic Production
April 24, 2015
Growth Products Ltd. announced that pH Reducer has been certified as a Registered Material for Use in Organic Agriculture by Read More
FruitsEast, Midwest Braces For Cold Snap
April 24, 2015
Growers keep an eye on the thermometer as temperatures dip into the mid to low 20s. Read More
anthracnose of tomato; diseases of tomato; crop protection
Disease ControlAnthracnose Can Cause Red Alert For Tomato Growers
April 24, 2015
Learn how to identify, the survival and spread, as well as management methods for this disease. Read More
UF Glow mandarin
Varieties & RootstocksFlorida Citrus Growers Vie For Chance To Plant Experimental Selections
April 23, 2015
Meetings set to support launch of rapid release program’s third suite of new varieties. Read More
Apples & PearsAPHIS Proposal To Allow Apples From China Becomes Rule
April 23, 2015
Growers in China expected to follow set standards of production to prevent possible importing of new pests. Read More
Havana, Cuba view
CitrusRenewed Relations With Cuba A Concern For Florida Growers [Opinion]
April 23, 2015
It might be said Sunshine State producers are worried if wider trade is opened as the potential for competing winter crops is real. Read More
7-6-27 mandarin hybrid
Varieties & Rootstocks4 Fresh Florida Citrus Selections To Watch For
April 22, 2015
Meet the newest group of UF/IFAS FAST TRACK release candidates. Read More
Tomato necrotic streak virus symptoms
TomatoesNew Virus Detected In South Florida Tomatoes
April 22, 2015
Infectious species said to be a distant relative of tobacco streak virus, which is the cause of bean red node. Read More
The Latest
Apple Grower of the YearTerhune Orchards Celebrates 40th Anniversary This Weeke…
April 24, 2015
It’s a Mount-u-mental celebration for the 2005 Apple Grower Of The Year and family. Read More
OrganicWebcast: Weed Control Options For Organic Tomato Produ…
April 24, 2015
As part of a series of free educational webcasts on The Plant Management Network, a presentation on weed control considerations Read More
ProductionpH Reducer Certified For Use In Organic Production
April 24, 2015
Growth Products Ltd. announced that pH Reducer has been certified as a Registered Material for Use in Organic Agriculture by Read More
FruitsEast, Midwest Braces For Cold Snap
April 24, 2015
Growers keep an eye on the thermometer as temperatures dip into the mid to low 20s. Read More
anthracnose of tomato; diseases of tomato; crop protection
Disease ControlAnthracnose Can Cause Red Alert For Tomato Growers
April 24, 2015
Learn how to identify, the survival and spread, as well as management methods for this disease. Read More
UF Glow mandarin
Varieties & RootstocksFlorida Citrus Growers Vie For Chance To Plant Experime…
April 23, 2015
Meetings set to support launch of rapid release program’s third suite of new varieties. Read More
Apples & PearsAPHIS Proposal To Allow Apples From China Becomes Rule
April 23, 2015
Growers in China expected to follow set standards of production to prevent possible importing of new pests. Read More
Havana, Cuba view
CitrusRenewed Relations With Cuba A Concern For Florida Growe…
April 23, 2015
It might be said Sunshine State producers are worried if wider trade is opened as the potential for competing winter crops is real. Read More
7-6-27 mandarin hybrid
Varieties & Rootstocks4 Fresh Florida Citrus Selections To Watch For
April 22, 2015
Meet the newest group of UF/IFAS FAST TRACK release candidates. Read More
Tomato necrotic streak virus symptoms
TomatoesNew Virus Detected In South Florida Tomatoes
April 22, 2015
Infectious species said to be a distant relative of tobacco streak virus, which is the cause of bean red node. Read More
Apples & PearsFruit Processor To Expand Operations In Virginia
April 22, 2015
Andros Foods North America to invest $73 million in area and hiring 160. Read More
FruitsOverturned Truck Dumps Millions Of Bees On Highway
April 22, 2015
The truck with 448 hives was headed to a farm in Washington. Read More
Crop ProtectionBrandt Adds Zonix Biofungicide To Portfolio
April 21, 2015
Brandt, a manufacturer of agricultural specialty products, has added Zonix biofungicide to its line of organic and sustainable products. Zonix, manufactured Read More
CitrusFarm Bureau Supports Bill To Halt Controversial WOTUS R…
April 21, 2015
 Legislation would force EPA and the U.S. Army of Engineers to withdraw Waters of the United States proposal. Read More
EquipmentResearchers Seeking Agricultural Applications For Unman…
April 21, 2015
From stress detection to yield estimation, technology offers promise in farming. Read More
CitrusRecord Number Of Organic Producers In U.S.
April 21, 2015
USDA’s Agricultural Marketing Services says domestic production has increased by 250% across all markets since 2002. Read More
FertilizerVerdesian Life Sciences Introduces New Phosphite-Free P…
April 21, 2015
Product designed as alternative for nut and stone fruit growers with concerns about MRLs. Read More
Disease ControlNew Fungicide For Wide Range Of Fruits, Vegetables In C…
April 21, 2015
FMC officially enters biological marketplace with first sales of Fracture fungicide. Read More