Apple Harvesters Use Vision To “See” Fruit

Editor’s Note: This past October, Tony Koselka, Jillian Cannons, and Ryan Carlon of Vision Robotics Corporation (VRC) returned to Washington for further field trials of the Newton Scout, a vision-based system for automatic crop load assessment. The following report highlights their results.

Developed in part through the USDA-funded project Comprehensive Automation for Specialty Crops (CASC), the Newton Scout will revolutionize orchard operations by providing growers with a map that accurately maps fruit placement, fruit size, and number throughout the growing season. Fruit maps will be used to determine crop load, which in turn will give growers the information to develop harvest and marketing strategies. And this is just the beginning. Developing reliable vision systems that are robust enough for use in orchards opens up all kinds of possibilities in the area of pest and disease detection, plant health assessment, and the use of robotics at harvest.

The Newton Scout integrates stereo cameras with a custom image processing algorithm. The algorithms use classifiers to identify the apples, where the simplest classifier uses color and shape. For red apples, color is the primary classifier. For detecting green fruit, the Scout detects using statistical classifiers generated by boosting algorithms. The most challenging aspects of detecting fruit are occlusions, shadows, shape and color coincidences, and ambient lighting conditions such as bright sunlight. Once an apple is identified, the stereo cameras enable the system to determine its location and size.

The Vision Robotics field trial was conducted at Allan Brothers Othello Orchard in Washington. Prior to VRC’s arrival, Washington State University (WSU) Extension Educator Karen Lewis, and interns with the Washington Tree Fruit Research Commission, hand tagged, mapped, and sized fruit on several rows of fruit. This data allows for the groundtruthing or validation component of the test. Koselka took time out to speak about the development of a similar unit that counts oranges with the interns.

On one of the field trials for that unit, Koselka asked the grower to estimate his crop. The farm manager responded with 1,200 bushels. For this test, Vision Robotics scouted and counted 1,900 — a sizable difference and one that would have a large financial impact.

As Koselka stated, “not only will it [Scout] be able to accurately count a crop and locate each piece of fruit, it will ultimately feed into a robot harvester that will pick all the apples.” And if that was not enough, Koselka added, “Oh, and a pruner is not far off either!”

Even More Potential

The Newton Scout utility will be enhanced in the future with the addition of other possible technologies, including near infrared (NIR). As Newton evolves, it will have multiple uses throughout the fruit growing cycle — for example, assessment of sugar and acid content and internal damage. CASC is also making progress in other automation fronts that bridge with Vision Robotics’ work.

Carnegie Mellon University (CMU) researchers, led by Sanjiv Singh, are developing an Autonomous Prime Mover (APM) capable of towing the Newton Scout safely through entire orchard blocks. In the future, they will also automate an agricultural harvesting platform so that workers no longer need to climb on ladders to pick fruit. The APM’s multifunctionality, autonomous drive, and wireless networking could move it into the center of precision management of specialty crops.

The CASC team, led by CMU, includes engineers, scientists, Extension educators, growers, equipment manufacturers, and industry representatives. This plurality of expertise has led to synergistic endeavors that push the envelope in both the engineering and the science components of the project. For example, Ben Grocholsky at CMU is developing a geospatial system to map and locate each tree in an orchard; in cooperation with the plant science group, he was able to augment this information with local temperature and NDVI data that has the future possibility of being incorporated to detect plant stress and disease.

In another example of the synergy of the CASC team, during a field trial of an automated caliper measurement device, growers told the CASC team that a simpler tree counter would also be very useful. Singh’s group returned to CMU and produced a prototype tree counter ready to be tested in weeks; they will return to nurseries and test the counter this fall. The total time from conception to development will be less than four months, and if successful, this device could go into production within a year.

It will only be a matter of time before specialty crop growers also start using vehicles, sensors, devices, and software that automate farm operations, increase farm efficiency, and meet the ultimate goal of delivering to the consumer a consistently high quality affordable eating experience while returning money to the farming operation.

For more information on Comprehensive Automation for Specialty Crops, please visit www.cascrop.com.

Leave a Reply

Featured Stories

VegetablesClifton Seed Co. Presents Their Latest Vegetable Varieties
November 27, 2014
For more information, contact: Clifton Seed Co. P.O. Box 206 Faison, NC 28341 910-267-2690; fax: 910-267-2692 cliftonseed.com Read More
CitrusBe Willing To Take A ‘Wild Turkey’ Approach To Farm Fixes [Opinion]
November 27, 2014
Editor Frank Giles says Florida Grower will be stepping up to the bully pulpit in 2015 addressing topics that impact the industry the most. Read More
CitrusNew Partnership To Expand ‘Fresh From Florida’ Footprint In Taiwan
November 26, 2014
Latest deal puts exclamation point on successful Asia marketing campaign. Read More
Insect & Disease UpdateCalifornia OKs Asian Citrus Psyllid Insecticide
November 26, 2014
Agriphar Crop Solutions’ Micromite 80WGS is also effective on several other citrus pests. Read More
VegetablesAmerican Takii Presents Its Latest Vegetable Varieties
November 26, 2014
For more information, contact: American Takii 301 Natividad Rd. Salinas, CA 93906 831-443-4901; fax: 831-443-3976 takii.com Read More
FruitsWet December Predicted For Parched California
November 25, 2014
National Weather Service says above-normal precipitation expected; it may continue through the winter in central and southern parts of the Golden State. Read More
Apples & PearsEPA Approves Three New Moth Mating Disruption Products
November 25, 2014
Codling moth and oriental fruit moth are targeted by apple and pear growers. Read More
VegetablesSakata Hosts Annual Fall Field Trials
November 25, 2014
Growers and dealer partners gather to see new varieties at the company's South Florida field station. Read More
VegetablesStokes Seeds Presents Their Latest Vegetable Varieties
November 25, 2014
For more information, contact: Stokes Seeds P.O. Box 548 Buffalo, NY 14240-0548 800-263-7233; fax: 800-272-5560 stokesseeds.com Read More
VegetablesAbbott & Cobb Presents Their Latest Vegetable Varieties
November 25, 2014
For more information, contact: Abbott & Cobb P.O. Box 307 Feasterville, PA 19053 800-245-SEED; fax:215-245-9043 abbotcobb.com Read More

The Latest

VegetablesClifton Seed Co. Presents Their Latest Vegetable Variet…
November 27, 2014
For more information, contact: Clifton Seed Co. P.O. Box 206 Faison, NC 28341 910-267-2690; fax: 910-267-2692 cliftonseed.com Read More
CitrusBe Willing To Take A ‘Wild Turkey’ Approach…
November 27, 2014
Editor Frank Giles says Florida Grower will be stepping up to the bully pulpit in 2015 addressing topics that impact the industry the most. Read More
CitrusNew Partnership To Expand ‘Fresh From Florida’ Foot…
November 26, 2014
Latest deal puts exclamation point on successful Asia marketing campaign. Read More
Insect & Disease UpdateCalifornia OKs Asian Citrus Psyllid Insecticide
November 26, 2014
Agriphar Crop Solutions’ Micromite 80WGS is also effective on several other citrus pests. Read More
VegetablesAmerican Takii Presents Its Latest Vegetable Varieties
November 26, 2014
For more information, contact: American Takii 301 Natividad Rd. Salinas, CA 93906 831-443-4901; fax: 831-443-3976 takii.com Read More
FruitsWet December Predicted For Parched California
November 25, 2014
National Weather Service says above-normal precipitation expected; it may continue through the winter in central and southern parts of the Golden State. Read More
Apples & PearsEPA Approves Three New Moth Mating Disruption Products
November 25, 2014
Codling moth and oriental fruit moth are targeted by apple and pear growers. Read More
VegetablesSakata Hosts Annual Fall Field Trials
November 25, 2014
Growers and dealer partners gather to see new varieties at the company's South Florida field station. Read More
VegetablesStokes Seeds Presents Their Latest Vegetable Varieties
November 25, 2014
For more information, contact: Stokes Seeds P.O. Box 548 Buffalo, NY 14240-0548 800-263-7233; fax: 800-272-5560 stokesseeds.com Read More
VegetablesAbbott & Cobb Presents Their Latest Vegetable Varie…
November 25, 2014
For more information, contact: Abbott & Cobb P.O. Box 307 Feasterville, PA 19053 800-245-SEED; fax:215-245-9043 abbotcobb.com Read More
VegetablesVariety Trials 2014: Rupp Seeds
November 25, 2014
Check out new pumpkins weighing between 18 and 20 pounds as well as new powdery-mildew tolerant butternut squash hybrids. Read More
Food SafetyMultistate Salmonella Outbreak Linked To Bean Sprouts
November 25, 2014
FDA and the Centers for Disease Control and Prevention along with state and local officials continue to investigate the food-borne illness outbreak. Read More
Insect & Disease UpdateCitrus Survey: New Management Techniques Trending
November 25, 2014
Growers widen their tactical approach to survive HLB. Read More
FruitsFruitGrowerConnect Brings Buyers And Sellers Together […
November 25, 2014
Suppliers and growers got down to business in San Diego discussing current needs, solutions, and ways to increase efficiencies on the farm. Read More
Vegetables15 Must Have Carrot Varieties [Slideshow]
November 24, 2014
Browse the slideshow for information on 13 carrot varieties to add to your collection. Read More
CitrusReasons Aplenty Florida Growers Should Feel Good Going …
November 24, 2014
Thanks to recent industry achievements, the next 12-month period is shaping up to be a year to remember --- fondly. Read More
BerriesSuspected Borer Could Be Big Bother To Florida Blueberr…
November 24, 2014
Producers warned to be on the lookout as scientists probe samples. Read More
NutsWatch Your Almonds’ Salt Intake
November 24, 2014
Because of the current California drought, almond growers must pay extra attention to soil salinity. Read More