Apple Harvesters Use Vision To “See” Fruit

Editor’s Note: This past October, Tony Koselka, Jillian Cannons, and Ryan Carlon of Vision Robotics Corporation (VRC) returned to Washington for further field trials of the Newton Scout, a vision-based system for automatic crop load assessment. The following report highlights their results.

Developed in part through the USDA-funded project Comprehensive Automation for Specialty Crops (CASC), the Newton Scout will revolutionize orchard operations by providing growers with a map that accurately maps fruit placement, fruit size, and number throughout the growing season. Fruit maps will be used to determine crop load, which in turn will give growers the information to develop harvest and marketing strategies. And this is just the beginning. Developing reliable vision systems that are robust enough for use in orchards opens up all kinds of possibilities in the area of pest and disease detection, plant health assessment, and the use of robotics at harvest.

The Newton Scout integrates stereo cameras with a custom image processing algorithm. The algorithms use classifiers to identify the apples, where the simplest classifier uses color and shape. For red apples, color is the primary classifier. For detecting green fruit, the Scout detects using statistical classifiers generated by boosting algorithms. The most challenging aspects of detecting fruit are occlusions, shadows, shape and color coincidences, and ambient lighting conditions such as bright sunlight. Once an apple is identified, the stereo cameras enable the system to determine its location and size.

The Vision Robotics field trial was conducted at Allan Brothers Othello Orchard in Washington. Prior to VRC’s arrival, Washington State University (WSU) Extension Educator Karen Lewis, and interns with the Washington Tree Fruit Research Commission, hand tagged, mapped, and sized fruit on several rows of fruit. This data allows for the groundtruthing or validation component of the test. Koselka took time out to speak about the development of a similar unit that counts oranges with the interns.

On one of the field trials for that unit, Koselka asked the grower to estimate his crop. The farm manager responded with 1,200 bushels. For this test, Vision Robotics scouted and counted 1,900 — a sizable difference and one that would have a large financial impact.

As Koselka stated, “not only will it [Scout] be able to accurately count a crop and locate each piece of fruit, it will ultimately feed into a robot harvester that will pick all the apples.” And if that was not enough, Koselka added, “Oh, and a pruner is not far off either!”

Even More Potential

The Newton Scout utility will be enhanced in the future with the addition of other possible technologies, including near infrared (NIR). As Newton evolves, it will have multiple uses throughout the fruit growing cycle — for example, assessment of sugar and acid content and internal damage. CASC is also making progress in other automation fronts that bridge with Vision Robotics’ work.

Carnegie Mellon University (CMU) researchers, led by Sanjiv Singh, are developing an Autonomous Prime Mover (APM) capable of towing the Newton Scout safely through entire orchard blocks. In the future, they will also automate an agricultural harvesting platform so that workers no longer need to climb on ladders to pick fruit. The APM’s multifunctionality, autonomous drive, and wireless networking could move it into the center of precision management of specialty crops.

The CASC team, led by CMU, includes engineers, scientists, Extension educators, growers, equipment manufacturers, and industry representatives. This plurality of expertise has led to synergistic endeavors that push the envelope in both the engineering and the science components of the project. For example, Ben Grocholsky at CMU is developing a geospatial system to map and locate each tree in an orchard; in cooperation with the plant science group, he was able to augment this information with local temperature and NDVI data that has the future possibility of being incorporated to detect plant stress and disease.

In another example of the synergy of the CASC team, during a field trial of an automated caliper measurement device, growers told the CASC team that a simpler tree counter would also be very useful. Singh’s group returned to CMU and produced a prototype tree counter ready to be tested in weeks; they will return to nurseries and test the counter this fall. The total time from conception to development will be less than four months, and if successful, this device could go into production within a year.

It will only be a matter of time before specialty crop growers also start using vehicles, sensors, devices, and software that automate farm operations, increase farm efficiency, and meet the ultimate goal of delivering to the consumer a consistently high quality affordable eating experience while returning money to the farming operation.

For more information on Comprehensive Automation for Specialty Crops, please visit www.cascrop.com.

Leave a Reply

Featured Stories
Apples & Pears
March 27, 2017
It’s a Family Reunion at IFTA
Some of my fondest memories on the road are at IFTA conferences. Read More
Apples & Pears
March 27, 2017
Out of this World Expectations for ‘Cosmic Crisp’ Apple
With millions of trees going into PNW orchards, Washington State growers are betting big on ‘Cosmic Crisp. The shift in varieties planted and more took center stage when the International Fruit Tree’s Annual Conference met in Wenatchee, WA. Read More
Miss Florida Citrus and Imperial Polk County 2017 winners Paige Todd and Rachel Smith
Citrus
March 27, 2017
Say Hello to the New Miss Florida Citrus
Storied tradition continues with crowning of the industry’s newest advocate. Read More
Irrigation
March 26, 2017
Tips to Maximize Water Efficiency in Your Almond Grove
Editor’s note: Spencer Cooper joined the Almond Board of California last fall as Senior Manager, Irrigation and Water Efficiency. Western Read More
SWD On Raspberry
Berries
March 26, 2017
Outsmarting SWD with Social Media
Entomologist in Kentucky uses social media for pest updates, trap catches. Read More
honeybees
Citrus
March 25, 2017
New Videos Offer Tips on Planting Pollinator Habitats
A new video series on planting flowering habitats is designed to help growers support crop pollinators. Read More
Freeze protected blueberry bushes in Florida
Berries
March 25, 2017
Florida Blueberry Crop Escapes Slap from Late-Season Cold Snap
Freeze protection tactics prove fruitful as local producers look to gain from Georgia’s loss. Read More
Apples & Pears
March 25, 2017
Two MAIA Releases Available for Growers, Hobbyists
‘Crunch-A-Bunch’ is an early ‘GoldRush’ and ‘Bakers Delight’ is a secret ingredient for baking. Read More
Nuts
March 24, 2017
Foreboding Fungus Vexes Almond Growers
In parts of California, almond growers have recently become concerned about Ganoderma root and butt rot. To put this concern Read More
Citrus
March 24, 2017
Marrone Bio Introduces Biostimulant to Reduce Sun Stress
Haven Anti-transpirant helps to reduce transpiration of fruit, nut, and vegetable crops. Read More
Campus of Massey University in New Zealand
Citrus
March 23, 2017
Count Down the World’s Top 30 Agriculture Colleges
Familiar names and several new faces among annual ranking’s elite. Read More
Pest Control
March 22, 2017
Fear of GMOs is Impacting Quality Food Research [Opinion]
Perhaps the greatest threat to our industry is a public that values instinctual opinion over careful research and the findings of experts who have dedicated years to their field of study. Read More
Business Planning
March 22, 2017
How HortTech Is Solving Grower Challenges
Dozens of tech companies are looking for solutions to real-world problems vegetable growers face. We take a look at what it could really mean for you and your business. Read More
Production
March 22, 2017
Weighing the Good and Bad of Six Soilless Media Options
Trying to decide which growing media is the right one for your greenhouse vegetable production? Take a look at these six options. Read More
Citrus
March 21, 2017
Industry Groups, Ag Leaders React to Proposed USDA, EPA Budget Cuts
New plan would slash USDA spending by 21%. Read More
The Latest
Apples & Pears
March 27, 2017
It’s a Family Reunion at IFTA
Some of my fondest memories on the road are at IFTA conferences. Read More
Apples & Pears
March 27, 2017
Out of this World Expectations for ̵…
With millions of trees going into PNW orchards, Washington State growers are betting big on ‘Cosmic Crisp. The shift in varieties planted and more took center stage when the International Fruit Tree’s Annual Conference met in Wenatchee, WA. Read More
Citrus
March 27, 2017
Say Hello to the New Miss Florida Citrus
Storied tradition continues with crowning of the industry’s newest advocate. Read More
Irrigation
March 26, 2017
Tips to Maximize Water Efficiency in You…
Editor’s note: Spencer Cooper joined the Almond Board of California last fall as Senior Manager, Irrigation and Water Efficiency. Western Read More
Berries
March 26, 2017
Outsmarting SWD with Social Media
Entomologist in Kentucky uses social media for pest updates, trap catches. Read More
Citrus
March 25, 2017
New Videos Offer Tips on Planting Pollin…
A new video series on planting flowering habitats is designed to help growers support crop pollinators. Read More
Berries
March 25, 2017
Florida Blueberry Crop Escapes Slap from…
Freeze protection tactics prove fruitful as local producers look to gain from Georgia’s loss. Read More
Apples & Pears
March 25, 2017
Two MAIA Releases Available for Growers,…
‘Crunch-A-Bunch’ is an early ‘GoldRush’ and ‘Bakers Delight’ is a secret ingredient for baking. Read More
Nuts
March 24, 2017
Foreboding Fungus Vexes Almond Growers
In parts of California, almond growers have recently become concerned about Ganoderma root and butt rot. To put this concern Read More
Citrus
March 24, 2017
Marrone Bio Introduces Biostimulant to R…
Haven Anti-transpirant helps to reduce transpiration of fruit, nut, and vegetable crops. Read More
Citrus
March 23, 2017
Count Down the World’s Top 30 Agricultur…
Familiar names and several new faces among annual ranking’s elite. Read More
Pest Control
March 22, 2017
Fear of GMOs is Impacting Quality Food R…
Perhaps the greatest threat to our industry is a public that values instinctual opinion over careful research and the findings of experts who have dedicated years to their field of study. Read More
Business Planning
March 22, 2017
How HortTech Is Solving Grower Challenge…
Dozens of tech companies are looking for solutions to real-world problems vegetable growers face. We take a look at what it could really mean for you and your business. Read More
Production
March 22, 2017
Weighing the Good and Bad of Six Soilles…
Trying to decide which growing media is the right one for your greenhouse vegetable production? Take a look at these six options. Read More
Citrus
March 21, 2017
Industry Groups, Ag Leaders React to Pro…
New plan would slash USDA spending by 21%. Read More
Cold Protection
March 21, 2017
Freezing Temperatures Hit South Carolina…
Peaches, blueberries, and strawberries among crops damaged. Read More
Citrus
March 21, 2017
Since When Did Eating Get So Complicated…
The bottom line to all this is the way we eat is undergoing massive disruption. Read More
Apples & Pears
March 21, 2017
Two More Lost Apple Varieties Found In S…
Amateur apple detective strikes again with two more finds in abandoned orchards near state park. Read More