Apple Harvesters Use Vision To “See” Fruit

Editor’s Note: This past October, Tony Koselka, Jillian Cannons, and Ryan Carlon of Vision Robotics Corporation (VRC) returned to Washington for further field trials of the Newton Scout, a vision-based system for automatic crop load assessment. The following report highlights their results.

Developed in part through the USDA-funded project Comprehensive Automation for Specialty Crops (CASC), the Newton Scout will revolutionize orchard operations by providing growers with a map that accurately maps fruit placement, fruit size, and number throughout the growing season. Fruit maps will be used to determine crop load, which in turn will give growers the information to develop harvest and marketing strategies. And this is just the beginning. Developing reliable vision systems that are robust enough for use in orchards opens up all kinds of possibilities in the area of pest and disease detection, plant health assessment, and the use of robotics at harvest.

The Newton Scout integrates stereo cameras with a custom image processing algorithm. The algorithms use classifiers to identify the apples, where the simplest classifier uses color and shape. For red apples, color is the primary classifier. For detecting green fruit, the Scout detects using statistical classifiers generated by boosting algorithms. The most challenging aspects of detecting fruit are occlusions, shadows, shape and color coincidences, and ambient lighting conditions such as bright sunlight. Once an apple is identified, the stereo cameras enable the system to determine its location and size.

The Vision Robotics field trial was conducted at Allan Brothers Othello Orchard in Washington. Prior to VRC’s arrival, Washington State University (WSU) Extension Educator Karen Lewis, and interns with the Washington Tree Fruit Research Commission, hand tagged, mapped, and sized fruit on several rows of fruit. This data allows for the groundtruthing or validation component of the test. Koselka took time out to speak about the development of a similar unit that counts oranges with the interns.

On one of the field trials for that unit, Koselka asked the grower to estimate his crop. The farm manager responded with 1,200 bushels. For this test, Vision Robotics scouted and counted 1,900 — a sizable difference and one that would have a large financial impact.

As Koselka stated, “not only will it [Scout] be able to accurately count a crop and locate each piece of fruit, it will ultimately feed into a robot harvester that will pick all the apples.” And if that was not enough, Koselka added, “Oh, and a pruner is not far off either!”

Even More Potential

The Newton Scout utility will be enhanced in the future with the addition of other possible technologies, including near infrared (NIR). As Newton evolves, it will have multiple uses throughout the fruit growing cycle — for example, assessment of sugar and acid content and internal damage. CASC is also making progress in other automation fronts that bridge with Vision Robotics’ work.

Carnegie Mellon University (CMU) researchers, led by Sanjiv Singh, are developing an Autonomous Prime Mover (APM) capable of towing the Newton Scout safely through entire orchard blocks. In the future, they will also automate an agricultural harvesting platform so that workers no longer need to climb on ladders to pick fruit. The APM’s multifunctionality, autonomous drive, and wireless networking could move it into the center of precision management of specialty crops.

The CASC team, led by CMU, includes engineers, scientists, Extension educators, growers, equipment manufacturers, and industry representatives. This plurality of expertise has led to synergistic endeavors that push the envelope in both the engineering and the science components of the project. For example, Ben Grocholsky at CMU is developing a geospatial system to map and locate each tree in an orchard; in cooperation with the plant science group, he was able to augment this information with local temperature and NDVI data that has the future possibility of being incorporated to detect plant stress and disease.

In another example of the synergy of the CASC team, during a field trial of an automated caliper measurement device, growers told the CASC team that a simpler tree counter would also be very useful. Singh’s group returned to CMU and produced a prototype tree counter ready to be tested in weeks; they will return to nurseries and test the counter this fall. The total time from conception to development will be less than four months, and if successful, this device could go into production within a year.

It will only be a matter of time before specialty crop growers also start using vehicles, sensors, devices, and software that automate farm operations, increase farm efficiency, and meet the ultimate goal of delivering to the consumer a consistently high quality affordable eating experience while returning money to the farming operation.

For more information on Comprehensive Automation for Specialty Crops, please visit www.cascrop.com.

Leave a Reply

Featured Stories
Berries
February 26, 2017
Advantages to Growing Southern Highbush Blueberries
Growers in some regions where temperatures don’t get too low could consider the benefits of Southern highbush varieties. Read More
Grapes
February 26, 2017
Winegrowers See Total Sustainability in Their Sights
Sonoma County Winegrowers eyes 2019 to become the U.S.’s first completely sustainable wine region. Read More
Fruits
February 25, 2017
Let’s Stand Up for Science
Yes, there’s an art to agriculture, but its very definition is the science of farming, so you have a deeply rooted connection to scientists. Read More
Florida citrus grove
Citrus
February 25, 2017
Citrus Research Investments Yielding Positive Results
Despite still having mounds of wood to chop, grower, federal, and state programs collectively are making a dent. Read More
Fruits
February 25, 2017
Major Challenges to the Fruit Industry Are Closer than You Think
Don’t underestimate the need for research, as innovation is becoming an urgent necessity for our future. Read More
iphone close-up
GenNext Growers
February 24, 2017
New Mobile Apps for Farmers to Grow on the Go
Thanks to the latest in software technology, real-time decisions can be made with one swipe of your finger. Read More
FFVA Emerging Leader Development Program Class 6 Farm Tour
GenNext Growers
February 24, 2017
In-The-Field Learning Invaluable for GenNext Growers
Annual tour allows Florida's future farming leaders the chance to have open and honest discussions with CEOs and managers and tap into their knowledge. Read More
Grower Achievement Award
February 24, 2017
Know a Vegetable Grower Who’s Innovative and Stands Above the Rest?
Then nomination him or her for American Vegetable Grower magazine’s 2017 Grower Achievement Award. Read More
Apples & Pears
February 24, 2017
Don’t Underestimate the Importance of a Solid Foundation in Your Orchard
Trellising study helps growers understand the basics of setting up an orchard infrastructure properly. Read More
Berries
February 23, 2017
Primocane: A Major Innovation in Berry Production
I’ve witnessed the rise of primocane-fruiting blackberries, from first selections to integration in the market. Read More
The Latest
Berries
February 26, 2017
Advantages to Growing Southern Highbush …
Growers in some regions where temperatures don’t get too low could consider the benefits of Southern highbush varieties. Read More
Grapes
February 26, 2017
Winegrowers See Total Sustainability in …
Sonoma County Winegrowers eyes 2019 to become the U.S.’s first completely sustainable wine region. Read More
Fruits
February 25, 2017
Let’s Stand Up for Science
Yes, there’s an art to agriculture, but its very definition is the science of farming, so you have a deeply rooted connection to scientists. Read More
Citrus
February 25, 2017
Citrus Research Investments Yielding Pos…
Despite still having mounds of wood to chop, grower, federal, and state programs collectively are making a dent. Read More
Fruits
February 25, 2017
Major Challenges to the Fruit Industry A…
Don’t underestimate the need for research, as innovation is becoming an urgent necessity for our future. Read More
GenNext Growers
February 24, 2017
New Mobile Apps for Farmers to Grow on t…
Thanks to the latest in software technology, real-time decisions can be made with one swipe of your finger. Read More
GenNext Growers
February 24, 2017
In-The-Field Learning Invaluable for Gen…
Annual tour allows Florida's future farming leaders the chance to have open and honest discussions with CEOs and managers and tap into their knowledge. Read More
Grower Achievement Award
February 24, 2017
Know a Vegetable Grower Who’s Innovative…
Then nomination him or her for American Vegetable Grower magazine’s 2017 Grower Achievement Award. Read More
Apples & Pears
February 24, 2017
Don’t Underestimate the Importance…
Trellising study helps growers understand the basics of setting up an orchard infrastructure properly. Read More
Berries
February 23, 2017
Primocane: A Major Innovation in Berry P…
I’ve witnessed the rise of primocane-fruiting blackberries, from first selections to integration in the market. Read More
Fruits
February 23, 2017
First California Ag Innovation Showcase …
University of California, Davis will be the setting for a ground-breaking conference on emerging technology for food and high-value agriculture. Read More
Disease Control
February 23, 2017
Spot Sclerotinia Before It Stops Your Ve…
Learn how to identify, the survival and spread, as well as management methods for this malady that goes by many names. Read More
Berries
February 22, 2017
Florida Strawberry Picking Contest a Big…
Fourth-annual fundraising event brings in more than $100,000 to benefit children of local farmworkers. Read More
Citrus
February 22, 2017
New Insecticide Available to Combat Suck…
Minecto Pro from Syngenta has received EPA approval for use in specialty and vegetable crops to control lepidopteran and other pests. Read More
Farm Management
February 22, 2017
Farms Get Bigger Even as the Number of F…
A new report from USDA shows long-term trends continuing: fewer farms and fewer acres used in farming even as individual operations get bigger on average. Read More
Farm Management
February 22, 2017
Wild Bee Populations Are out of Sync wit…
Some crops most dependent on pollinators, including pumpkins, watermelons, pears, peaches, plums, apples and blueberries, appeared to have the strongest pollination mismatch. Read More
Citrus
February 22, 2017
Skilled Labor Needed to Keep Agriculture…
Even with the challenge of feeding a growing global population at hand, has focus been lost on recruiting where jobs on the farm are need most? Read More
Varieties & Rootstocks
February 22, 2017
Citrus Nurseries Analyze the Need for Se…
It seems many growers are more certain about their scion choices than they are about available rootstocks. Read More