Apple Harvesters Use Vision To “See” Fruit

Editor’s Note: This past October, Tony Koselka, Jillian Cannons, and Ryan Carlon of Vision Robotics Corporation (VRC) returned to Washington for further field trials of the Newton Scout, a vision-based system for automatic crop load assessment. The following report highlights their results.

Developed in part through the USDA-funded project Comprehensive Automation for Specialty Crops (CASC), the Newton Scout will revolutionize orchard operations by providing growers with a map that accurately maps fruit placement, fruit size, and number throughout the growing season. Fruit maps will be used to determine crop load, which in turn will give growers the information to develop harvest and marketing strategies. And this is just the beginning. Developing reliable vision systems that are robust enough for use in orchards opens up all kinds of possibilities in the area of pest and disease detection, plant health assessment, and the use of robotics at harvest.

The Newton Scout integrates stereo cameras with a custom image processing algorithm. The algorithms use classifiers to identify the apples, where the simplest classifier uses color and shape. For red apples, color is the primary classifier. For detecting green fruit, the Scout detects using statistical classifiers generated by boosting algorithms. The most challenging aspects of detecting fruit are occlusions, shadows, shape and color coincidences, and ambient lighting conditions such as bright sunlight. Once an apple is identified, the stereo cameras enable the system to determine its location and size.

The Vision Robotics field trial was conducted at Allan Brothers Othello Orchard in Washington. Prior to VRC’s arrival, Washington State University (WSU) Extension Educator Karen Lewis, and interns with the Washington Tree Fruit Research Commission, hand tagged, mapped, and sized fruit on several rows of fruit. This data allows for the groundtruthing or validation component of the test. Koselka took time out to speak about the development of a similar unit that counts oranges with the interns.

On one of the field trials for that unit, Koselka asked the grower to estimate his crop. The farm manager responded with 1,200 bushels. For this test, Vision Robotics scouted and counted 1,900 — a sizable difference and one that would have a large financial impact.

As Koselka stated, “not only will it [Scout] be able to accurately count a crop and locate each piece of fruit, it will ultimately feed into a robot harvester that will pick all the apples.” And if that was not enough, Koselka added, “Oh, and a pruner is not far off either!”

Even More Potential

The Newton Scout utility will be enhanced in the future with the addition of other possible technologies, including near infrared (NIR). As Newton evolves, it will have multiple uses throughout the fruit growing cycle — for example, assessment of sugar and acid content and internal damage. CASC is also making progress in other automation fronts that bridge with Vision Robotics’ work.

Carnegie Mellon University (CMU) researchers, led by Sanjiv Singh, are developing an Autonomous Prime Mover (APM) capable of towing the Newton Scout safely through entire orchard blocks. In the future, they will also automate an agricultural harvesting platform so that workers no longer need to climb on ladders to pick fruit. The APM’s multifunctionality, autonomous drive, and wireless networking could move it into the center of precision management of specialty crops.

The CASC team, led by CMU, includes engineers, scientists, Extension educators, growers, equipment manufacturers, and industry representatives. This plurality of expertise has led to synergistic endeavors that push the envelope in both the engineering and the science components of the project. For example, Ben Grocholsky at CMU is developing a geospatial system to map and locate each tree in an orchard; in cooperation with the plant science group, he was able to augment this information with local temperature and NDVI data that has the future possibility of being incorporated to detect plant stress and disease.

In another example of the synergy of the CASC team, during a field trial of an automated caliper measurement device, growers told the CASC team that a simpler tree counter would also be very useful. Singh’s group returned to CMU and produced a prototype tree counter ready to be tested in weeks; they will return to nurseries and test the counter this fall. The total time from conception to development will be less than four months, and if successful, this device could go into production within a year.

It will only be a matter of time before specialty crop growers also start using vehicles, sensors, devices, and software that automate farm operations, increase farm efficiency, and meet the ultimate goal of delivering to the consumer a consistently high quality affordable eating experience while returning money to the farming operation.

For more information on Comprehensive Automation for Specialty Crops, please visit www.cascrop.com.

Leave a Reply

Featured Stories
Biocontrols 2015 Conference
CitrusGrowers Go In-Depth At Biocontrols 2015 Conference & Tradeshow
March 9, 2015
The first-of-its-kind event brought growers, researchers, regulators, and suppliers together for a two-day discussion on the latest biocontrols tools and technologies. Read More
Crop ProtectionAt Paramount Farming, Mating Disruption Pays Off
February 10, 2015
To say Brad Higbee is an expert in mating disruption, the use of pheromones to confuse target pests and reduce Read More
Crop ProtectionGrower Experienced In Biocontrols Says: Be Patient
February 4, 2015
The use of biocontrols in all aspects of agriculture is becoming more widespread. But incorporating biocontrols in a vegetable farming Read More
Richard Jones
Farm ManagementDon’t Burn Out This Spring
March 30, 2015
Finding ways to limit stress will benefit both you and your business. Read More
Apples & PearsSounding Off On GMOs: The Arctic Apple Decision
March 27, 2015
Industry leaders share their feelings on the recently approved genetically modified apples. Read More
BerriesBuilding A Better Black Raspberry
March 27, 2015
Studies focusing on consumer preference, aphid resistance, heat tolerance, and genetics aim to capitalize on an expanding market. Read More
CitrusMore California Citrus Placed Under Psyllid Quarantine
March 27, 2015
Asian citrus psyllid is found in Madera County – the heart of the San Joaquin Valley. Read More
FruitsAhead Of The Curve, Part 2
March 27, 2015
Get educated with great resources in the tree fruit industry available at your fingertips. Read More
Apples & PearsMeet USApple’s Class of 2015 Young Apple Leaders
March 27, 2015
Young decision-makers see trip as an opportunity make sure the voice of the apple industry is heard by representatives. Read More
GrapesGlassy-Winged Sharpshooter Found In Marin County
March 26, 2015
This pest, known to spread Pierce’s Disease, could devastate California winegrapes. Read More
GrapesFirst Raisin Grape That Dries Naturally On Vine Developed
March 26, 2015
The new Sunpreme variety eliminates the need for growers to cut into woody canes before harvest. Read More
Fresh From Florida Strawberry Parfait recipe
BerriesNew Partnership Sends Florida Strawberries To Puerto Rico
March 26, 2015
Latest addition extends Sunshine State’s agricultural export reach in the Caribbean. Read More
drought management; irrigation; water management
FruitsHow To Become A Soil Sleuth
March 26, 2015
To correctly apply the proper nutrients at the right rates, you need to know where your fields are deficient and how to correctly diagnose a disorder. Read More
The Latest
Richard Jones
Farm ManagementDon’t Burn Out This Spring
March 30, 2015
Finding ways to limit stress will benefit both you and your business. Read More
Apples & PearsSounding Off On GMOs: The Arctic Apple Decision
March 27, 2015
Industry leaders share their feelings on the recently approved genetically modified apples. Read More
BerriesBuilding A Better Black Raspberry
March 27, 2015
Studies focusing on consumer preference, aphid resistance, heat tolerance, and genetics aim to capitalize on an expanding market. Read More
CitrusMore California Citrus Placed Under Psyllid Quarantine
March 27, 2015
Asian citrus psyllid is found in Madera County – the heart of the San Joaquin Valley. Read More
FruitsAhead Of The Curve, Part 2
March 27, 2015
Get educated with great resources in the tree fruit industry available at your fingertips. Read More
Apples & PearsMeet USApple’s Class of 2015 Young Apple Leaders
March 27, 2015
Young decision-makers see trip as an opportunity make sure the voice of the apple industry is heard by representatives. Read More
GrapesGlassy-Winged Sharpshooter Found In Marin County
March 26, 2015
This pest, known to spread Pierce’s Disease, could devastate California winegrapes. Read More
GrapesFirst Raisin Grape That Dries Naturally On Vine Develop…
March 26, 2015
The new Sunpreme variety eliminates the need for growers to cut into woody canes before harvest. Read More
Fresh From Florida Strawberry Parfait recipe
BerriesNew Partnership Sends Florida Strawberries To Puerto Ri…
March 26, 2015
Latest addition extends Sunshine State’s agricultural export reach in the Caribbean. Read More
drought management; irrigation; water management
FruitsHow To Become A Soil Sleuth
March 26, 2015
To correctly apply the proper nutrients at the right rates, you need to know where your fields are deficient and how to correctly diagnose a disorder. Read More
NutsWhy You Should Use Low-Rate Applications Of Potassium
March 26, 2015
These rates provide growers a low-cost method comparable to conventional applications. Read More
Cold ProtectionWind Machine, Electric Pruners Donated To MSU Research …
March 25, 2015
New equipment to help protect research blocks from frost damage and provide center with latest technology. Read More
CitrusProposed Farm Bill Provision To Limit Payments To Non-F…
March 25, 2015
USDA has proposed a new rule which would limit payment to those who are only actively engaged in farming. Read More
Pennsylvania hard cider guild logo
Apples & PearsPennsylvania Hard Cider Producers Founded Independent T…
March 25, 2015
The Pennsylvania Cider Guild is designed to help promote and educate members and consumers. Read More
FruitsHow To Get Started With Partnerships And Sponsorships
March 25, 2015
Mark Saunders of Saunders Farm says your business is never too small or too big to get more value for what you do on your farm. Read More
FruitsValue Through Partnerships And Sponsorships [Opinion]
March 25, 2015
Mark Saunders of Saunders Farms says admission is one way to get value for what you do, but sponsorships and partnerships are the next level. Read More
chateau hough urban vineyard Cleveland Ohio
GrapesFrom Riots To Vineyard — A Story of Urban Revival
March 25, 2015
Ex-convict’s inner-city vineyard in the Hough neighborhood in Cleveland an opportunity for those leaving incarceration for a bright future in viticulture. Read More
FruitsWorld Ag Expo 2015: New Tools For Fruit Growers
March 24, 2015
From wind machines to solar-powered pumps, here is some of the new equipment on display this year's edition of the nation's largest farm show. Read More