Apple Harvesters Use Vision To “See” Fruit

Editor’s Note: This past October, Tony Koselka, Jillian Cannons, and Ryan Carlon of Vision Robotics Corporation (VRC) returned to Washington for further field trials of the Newton Scout, a vision-based system for automatic crop load assessment. The following report highlights their results.

Developed in part through the USDA-funded project Comprehensive Automation for Specialty Crops (CASC), the Newton Scout will revolutionize orchard operations by providing growers with a map that accurately maps fruit placement, fruit size, and number throughout the growing season. Fruit maps will be used to determine crop load, which in turn will give growers the information to develop harvest and marketing strategies. And this is just the beginning. Developing reliable vision systems that are robust enough for use in orchards opens up all kinds of possibilities in the area of pest and disease detection, plant health assessment, and the use of robotics at harvest.

The Newton Scout integrates stereo cameras with a custom image processing algorithm. The algorithms use classifiers to identify the apples, where the simplest classifier uses color and shape. For red apples, color is the primary classifier. For detecting green fruit, the Scout detects using statistical classifiers generated by boosting algorithms. The most challenging aspects of detecting fruit are occlusions, shadows, shape and color coincidences, and ambient lighting conditions such as bright sunlight. Once an apple is identified, the stereo cameras enable the system to determine its location and size.

The Vision Robotics field trial was conducted at Allan Brothers Othello Orchard in Washington. Prior to VRC’s arrival, Washington State University (WSU) Extension Educator Karen Lewis, and interns with the Washington Tree Fruit Research Commission, hand tagged, mapped, and sized fruit on several rows of fruit. This data allows for the groundtruthing or validation component of the test. Koselka took time out to speak about the development of a similar unit that counts oranges with the interns.

On one of the field trials for that unit, Koselka asked the grower to estimate his crop. The farm manager responded with 1,200 bushels. For this test, Vision Robotics scouted and counted 1,900 — a sizable difference and one that would have a large financial impact.

As Koselka stated, “not only will it [Scout] be able to accurately count a crop and locate each piece of fruit, it will ultimately feed into a robot harvester that will pick all the apples.” And if that was not enough, Koselka added, “Oh, and a pruner is not far off either!”

Even More Potential

The Newton Scout utility will be enhanced in the future with the addition of other possible technologies, including near infrared (NIR). As Newton evolves, it will have multiple uses throughout the fruit growing cycle — for example, assessment of sugar and acid content and internal damage. CASC is also making progress in other automation fronts that bridge with Vision Robotics’ work.

Carnegie Mellon University (CMU) researchers, led by Sanjiv Singh, are developing an Autonomous Prime Mover (APM) capable of towing the Newton Scout safely through entire orchard blocks. In the future, they will also automate an agricultural harvesting platform so that workers no longer need to climb on ladders to pick fruit. The APM’s multifunctionality, autonomous drive, and wireless networking could move it into the center of precision management of specialty crops.

The CASC team, led by CMU, includes engineers, scientists, Extension educators, growers, equipment manufacturers, and industry representatives. This plurality of expertise has led to synergistic endeavors that push the envelope in both the engineering and the science components of the project. For example, Ben Grocholsky at CMU is developing a geospatial system to map and locate each tree in an orchard; in cooperation with the plant science group, he was able to augment this information with local temperature and NDVI data that has the future possibility of being incorporated to detect plant stress and disease.

In another example of the synergy of the CASC team, during a field trial of an automated caliper measurement device, growers told the CASC team that a simpler tree counter would also be very useful. Singh’s group returned to CMU and produced a prototype tree counter ready to be tested in weeks; they will return to nurseries and test the counter this fall. The total time from conception to development will be less than four months, and if successful, this device could go into production within a year.

It will only be a matter of time before specialty crop growers also start using vehicles, sensors, devices, and software that automate farm operations, increase farm efficiency, and meet the ultimate goal of delivering to the consumer a consistently high quality affordable eating experience while returning money to the farming operation.

For more information on Comprehensive Automation for Specialty Crops, please visit www.cascrop.com.

Leave a Reply

Featured Stories
FruitsCalifornia Drought Meeting Called
January 30, 2015
State and federal officials will discuss potential actions for the coming year. Read More
GrapesRecord Crowds Flock To California Winegrape Show
January 30, 2015
Unified Wine & Grape Symposium attracts 14,000 industry professionals. Read More
Insect & Disease UpdateResearchers Develop Electronic Citrus Greening Detector
January 30, 2015
New, easy-to-use technology could help growers get a leg up on the disease. Read More
VegetablesSakata Seed Highlights Specialty Golden Beet
January 29, 2015
The company highlights specialty golden beet, Touchstone Gold. Read More
FruitsUse The Off-Season To Focus On Farm Safety
January 29, 2015
Prevention, planning, and preparedness are essential for all agritainment operations. Read More
Protected AgricultureHow To Successfully Produce Vegetables In Controlled Environments
January 28, 2015
Greenhouse expert Merle Jensen offers pointers on growing systems, pest control, plant nutrition, harvesting, and more. Read More
IPMWhat You Need To Know To Maintain An Effective IPM Program In The Greenhouse
January 28, 2015
Greenhouse expert Karin Tifft discusses the importance of scouting, pest identification, control options, and the importance of sanitation. Read More
Apples & PearsPears Get A Hefty Shout-Out From Rapper
January 28, 2015
The pome fruit is on mainstream media's radar thanks to a rapper's quote and ensuing social media blitz. Read More
FruitsHighlights From The Mid-Atlantic Fruit And Vegetable Convention
January 28, 2015
Bio-control use, media literacy, and disease control were just a few of the topics discussed at the convention held in Hershey, PA. Read More
GrapesCalifornia Table Grape Growers Offer Agricultural Scholarships
January 28, 2015
The application deadline for the agricultural scholarships is 5 p.m., Feb. 13. Read More
The Latest
FruitsCalifornia Drought Meeting Called
January 30, 2015
State and federal officials will discuss potential actions for the coming year. Read More
GrapesRecord Crowds Flock To California Winegrape Show
January 30, 2015
Unified Wine & Grape Symposium attracts 14,000 industry professionals. Read More
Insect & Disease UpdateResearchers Develop Electronic Citrus Greening Detector
January 30, 2015
New, easy-to-use technology could help growers get a leg up on the disease. Read More
VegetablesSakata Seed Highlights Specialty Golden Beet
January 29, 2015
The company highlights specialty golden beet, Touchstone Gold. Read More
FruitsUse The Off-Season To Focus On Farm Safety
January 29, 2015
Prevention, planning, and preparedness are essential for all agritainment operations. Read More
Protected AgricultureHow To Successfully Produce Vegetables In Controlled En…
January 28, 2015
Greenhouse expert Merle Jensen offers pointers on growing systems, pest control, plant nutrition, harvesting, and more. Read More
IPMWhat You Need To Know To Maintain An Effective IPM Prog…
January 28, 2015
Greenhouse expert Karin Tifft discusses the importance of scouting, pest identification, control options, and the importance of sanitation. Read More
Apples & PearsPears Get A Hefty Shout-Out From Rapper
January 28, 2015
The pome fruit is on mainstream media's radar thanks to a rapper's quote and ensuing social media blitz. Read More
FruitsHighlights From The Mid-Atlantic Fruit And Vegetable Co…
January 28, 2015
Bio-control use, media literacy, and disease control were just a few of the topics discussed at the convention held in Hershey, PA. Read More
GrapesCalifornia Table Grape Growers Offer Agricultural Schol…
January 28, 2015
The application deadline for the agricultural scholarships is 5 p.m., Feb. 13. Read More
GrapesNew Wine Hybrid Establishment Cost Resource Available
January 28, 2015
Research from Cornell University breaks down the cost of producing hybrid winegrapes. Read More
FertilizerEasy Access To Fertilization Guidelines
January 28, 2015
New, easy-to-follow brochures take the guesswork out of almond and walnut fertilization. Read More
BerriesDormant Pruning Tips For Your Berries
January 28, 2015
Proper dormant pruning increases berry size and improves overall health of blackberry and raspberry plants. Read More
NutsHow Freezing Temperature Affect Bearing and New Trees
January 28, 2015
Key points before and after the critical bloom period make exposure to freezing temperatures relevant. Read More
CitrusFlorida Citrus Is Still A Big Deal [Opinion]
January 28, 2015
Even while in the grips of greening, citrus continues to be a big contributor to the Sunshine State's economy. Read More
FruitsUSDA Forecast Predicts Normal Water Supply For Parts Of…
January 27, 2015
The Southwest, Sierra Nevada region, and Pacific Northwest are starting the year drier than normal. Read More
Apples & PearsThe Story Of Honeycrisp Is One Of Challenges And Succes…
January 27, 2015
‘Everything’s different’ in this world of 150 million 40-pound box world of apple growing. Read More
BerriesWhy You Should Be Growing Black Raspberries
January 27, 2015
Amazing health benefits for consumers and production and marketing benefits for the grower make this berry a must-consider option for your operation. Read More