Apple Harvesters Use Vision To “See” Fruit

Editor’s Note: This past October, Tony Koselka, Jillian Cannons, and Ryan Carlon of Vision Robotics Corporation (VRC) returned to Washington for further field trials of the Newton Scout, a vision-based system for automatic crop load assessment. The following report highlights their results.

Developed in part through the USDA-funded project Comprehensive Automation for Specialty Crops (CASC), the Newton Scout will revolutionize orchard operations by providing growers with a map that accurately maps fruit placement, fruit size, and number throughout the growing season. Fruit maps will be used to determine crop load, which in turn will give growers the information to develop harvest and marketing strategies. And this is just the beginning. Developing reliable vision systems that are robust enough for use in orchards opens up all kinds of possibilities in the area of pest and disease detection, plant health assessment, and the use of robotics at harvest.

The Newton Scout integrates stereo cameras with a custom image processing algorithm. The algorithms use classifiers to identify the apples, where the simplest classifier uses color and shape. For red apples, color is the primary classifier. For detecting green fruit, the Scout detects using statistical classifiers generated by boosting algorithms. The most challenging aspects of detecting fruit are occlusions, shadows, shape and color coincidences, and ambient lighting conditions such as bright sunlight. Once an apple is identified, the stereo cameras enable the system to determine its location and size.

The Vision Robotics field trial was conducted at Allan Brothers Othello Orchard in Washington. Prior to VRC’s arrival, Washington State University (WSU) Extension Educator Karen Lewis, and interns with the Washington Tree Fruit Research Commission, hand tagged, mapped, and sized fruit on several rows of fruit. This data allows for the groundtruthing or validation component of the test. Koselka took time out to speak about the development of a similar unit that counts oranges with the interns.

On one of the field trials for that unit, Koselka asked the grower to estimate his crop. The farm manager responded with 1,200 bushels. For this test, Vision Robotics scouted and counted 1,900 — a sizable difference and one that would have a large financial impact.

As Koselka stated, “not only will it [Scout] be able to accurately count a crop and locate each piece of fruit, it will ultimately feed into a robot harvester that will pick all the apples.” And if that was not enough, Koselka added, “Oh, and a pruner is not far off either!”

Even More Potential

The Newton Scout utility will be enhanced in the future with the addition of other possible technologies, including near infrared (NIR). As Newton evolves, it will have multiple uses throughout the fruit growing cycle — for example, assessment of sugar and acid content and internal damage. CASC is also making progress in other automation fronts that bridge with Vision Robotics’ work.

Carnegie Mellon University (CMU) researchers, led by Sanjiv Singh, are developing an Autonomous Prime Mover (APM) capable of towing the Newton Scout safely through entire orchard blocks. In the future, they will also automate an agricultural harvesting platform so that workers no longer need to climb on ladders to pick fruit. The APM’s multifunctionality, autonomous drive, and wireless networking could move it into the center of precision management of specialty crops.

The CASC team, led by CMU, includes engineers, scientists, Extension educators, growers, equipment manufacturers, and industry representatives. This plurality of expertise has led to synergistic endeavors that push the envelope in both the engineering and the science components of the project. For example, Ben Grocholsky at CMU is developing a geospatial system to map and locate each tree in an orchard; in cooperation with the plant science group, he was able to augment this information with local temperature and NDVI data that has the future possibility of being incorporated to detect plant stress and disease.

In another example of the synergy of the CASC team, during a field trial of an automated caliper measurement device, growers told the CASC team that a simpler tree counter would also be very useful. Singh’s group returned to CMU and produced a prototype tree counter ready to be tested in weeks; they will return to nurseries and test the counter this fall. The total time from conception to development will be less than four months, and if successful, this device could go into production within a year.

It will only be a matter of time before specialty crop growers also start using vehicles, sensors, devices, and software that automate farm operations, increase farm efficiency, and meet the ultimate goal of delivering to the consumer a consistently high quality affordable eating experience while returning money to the farming operation.

For more information on Comprehensive Automation for Specialty Crops, please visit www.cascrop.com.

Leave a Reply

Featured Stories
New citrus grove in Florida
Citrus
August 24, 2016
Florida Citrus Granted New Incentive To Grow On
A state-backed $5.5 million grove renovation/re-establishment incentive is now available and accepting applications. Read More
Judi Whitson, 2016 Florida Woman of the Year in Agriculture
Citrus
August 24, 2016
Meet Florida’s 2016 Woman Of The Year In Agriculture
Dedication to fostering the state’s farming future defines the latest winner of longtime honor. Read More
University of New Hampshire scientists constructed a linkage map of the seven chromosomes of the diploid Fragaria iinumae, which allows them to fill in a piece of the genetic puzzle about the eight sets of chromosomes of the cultivated strawberry.
Berries
August 24, 2016
Researchers Unravel Genetic Ancestry Of Cultivated Strawberry
A genetic analysis conducted by New Hampshire Agricultural Experiment Station researchers aims to improve modern cultivation efforts of strawberry growers. Read More
Justin Clements, a fifth-year doctoral student in the Molecular and Environmental Toxicology Center working in the Department of Entomology at the University of Wisconsin-Madison, is the recipient of NPC's 2016-2017 Academic Scholarship.
Potatoes
August 24, 2016
Fifth-Year Doctoral Student Receives NPC’s Scholarship For Potato Research
The National Potato Council (NPC) has announced that Justin Clements, a fifth-year doctoral student in the Molecular and Environmental Toxicology Read More
2011 WA IFTA Tour
Fruits
August 24, 2016
Don’t Underestimate The Value Of Production Bus Tours For Your Business [Opinion]
Sure, you can see and talk with virtually anyone in the world through your smart phone, but you can’t shake their hand. Read More
Crowd protesting GMOs stock image FEATURE
Farm Management
August 24, 2016
Swaying Views On GMO Foods No Easy Task
Featured speaker for the upcoming FFVA Annual Convention says tides are slowly turning in how people digest the concept of genetic modification. Read More
A subsurface microirrigation system.
(Photo credit: Peter Jacoby)
Grapes
August 23, 2016
Deeper Irrigation Method Showing Promise For Vineyards
A new subsurface irrigation system is showing promise for slashing water usage in vineyards. Many vineyards use drip lines that Read More
Tar spot of corn
Disease Control
August 23, 2016
Florida Sweet Corn Has New Deadly Stalker In Tar Spot
Learn how to identify, the survival and spread, as well as management methods for this disease. Read More
sprayer nozzles
Fruits
August 23, 2016
New Insecticides Geared To Give Growers Edge Over Pests
Check out three unique chemistries from several of the industry's leading crop protection suppliers. Read More
Prepping apples for cleaning, waxing, and sorting.
Apple Grower of the Year
August 22, 2016
Apple Crop Forecasts Live On Twitter
You'll be able to get up-to-the-minute reports of not only the various regions of the U.S., but key production regions around the world. Read More
The Latest
Citrus
August 24, 2016
Florida Citrus Granted New Incentive To …
A state-backed $5.5 million grove renovation/re-establishment incentive is now available and accepting applications. Read More
Citrus
August 24, 2016
Meet Florida’s 2016 Woman Of The Year In…
Dedication to fostering the state’s farming future defines the latest winner of longtime honor. Read More
Berries
August 24, 2016
Researchers Unravel Genetic Ancestry Of …
A genetic analysis conducted by New Hampshire Agricultural Experiment Station researchers aims to improve modern cultivation efforts of strawberry growers. Read More
Potatoes
August 24, 2016
Fifth-Year Doctoral Student Receives NPC…
The National Potato Council (NPC) has announced that Justin Clements, a fifth-year doctoral student in the Molecular and Environmental Toxicology Read More
Fruits
August 24, 2016
Don’t Underestimate The Value Of P…
Sure, you can see and talk with virtually anyone in the world through your smart phone, but you can’t shake their hand. Read More
Farm Management
August 24, 2016
Swaying Views On GMO Foods No Easy Task
Featured speaker for the upcoming FFVA Annual Convention says tides are slowly turning in how people digest the concept of genetic modification. Read More
Grapes
August 23, 2016
Deeper Irrigation Method Showing Promise…
A new subsurface irrigation system is showing promise for slashing water usage in vineyards. Many vineyards use drip lines that Read More
Disease Control
August 23, 2016
Florida Sweet Corn Has New Deadly Stalke…
Learn how to identify, the survival and spread, as well as management methods for this disease. Read More
Fruits
August 23, 2016
New Insecticides Geared To Give Growers …
Check out three unique chemistries from several of the industry's leading crop protection suppliers. Read More
Apple Grower of the Year
August 22, 2016
Apple Crop Forecasts Live On Twitter
You'll be able to get up-to-the-minute reports of not only the various regions of the U.S., but key production regions around the world. Read More
Oranges
August 22, 2016
Major OJ Purchase Promises Relief For Fl…
$30 million planned acquisition from USDA to be used for surplus removal. Read More
Varieties & Rootstocks
August 22, 2016
Florida Citrus Breeders Squeezed For Tim…
The pressure is on to develop a more HLB-tolerant orange. Read More
Pest Control
August 21, 2016
Spotted Wing Drosophila: For Michigan, …
Why the pest can be so much more destructive for Eastern cherry growers might be due to what they’re not farming. Read More
Grapes
August 19, 2016
European Grapevine Moth Eradicated From …
Agricultural officials confirm eradication of invasive pest, lift quarantine restrictions. Read More
Fruits
August 19, 2016
Parts Of New York Now In Extreme Drought
Water deficits starting to cause concern for fire, water shortages in the Empire State. Read More
Citrus
August 19, 2016
Rough Winter In Store For Much Of The U.…
The Old Farmer’s Almanac and its competitor The Farmers’ Almanac predict the 2016-2017 winter will be cold for much of the country. Read More
Citrus
August 18, 2016
Florida Farmer Takes Fresh Approach To G…
New easy-peel, seedless varieties offer market potential. Read More
GenNext Growers
August 18, 2016
USDA Invests $17.8M To Educate Next Gene…
Money will bolster efforts of partner organizations to increase next generation of growers. Read More
[gravityform id="62" title="false" description="false"]