Apple Harvesters Use Vision To “See” Fruit

Editor’s Note: This past October, Tony Koselka, Jillian Cannons, and Ryan Carlon of Vision Robotics Corporation (VRC) returned to Washington for further field trials of the Newton Scout, a vision-based system for automatic crop load assessment. The following report highlights their results.

Developed in part through the USDA-funded project Comprehensive Automation for Specialty Crops (CASC), the Newton Scout will revolutionize orchard operations by providing growers with a map that accurately maps fruit placement, fruit size, and number throughout the growing season. Fruit maps will be used to determine crop load, which in turn will give growers the information to develop harvest and marketing strategies. And this is just the beginning. Developing reliable vision systems that are robust enough for use in orchards opens up all kinds of possibilities in the area of pest and disease detection, plant health assessment, and the use of robotics at harvest.

The Newton Scout integrates stereo cameras with a custom image processing algorithm. The algorithms use classifiers to identify the apples, where the simplest classifier uses color and shape. For red apples, color is the primary classifier. For detecting green fruit, the Scout detects using statistical classifiers generated by boosting algorithms. The most challenging aspects of detecting fruit are occlusions, shadows, shape and color coincidences, and ambient lighting conditions such as bright sunlight. Once an apple is identified, the stereo cameras enable the system to determine its location and size.

The Vision Robotics field trial was conducted at Allan Brothers Othello Orchard in Washington. Prior to VRC’s arrival, Washington State University (WSU) Extension Educator Karen Lewis, and interns with the Washington Tree Fruit Research Commission, hand tagged, mapped, and sized fruit on several rows of fruit. This data allows for the groundtruthing or validation component of the test. Koselka took time out to speak about the development of a similar unit that counts oranges with the interns.

On one of the field trials for that unit, Koselka asked the grower to estimate his crop. The farm manager responded with 1,200 bushels. For this test, Vision Robotics scouted and counted 1,900 — a sizable difference and one that would have a large financial impact.

As Koselka stated, “not only will it [Scout] be able to accurately count a crop and locate each piece of fruit, it will ultimately feed into a robot harvester that will pick all the apples.” And if that was not enough, Koselka added, “Oh, and a pruner is not far off either!”

Even More Potential

The Newton Scout utility will be enhanced in the future with the addition of other possible technologies, including near infrared (NIR). As Newton evolves, it will have multiple uses throughout the fruit growing cycle — for example, assessment of sugar and acid content and internal damage. CASC is also making progress in other automation fronts that bridge with Vision Robotics’ work.

Carnegie Mellon University (CMU) researchers, led by Sanjiv Singh, are developing an Autonomous Prime Mover (APM) capable of towing the Newton Scout safely through entire orchard blocks. In the future, they will also automate an agricultural harvesting platform so that workers no longer need to climb on ladders to pick fruit. The APM’s multifunctionality, autonomous drive, and wireless networking could move it into the center of precision management of specialty crops.

The CASC team, led by CMU, includes engineers, scientists, Extension educators, growers, equipment manufacturers, and industry representatives. This plurality of expertise has led to synergistic endeavors that push the envelope in both the engineering and the science components of the project. For example, Ben Grocholsky at CMU is developing a geospatial system to map and locate each tree in an orchard; in cooperation with the plant science group, he was able to augment this information with local temperature and NDVI data that has the future possibility of being incorporated to detect plant stress and disease.

In another example of the synergy of the CASC team, during a field trial of an automated caliper measurement device, growers told the CASC team that a simpler tree counter would also be very useful. Singh’s group returned to CMU and produced a prototype tree counter ready to be tested in weeks; they will return to nurseries and test the counter this fall. The total time from conception to development will be less than four months, and if successful, this device could go into production within a year.

It will only be a matter of time before specialty crop growers also start using vehicles, sensors, devices, and software that automate farm operations, increase farm efficiency, and meet the ultimate goal of delivering to the consumer a consistently high quality affordable eating experience while returning money to the farming operation.

For more information on Comprehensive Automation for Specialty Crops, please visit www.cascrop.com.

Leave a Reply

Featured Stories

VegetablesSeminis Hosts Growers For Its 2014 Southeastern Field Day
December 19, 2014
Seed company showcases new varieties available now and in the pipeline. Read More
CitrusFeed The People With Food And Food For Thought [Opinion]
December 19, 2014
Educating the general public about agriculture is a daunting task that can be accomplished one bite at a time. Read More
CucurbitsNew Product Takes Aim At Nematodes
December 19, 2014
NIMITZ receives EPA and Florida registrations. Read More
Crop ProtectionSuccess Sans Methyl Bromide Could Prove Costly
December 19, 2014
Economic factors must be accounted for when seeking a suitable replacement. Read More
VegetablesEnza Zaden Cuts Ribbon For New Florida Vegetable Research Facility
December 18, 2014
Seed company shows off new digs and new varieties at its Myakka City field station. Read More
VegetablesRispens Seeds Highlights Their Latest Vegetable Varieties
December 18, 2014
For more information, contact: Rispens Seeds 1357 Dutch American Way, P.O. Box 310, Beecher, IL 60401 708-946-6560; 888-874-0241; fax: 708-946-6115 Read More
Protected Agriculture2014 Marks The 50th Anniversary Of The Double-Poly Greenhouse [sponsor content]
December 18, 2014
Double-poly greenhouses revolutionized the greenhouse industry back in the 1960s, opening the way for developments that still impact the industry 50 years later. Read More
CitrusSouthwest Florida Citrus Bus Tour Yields Learning Opportunity
December 18, 2014
Find out what Florida Grower editor Frank Giles saw and heard during Peace River Valley Citrus Growers Association's annual outing. Read More
Fruits6 Food Trends For 2015
December 17, 2014
University of Florida experts predict what will be cooking for the upcoming year. Read More
VegetablesReed’s Seeds Showcases Their Latest Vegetable Variety Offerings
December 17, 2014
For more information, contact: Reed’s Seeds 3334 NYS 215 Cortland, NY 13045 607-753-9095; fax: 607-753-9511 rsjackie@twcny.rr.com Read More

The Latest

VegetablesSeminis Hosts Growers For Its 2014 Southeastern Field D…
December 19, 2014
Seed company showcases new varieties available now and in the pipeline. Read More
CitrusFeed The People With Food And Food For Thought [Opinion…
December 19, 2014
Educating the general public about agriculture is a daunting task that can be accomplished one bite at a time. Read More
CucurbitsNew Product Takes Aim At Nematodes
December 19, 2014
NIMITZ receives EPA and Florida registrations. Read More
Crop ProtectionSuccess Sans Methyl Bromide Could Prove Costly
December 19, 2014
Economic factors must be accounted for when seeking a suitable replacement. Read More
VegetablesEnza Zaden Cuts Ribbon For New Florida Vegetable Resear…
December 18, 2014
Seed company shows off new digs and new varieties at its Myakka City field station. Read More
VegetablesRispens Seeds Highlights Their Latest Vegetable Varieti…
December 18, 2014
For more information, contact: Rispens Seeds 1357 Dutch American Way, P.O. Box 310, Beecher, IL 60401 708-946-6560; 888-874-0241; fax: 708-946-6115 Read More
Protected Agriculture2014 Marks The 50th Anniversary Of The Double-Poly Gree…
December 18, 2014
Double-poly greenhouses revolutionized the greenhouse industry back in the 1960s, opening the way for developments that still impact the industry 50 years later. Read More
CitrusSouthwest Florida Citrus Bus Tour Yields Learning Oppor…
December 18, 2014
Find out what Florida Grower editor Frank Giles saw and heard during Peace River Valley Citrus Growers Association's annual outing. Read More
Fruits6 Food Trends For 2015
December 17, 2014
University of Florida experts predict what will be cooking for the upcoming year. Read More
VegetablesReed’s Seeds Showcases Their Latest Vegetable Var…
December 17, 2014
For more information, contact: Reed’s Seeds 3334 NYS 215 Cortland, NY 13045 607-753-9095; fax: 607-753-9511 rsjackie@twcny.rr.com Read More
CitrusStudy: Florida Citrus Valued At More Than $10 Billion
December 17, 2014
Industry continues to have large impact on the state’s economy. Read More
Apples & PearsNovember Cold May Hurt Washington Orchards
December 17, 2014
But it’s too early to tell if well-below-normal temperatures will increase tree mortality. Read More
NutsAlmond Growers Lose Trees In Storm
December 17, 2014
Last week’s welcome rains in California knocked down trees, especially older ones. Read More
NutsAlmond Conference Draws Record Attendance
December 17, 2014
Nearly 3,000 industry professionals attend 42nd annual event in Sacramento, CA. Read More
Insect & Disease UpdateManage HLB From The Bottom Up
December 17, 2014
Finding the right balance between the roots and shoots is critical to disease mitigation. Read More
VegetablesVariety Trials 2014: Seedway
December 17, 2014
As part of American Vegetable Grower’s (AVG) 2014 variety trial coverage, we visited Seedway’s seed trial at Spiral Path Organic Farm in Loysville, Read More
OrganicUSDA Proposes Expansion Of Organic Assessment Exemption
December 17, 2014
Action would increase the entities exempt from paying assessments for promotion programs on products certified as “organic” or “100% organic.” Read More
MarketingThe U.S. Potato Board Receives $5 Million In Funding To…
December 17, 2014
The funds will be used to help increase consumption of frozen, dehydrated, chip stock, table stock and seed potatoes around the world. Read More