Apple Harvesters Use Vision To “See” Fruit

Editor’s Note: This past October, Tony Koselka, Jillian Cannons, and Ryan Carlon of Vision Robotics Corporation (VRC) returned to Washington for further field trials of the Newton Scout, a vision-based system for automatic crop load assessment. The following report highlights their results.

Developed in part through the USDA-funded project Comprehensive Automation for Specialty Crops (CASC), the Newton Scout will revolutionize orchard operations by providing growers with a map that accurately maps fruit placement, fruit size, and number throughout the growing season. Fruit maps will be used to determine crop load, which in turn will give growers the information to develop harvest and marketing strategies. And this is just the beginning. Developing reliable vision systems that are robust enough for use in orchards opens up all kinds of possibilities in the area of pest and disease detection, plant health assessment, and the use of robotics at harvest.

The Newton Scout integrates stereo cameras with a custom image processing algorithm. The algorithms use classifiers to identify the apples, where the simplest classifier uses color and shape. For red apples, color is the primary classifier. For detecting green fruit, the Scout detects using statistical classifiers generated by boosting algorithms. The most challenging aspects of detecting fruit are occlusions, shadows, shape and color coincidences, and ambient lighting conditions such as bright sunlight. Once an apple is identified, the stereo cameras enable the system to determine its location and size.

The Vision Robotics field trial was conducted at Allan Brothers Othello Orchard in Washington. Prior to VRC’s arrival, Washington State University (WSU) Extension Educator Karen Lewis, and interns with the Washington Tree Fruit Research Commission, hand tagged, mapped, and sized fruit on several rows of fruit. This data allows for the groundtruthing or validation component of the test. Koselka took time out to speak about the development of a similar unit that counts oranges with the interns.

On one of the field trials for that unit, Koselka asked the grower to estimate his crop. The farm manager responded with 1,200 bushels. For this test, Vision Robotics scouted and counted 1,900 — a sizable difference and one that would have a large financial impact.

As Koselka stated, “not only will it [Scout] be able to accurately count a crop and locate each piece of fruit, it will ultimately feed into a robot harvester that will pick all the apples.” And if that was not enough, Koselka added, “Oh, and a pruner is not far off either!”

Even More Potential

The Newton Scout utility will be enhanced in the future with the addition of other possible technologies, including near infrared (NIR). As Newton evolves, it will have multiple uses throughout the fruit growing cycle — for example, assessment of sugar and acid content and internal damage. CASC is also making progress in other automation fronts that bridge with Vision Robotics’ work.

Carnegie Mellon University (CMU) researchers, led by Sanjiv Singh, are developing an Autonomous Prime Mover (APM) capable of towing the Newton Scout safely through entire orchard blocks. In the future, they will also automate an agricultural harvesting platform so that workers no longer need to climb on ladders to pick fruit. The APM’s multifunctionality, autonomous drive, and wireless networking could move it into the center of precision management of specialty crops.

The CASC team, led by CMU, includes engineers, scientists, Extension educators, growers, equipment manufacturers, and industry representatives. This plurality of expertise has led to synergistic endeavors that push the envelope in both the engineering and the science components of the project. For example, Ben Grocholsky at CMU is developing a geospatial system to map and locate each tree in an orchard; in cooperation with the plant science group, he was able to augment this information with local temperature and NDVI data that has the future possibility of being incorporated to detect plant stress and disease.

In another example of the synergy of the CASC team, during a field trial of an automated caliper measurement device, growers told the CASC team that a simpler tree counter would also be very useful. Singh’s group returned to CMU and produced a prototype tree counter ready to be tested in weeks; they will return to nurseries and test the counter this fall. The total time from conception to development will be less than four months, and if successful, this device could go into production within a year.

It will only be a matter of time before specialty crop growers also start using vehicles, sensors, devices, and software that automate farm operations, increase farm efficiency, and meet the ultimate goal of delivering to the consumer a consistently high quality affordable eating experience while returning money to the farming operation.

For more information on Comprehensive Automation for Specialty Crops, please visit www.cascrop.com.

Leave a Reply

Featured Stories
Apple Grower of the YearBill Dodd Is An Apple Grower For Our Future
August 3, 2015
The judging for the Apple Grower Of The Year award is an interesting process that takes us through new twists Read More
GenNext GrowersPotato Industry Leadership Institute Program To Be Held In February
August 3, 2015
The application and scholarship deadline is Oct. 15, 2015. Read More
Vegetables13 Of The Latest Pepper Varieties
August 3, 2015
This month’s variety specs feature focuses on pepper varieties. Features highlighted include great disease resistance packages, excellent yield potential, and more. Be Read More
Apple Grower of the YearBill Dodd, The 2015 Apple Grower Of The Year
August 3, 2015
  Informed of his selection as the 2015 Apple Grower Of The Year by American Fruit Grower® and Western Fruit Grower™ Read More
Farm ManagementLocally Grown The Lipman Way
August 3, 2015
With Lipman Local, the Florida operation has developed a mutually beneficial program that sources local produce around the country and helps smaller growers raise their game. Read More
Farm ManagementCorrect Misconceptions About Farming [Opinion]
August 3, 2015
Misinformation about agriculture abounds. We need to put a stop to it and set the record straight. Read More
UFR-2 Vernia citrus rootstock
Citrus Achievement AwardLessons Learned Via Citrus Variety Evaluation Never End
August 1, 2015
Orie Lee, the 2015 Citrus Achievement Award winner, discusses his decades-long involvement with Vernia. Read More
Florida Ag ExpoResearchers To Put 3 Major Crop Threats On Trial
August 1, 2015
Nematode, disease, and weed management on deck for the 10th annual Florida Ag Expo. Read More
woman smashing an alarm clock with a hammer
CitrusDon’t Become A Time Crunch Casualty
July 31, 2015
Finding the balance between work life and home life is easier said than done for GenNext Growers. Read More
PotatoesLate Blight Confirmed In Areas Of Idaho; Number Of Potato Psyllids Dips
July 31, 2015
The number of potato psyllids decline while additional fields are noted as having late blight. Read More
The Latest
Apple Grower of the YearBill Dodd Is An Apple Grower For Our Future
August 3, 2015
The judging for the Apple Grower Of The Year award is an interesting process that takes us through new twists Read More
GenNext GrowersPotato Industry Leadership Institute Program To Be Held…
August 3, 2015
The application and scholarship deadline is Oct. 15, 2015. Read More
Vegetables13 Of The Latest Pepper Varieties
August 3, 2015
This month’s variety specs feature focuses on pepper varieties. Features highlighted include great disease resistance packages, excellent yield potential, and more. Be Read More
Apple Grower of the YearBill Dodd, The 2015 Apple Grower Of The Year
August 3, 2015
  Informed of his selection as the 2015 Apple Grower Of The Year by American Fruit Grower® and Western Fruit Grower™ Read More
Farm ManagementLocally Grown The Lipman Way
August 3, 2015
With Lipman Local, the Florida operation has developed a mutually beneficial program that sources local produce around the country and helps smaller growers raise their game. Read More
Farm ManagementCorrect Misconceptions About Farming [Opinion]
August 3, 2015
Misinformation about agriculture abounds. We need to put a stop to it and set the record straight. Read More
UFR-2 Vernia citrus rootstock
Citrus Achievement AwardLessons Learned Via Citrus Variety Evaluation Never End
August 1, 2015
Orie Lee, the 2015 Citrus Achievement Award winner, discusses his decades-long involvement with Vernia. Read More
Florida Ag ExpoResearchers To Put 3 Major Crop Threats On Trial
August 1, 2015
Nematode, disease, and weed management on deck for the 10th annual Florida Ag Expo. Read More
woman smashing an alarm clock with a hammer
CitrusDon’t Become A Time Crunch Casualty
July 31, 2015
Finding the balance between work life and home life is easier said than done for GenNext Growers. Read More
PotatoesLate Blight Confirmed In Areas Of Idaho; Number Of Pota…
July 31, 2015
The number of potato psyllids decline while additional fields are noted as having late blight. Read More
Apples & PearsReTain Now Registered For Double Application
July 31, 2015
ReTain plant growth regulator (PGR) has been registered by EPA with a new label use that allows pome fruit growers Read More
Crop ProtectionLettuce Disease Gives E. Coli A Boost
July 31, 2015
E. coli prefer cut, injured, and young lettuce leaves. Read More
Farm ManagementAn Excellent Industry Education [Opinion]
July 31, 2015
The end of one era, and the ushering in of another. Read More
Stone FruitThe Strong Staying Power Of The Redhaven Name In Peache…
July 31, 2015
People tend to be nostalgic about peach varieties, but the switch of varieties grown by a region happens for a Read More
Crop ProtectionResources For Knowing Your Soil
July 30, 2015
Getting to know your soil and learning other environmental and natural resources data can be downloaded and is available for use with many computer software platforms for general resource analysis and farm management. Read More
money
CitrusBankrupt Florida Farm To Yield Plenty Of Land, Equipmen…
July 30, 2015
As many as seven different buyers could be part of this Chapter 11 proceeding. Read More
Sanjay Shukla
ProductionGeometrically Speaking, Thinking Smaller Might Produce …
July 30, 2015
University of Florida scientist develops compact planting bed formula that cuts water, fertilizer, and pesticide use in half. Read More
GrapesDrought’s Impact On California Winegrapes
July 30, 2015
The impact is minimal this year, but the future is uncertain if the drought continues. Read More