Apple Harvesters Use Vision To “See” Fruit

Editor’s Note: This past October, Tony Koselka, Jillian Cannons, and Ryan Carlon of Vision Robotics Corporation (VRC) returned to Washington for further field trials of the Newton Scout, a vision-based system for automatic crop load assessment. The following report highlights their results.

Developed in part through the USDA-funded project Comprehensive Automation for Specialty Crops (CASC), the Newton Scout will revolutionize orchard operations by providing growers with a map that accurately maps fruit placement, fruit size, and number throughout the growing season. Fruit maps will be used to determine crop load, which in turn will give growers the information to develop harvest and marketing strategies. And this is just the beginning. Developing reliable vision systems that are robust enough for use in orchards opens up all kinds of possibilities in the area of pest and disease detection, plant health assessment, and the use of robotics at harvest.

The Newton Scout integrates stereo cameras with a custom image processing algorithm. The algorithms use classifiers to identify the apples, where the simplest classifier uses color and shape. For red apples, color is the primary classifier. For detecting green fruit, the Scout detects using statistical classifiers generated by boosting algorithms. The most challenging aspects of detecting fruit are occlusions, shadows, shape and color coincidences, and ambient lighting conditions such as bright sunlight. Once an apple is identified, the stereo cameras enable the system to determine its location and size.

The Vision Robotics field trial was conducted at Allan Brothers Othello Orchard in Washington. Prior to VRC’s arrival, Washington State University (WSU) Extension Educator Karen Lewis, and interns with the Washington Tree Fruit Research Commission, hand tagged, mapped, and sized fruit on several rows of fruit. This data allows for the groundtruthing or validation component of the test. Koselka took time out to speak about the development of a similar unit that counts oranges with the interns.

On one of the field trials for that unit, Koselka asked the grower to estimate his crop. The farm manager responded with 1,200 bushels. For this test, Vision Robotics scouted and counted 1,900 — a sizable difference and one that would have a large financial impact.

As Koselka stated, “not only will it [Scout] be able to accurately count a crop and locate each piece of fruit, it will ultimately feed into a robot harvester that will pick all the apples.” And if that was not enough, Koselka added, “Oh, and a pruner is not far off either!”

Even More Potential

The Newton Scout utility will be enhanced in the future with the addition of other possible technologies, including near infrared (NIR). As Newton evolves, it will have multiple uses throughout the fruit growing cycle — for example, assessment of sugar and acid content and internal damage. CASC is also making progress in other automation fronts that bridge with Vision Robotics’ work.

Carnegie Mellon University (CMU) researchers, led by Sanjiv Singh, are developing an Autonomous Prime Mover (APM) capable of towing the Newton Scout safely through entire orchard blocks. In the future, they will also automate an agricultural harvesting platform so that workers no longer need to climb on ladders to pick fruit. The APM’s multifunctionality, autonomous drive, and wireless networking could move it into the center of precision management of specialty crops.

The CASC team, led by CMU, includes engineers, scientists, Extension educators, growers, equipment manufacturers, and industry representatives. This plurality of expertise has led to synergistic endeavors that push the envelope in both the engineering and the science components of the project. For example, Ben Grocholsky at CMU is developing a geospatial system to map and locate each tree in an orchard; in cooperation with the plant science group, he was able to augment this information with local temperature and NDVI data that has the future possibility of being incorporated to detect plant stress and disease.

In another example of the synergy of the CASC team, during a field trial of an automated caliper measurement device, growers told the CASC team that a simpler tree counter would also be very useful. Singh’s group returned to CMU and produced a prototype tree counter ready to be tested in weeks; they will return to nurseries and test the counter this fall. The total time from conception to development will be less than four months, and if successful, this device could go into production within a year.

It will only be a matter of time before specialty crop growers also start using vehicles, sensors, devices, and software that automate farm operations, increase farm efficiency, and meet the ultimate goal of delivering to the consumer a consistently high quality affordable eating experience while returning money to the farming operation.

For more information on Comprehensive Automation for Specialty Crops, please visit www.cascrop.com.

Leave a Reply

Featured Stories
Crop ProtectionAt Paramount Farming, Mating Disruption Pays Off
February 10, 2015
To say Brad Higbee is an expert in mating disruption, the use of pheromones to confuse target pests and reduce Read More
Crop ProtectionGrower Experienced In Biocontrols Says: Be Patient
February 4, 2015
The use of biocontrols in all aspects of agriculture is becoming more widespread. But incorporating biocontrols in a vegetable farming Read More
Sweet CornAbbott & Cobb Sweet On New Sweet Corn Technology
March 3, 2015
SuperSeedWare gene type to aid varieties through marker-assisted selection. Read More
FruitsWest Coast Ports To Open With New Contract
March 3, 2015
Western fruit, vegetable industries lost millions when unable to export produce at ports. Read More
FruitsWorkshop Set In Florida To Help Grow Local Food Movement
March 2, 2015
Meeting part of nationwide effort to train people how to attain grants for USDA’s Farmers Market and Local Food Promotion programs. Read More
Citrus18 Must-Have Apps For GenNext Growers
March 1, 2015
We asked about your favorite mobile applications, and here's what you had to say. Read More
Florida Ag ExpoGrowers Get The Lowdown On Cooling Crops
March 1, 2015
Demonstration shows how hydrocooling can help get the most out of fruits and vegetables once harvested. Read More
FruitsDon’t Be Afraid To Charge More For Value Of Agritainment [Opinion]
February 28, 2015
A higher admission price helps to sets a level of expectation and helps offset the costs of running a successful farm marketing operation. Read More
Apples & PearsCompany Responsible For First GMO Apples Is Sold
February 27, 2015
Okanagan Specialty Fruits, the company behind the world's first non-browning apples, fetches $41 million. Read More
Stone FruitPeach Training Systems Aren’t One-Size-Fits-All
February 27, 2015
Although pedestrian, ladderless orchards are increasing in California, growers in Pennsylvania are opting for tall V-training systems. Read More
IrrigationNo Federal Water For California’s Central Valley Growers – Again
February 27, 2015
Skimpy snowpack leads Bureau of Reclamation to make unprecedented announcement that there will be a zero initial allocation for the second year in a row. Read More
Apples & PearsThe Recession That Keeps On Taking
February 27, 2015
We’re still seeing the aftermath of the Great Recession on consumers’ purchasing habits seven years later. Read More
The Latest
Sweet CornAbbott & Cobb Sweet On New Sweet Corn Technology
March 3, 2015
SuperSeedWare gene type to aid varieties through marker-assisted selection. Read More
FruitsWest Coast Ports To Open With New Contract
March 3, 2015
Western fruit, vegetable industries lost millions when unable to export produce at ports. Read More
FruitsWorkshop Set In Florida To Help Grow Local Food Movemen…
March 2, 2015
Meeting part of nationwide effort to train people how to attain grants for USDA’s Farmers Market and Local Food Promotion programs. Read More
Citrus18 Must-Have Apps For GenNext Growers
March 1, 2015
We asked about your favorite mobile applications, and here's what you had to say. Read More
Florida Ag ExpoGrowers Get The Lowdown On Cooling Crops
March 1, 2015
Demonstration shows how hydrocooling can help get the most out of fruits and vegetables once harvested. Read More
FruitsDon’t Be Afraid To Charge More For Value Of Agrit…
February 28, 2015
A higher admission price helps to sets a level of expectation and helps offset the costs of running a successful farm marketing operation. Read More
Apples & PearsCompany Responsible For First GMO Apples Is Sold
February 27, 2015
Okanagan Specialty Fruits, the company behind the world's first non-browning apples, fetches $41 million. Read More
Stone FruitPeach Training Systems Aren’t One-Size-Fits-All
February 27, 2015
Although pedestrian, ladderless orchards are increasing in California, growers in Pennsylvania are opting for tall V-training systems. Read More
IrrigationNo Federal Water For California’s Central Valley Grower…
February 27, 2015
Skimpy snowpack leads Bureau of Reclamation to make unprecedented announcement that there will be a zero initial allocation for the second year in a row. Read More
Apples & PearsThe Recession That Keeps On Taking
February 27, 2015
We’re still seeing the aftermath of the Great Recession on consumers’ purchasing habits seven years later. Read More
Grower Achievement AwardSubmit Your Nomination For The 2015 Grower Achievement …
February 27, 2015
American Vegetable Grower magazine's annual award is designed to recognize an innovative operation that represents the spirit of excellence. Read More
FruitsMaximize Produce Profits By Focusing On Soil Health
February 27, 2015
Cover crops are just one of the ways you can help boost your trees’ and vines’ performance and your bottom line. Read More
FNV logo
CitrusPMA Announces Funding For Major Produce Marketing Progr…
February 26, 2015
The produce industry is getting into brand marketing in a big way. The Produce Marketing Association (PMA) has announced the launch of a new promotional campaign - FNV - designed to take a page from the marketing strategies of big-money consumer brands like Nike and Apple. Read More
GrapesWashington Winegrape Industry Announces Honors
February 26, 2015
Awards included Grower of the Year, Industry Service, Lifetime Achievement, Restaurant of the Year and Posters. Read More
GrapesSonoma Agriculture Preservation Plan Unveiled
February 26, 2015
Winegrowers’ 100-year business plan is the first of its kind in agriculture. Read More
Apples & PearsNew Plant Growth Regulator Use Expanded
February 26, 2015
Ten more states grant registrations for Fine Americas’ new Kudos PGR. Read More
Apples & PearsFrom The Road: IFTA Conference Day 5
February 26, 2015
Carlson lecture highlights where IFTA has been and looks to the next generation to continue the legacy. Read More
FruitsHave Faith In Science – Which Means GMOs [Opinion…
February 25, 2015
When it comes to farming, sticking with established, commonly accepted science is the way to go. Read More