Apple Harvesters Use Vision To “See” Fruit

Editor’s Note: This past October, Tony Koselka, Jillian Cannons, and Ryan Carlon of Vision Robotics Corporation (VRC) returned to Washington for further field trials of the Newton Scout, a vision-based system for automatic crop load assessment. The following report highlights their results.

Developed in part through the USDA-funded project Comprehensive Automation for Specialty Crops (CASC), the Newton Scout will revolutionize orchard operations by providing growers with a map that accurately maps fruit placement, fruit size, and number throughout the growing season. Fruit maps will be used to determine crop load, which in turn will give growers the information to develop harvest and marketing strategies. And this is just the beginning. Developing reliable vision systems that are robust enough for use in orchards opens up all kinds of possibilities in the area of pest and disease detection, plant health assessment, and the use of robotics at harvest.

The Newton Scout integrates stereo cameras with a custom image processing algorithm. The algorithms use classifiers to identify the apples, where the simplest classifier uses color and shape. For red apples, color is the primary classifier. For detecting green fruit, the Scout detects using statistical classifiers generated by boosting algorithms. The most challenging aspects of detecting fruit are occlusions, shadows, shape and color coincidences, and ambient lighting conditions such as bright sunlight. Once an apple is identified, the stereo cameras enable the system to determine its location and size.

The Vision Robotics field trial was conducted at Allan Brothers Othello Orchard in Washington. Prior to VRC’s arrival, Washington State University (WSU) Extension Educator Karen Lewis, and interns with the Washington Tree Fruit Research Commission, hand tagged, mapped, and sized fruit on several rows of fruit. This data allows for the groundtruthing or validation component of the test. Koselka took time out to speak about the development of a similar unit that counts oranges with the interns.

On one of the field trials for that unit, Koselka asked the grower to estimate his crop. The farm manager responded with 1,200 bushels. For this test, Vision Robotics scouted and counted 1,900 — a sizable difference and one that would have a large financial impact.

As Koselka stated, “not only will it [Scout] be able to accurately count a crop and locate each piece of fruit, it will ultimately feed into a robot harvester that will pick all the apples.” And if that was not enough, Koselka added, “Oh, and a pruner is not far off either!”

Even More Potential

The Newton Scout utility will be enhanced in the future with the addition of other possible technologies, including near infrared (NIR). As Newton evolves, it will have multiple uses throughout the fruit growing cycle — for example, assessment of sugar and acid content and internal damage. CASC is also making progress in other automation fronts that bridge with Vision Robotics’ work.

Carnegie Mellon University (CMU) researchers, led by Sanjiv Singh, are developing an Autonomous Prime Mover (APM) capable of towing the Newton Scout safely through entire orchard blocks. In the future, they will also automate an agricultural harvesting platform so that workers no longer need to climb on ladders to pick fruit. The APM’s multifunctionality, autonomous drive, and wireless networking could move it into the center of precision management of specialty crops.

The CASC team, led by CMU, includes engineers, scientists, Extension educators, growers, equipment manufacturers, and industry representatives. This plurality of expertise has led to synergistic endeavors that push the envelope in both the engineering and the science components of the project. For example, Ben Grocholsky at CMU is developing a geospatial system to map and locate each tree in an orchard; in cooperation with the plant science group, he was able to augment this information with local temperature and NDVI data that has the future possibility of being incorporated to detect plant stress and disease.

In another example of the synergy of the CASC team, during a field trial of an automated caliper measurement device, growers told the CASC team that a simpler tree counter would also be very useful. Singh’s group returned to CMU and produced a prototype tree counter ready to be tested in weeks; they will return to nurseries and test the counter this fall. The total time from conception to development will be less than four months, and if successful, this device could go into production within a year.

It will only be a matter of time before specialty crop growers also start using vehicles, sensors, devices, and software that automate farm operations, increase farm efficiency, and meet the ultimate goal of delivering to the consumer a consistently high quality affordable eating experience while returning money to the farming operation.

For more information on Comprehensive Automation for Specialty Crops, please visit www.cascrop.com.

Leave a Reply

Featured Stories
Marco Rubio tours a Central Florida citrus grove with industry dignitaries
Insect & Disease Update
July 22, 2016
Marco Rubio Backs Bill To Save Florida Citrus
Senator co-sponsors measure that would provide growers with incentives to plant more trees. Read More
Holly Rippon-Butler
GenNext Growers
July 22, 2016
Farming’s Next Generation Seeking Room To Grow
Several innovative resources are available to help first-time growers acquire the property they need. Read More
Live music woman playing guitar free image FEATURE
Farm Marketing
July 22, 2016
How To Book A Great Band For Your Festivals
Want to add live music to your fall festivals? Here's a step-by-step guide on how to find the right band for the right price. Read More
Valery Cordrey on a radio show
Farm Marketing
July 22, 2016
How To Entice The Local Press To Cover Your Farm Market
Sometimes getting publicity for your business is easier than you think. All you need to do is figure out how Read More
cantaloupe
Cucurbits
July 22, 2016
California Cantaloupe Growers To Celebrate First-Ever “Cantaloupe Week”
The Golden State’s cantaloupes are now in peak production from major areas in the San Joaquin Valley. Read More
Closeup of buckwheat blooms
Cucurbits
July 21, 2016
Study Shows Buckwheat Can Help Stifle Squash Pests
Unique cover crop choice attracts beneficial bugs, increasing pollination prospects and reducing insect threats. Read More
farm labor pic for web
Citrus
July 21, 2016
Farm Labor Delays Taking Toll On Crops [Opinion]
When it comes down to it, we have a choice to make. We either have to import our labor, or we'll have to import our food. Read More
Urban vegetable farm in rural Cuba
Citrus
July 20, 2016
My First Trip To Cuba Proves Fruitful
The five-day journey was enlightening, providing participants with a better perspective of the state of agriculture in this country 90 miles off Florida’s coast. Read More
Shinsuke Agehara of UF/IFAS holding an artichoke in the middle of a Central Florida farm plot
More Vegetables
July 19, 2016
So You Think You Can Grow Artichokes In Florida?
Field trials are in progress to determine if this highly sought-after superfood can survive the swelter of the Sunshine State. Read More
The Latest
Insect & Disease Update
July 22, 2016
Marco Rubio Backs Bill To Save Florida C…
Senator co-sponsors measure that would provide growers with incentives to plant more trees. Read More
GenNext Growers
July 22, 2016
Farming’s Next Generation Seeking …
Several innovative resources are available to help first-time growers acquire the property they need. Read More
Farm Marketing
July 22, 2016
How To Book A Great Band For Your Festiv…
Want to add live music to your fall festivals? Here's a step-by-step guide on how to find the right band for the right price. Read More
Farm Marketing
July 22, 2016
How To Entice The Local Press To Cover Y…
Sometimes getting publicity for your business is easier than you think. All you need to do is figure out how Read More
Cucurbits
July 22, 2016
California Cantaloupe Growers To Celebra…
The Golden State’s cantaloupes are now in peak production from major areas in the San Joaquin Valley. Read More
Farm Marketing
July 22, 2016
Find Out Where Growers Want To Sell Thei…
Earlier this year, we asked farm marketers which of their current business channels they would like to increase. Not surprisingly, Read More
Cucurbits
July 21, 2016
Study Shows Buckwheat Can Help Stifle Sq…
Unique cover crop choice attracts beneficial bugs, increasing pollination prospects and reducing insect threats. Read More
Citrus
July 21, 2016
Farm Labor Delays Taking Toll On Crops […
When it comes down to it, we have a choice to make. We either have to import our labor, or we'll have to import our food. Read More
Citrus
July 20, 2016
My First Trip To Cuba Proves Fruitful
The five-day journey was enlightening, providing participants with a better perspective of the state of agriculture in this country 90 miles off Florida’s coast. Read More
More Vegetables
July 19, 2016
So You Think You Can Grow Artichokes In …
Field trials are in progress to determine if this highly sought-after superfood can survive the swelter of the Sunshine State. Read More
Citrus
July 19, 2016
Funds Allocated To Address Impact Of Cli…
NIFA announces $8.4 for study and development of new approaches to mitigate effects of climate change. Read More
Disease Control
July 19, 2016
Open Your Eyes To Common Smut In Sweet C…
Learn how to identify, the survival and spread, as well as management methods for this formidable fungus. Read More
Crop Protection
July 18, 2016
Early Infection Of Downy Mildew Confirme…
The pathogen has been confirmed in Michigan cucumbers in Bay County for the 2016 season. Read More
Citrus
July 18, 2016
USDA Awards More Than $26 Million For Af…
The funding is to support more than 400 rental units in four states. Read More
Crop Protection
July 18, 2016
Potato Growers: Be On The Lookout For Ne…
Standard measures have been typically less effective against new strains. Read More
Business Planning
July 18, 2016
Financial Fortitude Key For Small Farm S…
Local threats like citrus greening create additional challenges for many of Florida's more modest enterprises. Read More
Citrus
July 18, 2016
Bayer Ups Bid To Buy Monsanto
Reports indicate new offer at nearly $65 billion. Read More
Cucurbits
July 18, 2016
21 Pumpkin Varieties You Need To Know
Leading seed breeders and distributors highlight some of the latest pumpkin varieties available. Read More
[gravityform id="62" title="false" description="false"]