Apple Harvesters Use Vision To “See” Fruit

Editor’s Note: This past October, Tony Koselka, Jillian Cannons, and Ryan Carlon of Vision Robotics Corporation (VRC) returned to Washington for further field trials of the Newton Scout, a vision-based system for automatic crop load assessment. The following report highlights their results.

Developed in part through the USDA-funded project Comprehensive Automation for Specialty Crops (CASC), the Newton Scout will revolutionize orchard operations by providing growers with a map that accurately maps fruit placement, fruit size, and number throughout the growing season. Fruit maps will be used to determine crop load, which in turn will give growers the information to develop harvest and marketing strategies. And this is just the beginning. Developing reliable vision systems that are robust enough for use in orchards opens up all kinds of possibilities in the area of pest and disease detection, plant health assessment, and the use of robotics at harvest.

The Newton Scout integrates stereo cameras with a custom image processing algorithm. The algorithms use classifiers to identify the apples, where the simplest classifier uses color and shape. For red apples, color is the primary classifier. For detecting green fruit, the Scout detects using statistical classifiers generated by boosting algorithms. The most challenging aspects of detecting fruit are occlusions, shadows, shape and color coincidences, and ambient lighting conditions such as bright sunlight. Once an apple is identified, the stereo cameras enable the system to determine its location and size.

The Vision Robotics field trial was conducted at Allan Brothers Othello Orchard in Washington. Prior to VRC’s arrival, Washington State University (WSU) Extension Educator Karen Lewis, and interns with the Washington Tree Fruit Research Commission, hand tagged, mapped, and sized fruit on several rows of fruit. This data allows for the groundtruthing or validation component of the test. Koselka took time out to speak about the development of a similar unit that counts oranges with the interns.

On one of the field trials for that unit, Koselka asked the grower to estimate his crop. The farm manager responded with 1,200 bushels. For this test, Vision Robotics scouted and counted 1,900 — a sizable difference and one that would have a large financial impact.

As Koselka stated, “not only will it [Scout] be able to accurately count a crop and locate each piece of fruit, it will ultimately feed into a robot harvester that will pick all the apples.” And if that was not enough, Koselka added, “Oh, and a pruner is not far off either!”

Even More Potential

The Newton Scout utility will be enhanced in the future with the addition of other possible technologies, including near infrared (NIR). As Newton evolves, it will have multiple uses throughout the fruit growing cycle — for example, assessment of sugar and acid content and internal damage. CASC is also making progress in other automation fronts that bridge with Vision Robotics’ work.

Carnegie Mellon University (CMU) researchers, led by Sanjiv Singh, are developing an Autonomous Prime Mover (APM) capable of towing the Newton Scout safely through entire orchard blocks. In the future, they will also automate an agricultural harvesting platform so that workers no longer need to climb on ladders to pick fruit. The APM’s multifunctionality, autonomous drive, and wireless networking could move it into the center of precision management of specialty crops.

The CASC team, led by CMU, includes engineers, scientists, Extension educators, growers, equipment manufacturers, and industry representatives. This plurality of expertise has led to synergistic endeavors that push the envelope in both the engineering and the science components of the project. For example, Ben Grocholsky at CMU is developing a geospatial system to map and locate each tree in an orchard; in cooperation with the plant science group, he was able to augment this information with local temperature and NDVI data that has the future possibility of being incorporated to detect plant stress and disease.

In another example of the synergy of the CASC team, during a field trial of an automated caliper measurement device, growers told the CASC team that a simpler tree counter would also be very useful. Singh’s group returned to CMU and produced a prototype tree counter ready to be tested in weeks; they will return to nurseries and test the counter this fall. The total time from conception to development will be less than four months, and if successful, this device could go into production within a year.

It will only be a matter of time before specialty crop growers also start using vehicles, sensors, devices, and software that automate farm operations, increase farm efficiency, and meet the ultimate goal of delivering to the consumer a consistently high quality affordable eating experience while returning money to the farming operation.

For more information on Comprehensive Automation for Specialty Crops, please visit www.cascrop.com.

Leave a Reply

Featured Stories
broccoli_eastern-magic
Vegetables
December 10, 2016
Sakata Seed America 2016 Vegetable Variety Showcase
For more information, contact Sakata Seed America: 18095 Serene Dr., Morgan Hill, CA 408-778-7758; WEBSITE SakataVegetables.com FAX 408-778-7768; EMAIL [email protected] Read More
A new Italian cross of Gala and Elstar, Red Rubens® combines intense apple flavor with a pleasing sweet-tartness. The fruit is crisp, juicy and conical in shape. A favorite in Europe, it now is available in the U.S. by exclusive license with North American INN, LLC.  This is one of Van Well Nursery’s “Faces of the Future.” USPP#14,177
Fruits
December 10, 2016
Van Well Nursery 2017 Variety Showcase [Slideshow]
Van Well Nursery in Wenatchee, WA, presents a new Cripps Pink sport, new scab-free cultivar, new Italian cross of Gala and Elstar, and more. Read More
Photo by Frank Giles
Citrus
December 9, 2016
Latest Florida Citrus Crop Forecast Runs Flat
Despite crop protection challenges, all-orange production projection from USDA remains unchanged. Read More
Buckeye® Prime is a beautiful mutation of the original Buckeye® Gala with a brighter cherry-red color and much heavier striping. Buckeye® Prime retains the original Buckeye ability to color in hot climates. Buckeye® Prime is virtually 100% red with a bold, darker-red stripe and the same ability as its parent to produce high-quality, beautiful apples. USPP 24,639
Fruits
December 9, 2016
Summit Tree Sales 2017 Variety Showcase [Slideshow]
Summit Tree Sales in Lawrence, MI, presents an early apple for the fresh market, early-to-mid-season cherries, a sweet freestone peach, and more. Read More
Knox Medical medicinal cannabis product display
Citrus
December 8, 2016
A Look Inside Florida’s Newest Cannabis Cultivation Facility
Given the official go-ahead to dispense alternative medicine, Knox Medical unveils its state-of-the-art lab. Read More
Scarlet Crush™
Our new Honeycrisp/Pink Lady® cross is sweet with citrusy notes, plus all that famous Honeycrisp snap and juiciness. Up to to 3” pink/rosy-red fruit is uniform, conical and very attractive. A fine cooking and fresh-eating apple. Keeps up to 6 months. Ripens as early as late September. USPPAF. Best pollinators: Starkspur® Red Rome Beauty, Stark® Jon-A-Red® Jonathan or any Golden Delicious. Zones: 3-6.
Fruits
December 8, 2016
Stark Bro’s Nurseries & Orchards Co. 2017 Variety Showcase [Slideshow]
For more information on these varieties, contact Stark Bro’s Nurseries & Orchards. Stark Bro’s Nurseries & Orchards P.O. Box 398 Read More
Past industry efforts to discover Paradox hybrid seedlings (Juglans hindsii x J. regia) with superior traits have led to advanced selections of Paradox clones, each with unique characteristics. The University of California selection, RX 1, is a top-choice for Sacramento Valley walnut orchardists and others looking for resistance to Phytopthora and good survivability. RX 1 is a cross of Juglans microcarpa x J. regia. It is showing to be one of the most adaptable walnut rootstocks, producing a large tree with the highest yields in replicated trials. USPP#20,649 P3
Fruits
December 7, 2016
Sierra Gold Nursery 2017 Variety Showcase [Slideshow]
Sierra Gold Nurseries in Yuba City, CA, offers two walnut rootstocks, a salt-tolerant pistachio rootstock, and more. Read More
Action shot of an automated strawberry picker from Harvest CROO
Equipment
December 6, 2016
USDA Announces $3 Million For Robotic Agriculture Research
More evidence being found about how people can benefit from automation in the field. Read More
billevans
Apple Grower of the Year
December 6, 2016
2014 Apple Grower Of The Year Bill Evans Dies
Legendary apple grower Bill Evans, who along with his wife, Jeannette, farmed more apple acreage than any grower in the Read More
weedy watermelon field
Crop Protection
December 6, 2016
New Fact Sheet Analyzes Weed Seeds And Their Longevity
Topics covered include how weed seeds travel, when they germinate, and ways they can be eliminated. Read More
The Latest
Vegetables
December 10, 2016
Sakata Seed America 2016 Vegetable Varie…
For more information, contact Sakata Seed America: 18095 Serene Dr., Morgan Hill, CA 408-778-7758; WEBSITE SakataVegetables.com FAX 408-778-7768; EMAIL [email protected] Read More
Fruits
December 10, 2016
Van Well Nursery 2017 Variety Showcase […
Van Well Nursery in Wenatchee, WA, presents a new Cripps Pink sport, new scab-free cultivar, new Italian cross of Gala and Elstar, and more. Read More
Citrus
December 9, 2016
Latest Florida Citrus Crop Forecast Runs…
Despite crop protection challenges, all-orange production projection from USDA remains unchanged. Read More
Fruits
December 9, 2016
Summit Tree Sales 2017 Variety Showcase …
Summit Tree Sales in Lawrence, MI, presents an early apple for the fresh market, early-to-mid-season cherries, a sweet freestone peach, and more. Read More
Citrus
December 8, 2016
A Look Inside Florida’s Newest Cannabis …
Given the official go-ahead to dispense alternative medicine, Knox Medical unveils its state-of-the-art lab. Read More
Fruits
December 8, 2016
Stark Bro’s Nurseries & Orchards Co.…
For more information on these varieties, contact Stark Bro’s Nurseries & Orchards. Stark Bro’s Nurseries & Orchards P.O. Box 398 Read More
Fruits
December 7, 2016
Sierra Gold Nursery 2017 Variety Showcas…
Sierra Gold Nurseries in Yuba City, CA, offers two walnut rootstocks, a salt-tolerant pistachio rootstock, and more. Read More
Equipment
December 6, 2016
USDA Announces $3 Million For Robotic Ag…
More evidence being found about how people can benefit from automation in the field. Read More
Apple Grower of the Year
December 6, 2016
2014 Apple Grower Of The Year Bill Evans…
Legendary apple grower Bill Evans, who along with his wife, Jeannette, farmed more apple acreage than any grower in the Read More
Crop Protection
December 6, 2016
New Fact Sheet Analyzes Weed Seeds And T…
Topics covered include how weed seeds travel, when they germinate, and ways they can be eliminated. Read More
Crop Protection
December 6, 2016
Scientists Aim To Improve Nitrogen, Wate…
Genetic research intends to develop varieties with enhanced nutrient uptake. Read More
Citrus
December 6, 2016
Report: Florida Cannabis Business To App…
Lofty projections include a compound annual growth rate of 140%. Read More
Citrus
December 6, 2016
Florida Citrus Community Tasked To Unify…
A group effort is necessary to forge ahead. Read More
Crop Protection
December 6, 2016
Orondis Opti Fungicide Receives Registra…
The product provides control of Oomycete diseases in vegetables and other specialty crops. Read More
Fruits
December 6, 2016
Fowler Nurseries, Inc. 2017 Variety Show…
For more information on these varieties, contact Fowler Nurseries, Inc. Fowler Nurseries, Inc. 525 Fowler Rd. Newcastle, CA 9565827 Phone: Read More
Vegetables
December 6, 2016
Rupp Seeds 2016 Vegetable Variety Showca…
For more information, contact, Rupp Seeds: 17919 County Rd. B, Wauseon, OH 43567 800-700-1199; WEBSITE www.RuppSeeds.com FAX 419-337-5491; EMAIL [email protected] Read More
Farm Marketing
December 6, 2016
Registration Is Open For 32nd Annual Nor…
The six-day event for agritourism operators heads to the Northeast with highlights including bus tours, educational sessions and workshops, and business exchange. Read More
Insect & Disease Update
December 5, 2016
Best Practices For Bactericides Key To V…
Approval finally cleared for antimicrobial applications, and now growers wait and hope to see positive results. Read More