Apple Harvesters Use Vision To “See” Fruit

Editor’s Note: This past October, Tony Koselka, Jillian Cannons, and Ryan Carlon of Vision Robotics Corporation (VRC) returned to Washington for further field trials of the Newton Scout, a vision-based system for automatic crop load assessment. The following report highlights their results.

Developed in part through the USDA-funded project Comprehensive Automation for Specialty Crops (CASC), the Newton Scout will revolutionize orchard operations by providing growers with a map that accurately maps fruit placement, fruit size, and number throughout the growing season. Fruit maps will be used to determine crop load, which in turn will give growers the information to develop harvest and marketing strategies. And this is just the beginning. Developing reliable vision systems that are robust enough for use in orchards opens up all kinds of possibilities in the area of pest and disease detection, plant health assessment, and the use of robotics at harvest.

The Newton Scout integrates stereo cameras with a custom image processing algorithm. The algorithms use classifiers to identify the apples, where the simplest classifier uses color and shape. For red apples, color is the primary classifier. For detecting green fruit, the Scout detects using statistical classifiers generated by boosting algorithms. The most challenging aspects of detecting fruit are occlusions, shadows, shape and color coincidences, and ambient lighting conditions such as bright sunlight. Once an apple is identified, the stereo cameras enable the system to determine its location and size.

The Vision Robotics field trial was conducted at Allan Brothers Othello Orchard in Washington. Prior to VRC’s arrival, Washington State University (WSU) Extension Educator Karen Lewis, and interns with the Washington Tree Fruit Research Commission, hand tagged, mapped, and sized fruit on several rows of fruit. This data allows for the groundtruthing or validation component of the test. Koselka took time out to speak about the development of a similar unit that counts oranges with the interns.

On one of the field trials for that unit, Koselka asked the grower to estimate his crop. The farm manager responded with 1,200 bushels. For this test, Vision Robotics scouted and counted 1,900 — a sizable difference and one that would have a large financial impact.

As Koselka stated, “not only will it [Scout] be able to accurately count a crop and locate each piece of fruit, it will ultimately feed into a robot harvester that will pick all the apples.” And if that was not enough, Koselka added, “Oh, and a pruner is not far off either!”

Even More Potential

The Newton Scout utility will be enhanced in the future with the addition of other possible technologies, including near infrared (NIR). As Newton evolves, it will have multiple uses throughout the fruit growing cycle — for example, assessment of sugar and acid content and internal damage. CASC is also making progress in other automation fronts that bridge with Vision Robotics’ work.

Carnegie Mellon University (CMU) researchers, led by Sanjiv Singh, are developing an Autonomous Prime Mover (APM) capable of towing the Newton Scout safely through entire orchard blocks. In the future, they will also automate an agricultural harvesting platform so that workers no longer need to climb on ladders to pick fruit. The APM’s multifunctionality, autonomous drive, and wireless networking could move it into the center of precision management of specialty crops.

The CASC team, led by CMU, includes engineers, scientists, Extension educators, growers, equipment manufacturers, and industry representatives. This plurality of expertise has led to synergistic endeavors that push the envelope in both the engineering and the science components of the project. For example, Ben Grocholsky at CMU is developing a geospatial system to map and locate each tree in an orchard; in cooperation with the plant science group, he was able to augment this information with local temperature and NDVI data that has the future possibility of being incorporated to detect plant stress and disease.

In another example of the synergy of the CASC team, during a field trial of an automated caliper measurement device, growers told the CASC team that a simpler tree counter would also be very useful. Singh’s group returned to CMU and produced a prototype tree counter ready to be tested in weeks; they will return to nurseries and test the counter this fall. The total time from conception to development will be less than four months, and if successful, this device could go into production within a year.

It will only be a matter of time before specialty crop growers also start using vehicles, sensors, devices, and software that automate farm operations, increase farm efficiency, and meet the ultimate goal of delivering to the consumer a consistently high quality affordable eating experience while returning money to the farming operation.

For more information on Comprehensive Automation for Specialty Crops, please visit www.cascrop.com.

Leave a Reply

Featured Stories

Farm ManagementBill Gates Uploading Lots Of Florida Farmland
October 22, 2014
Recent acquisition for the Microsoft icon involves more than 4,500 acres for nearly $28 million. Read More
GenNext GrowersRegister Now For The GenNext Growers Webcast
October 22, 2014
In the first installment of the GenNext Webcast Series, you will learn how to engage with lawmakers and ag associations to help shape a better future for your business and for agriculture. Read More
FruitsAnnual Santa Maria Strawberry Meeting Approaching
October 22, 2014
The meeting will focus on IPM practices, weed management, updates on fumigation, and more. Read More
Apples & PearsUSApple’s AppletizeMe Personality Quiz Help Consumers Find Ideal Apple Varieties and Perfect Food Pairings
October 22, 2014
Award-winning national apple month campaign returns with more featured varieties and informative videos. Read More
Farm ManagementUSDA Awards $18 Million To Stimulate Research And Development Innovation
October 22, 2014
The grants will provide resources to help small operations create breakthroughs, says USDA Secretary Tom Vilsack. Read More
CitrusEPA Launches Program To Reduce Pesticide Drift
October 22, 2014
The voluntary star-rating program aims to protect people, wildlife and the environment. Read More
CitrusMcCrometer Introduces New Ag Management Mobile App
October 22, 2014
The app displays data on weather, ET, soil moisture, water flow, and more. Read More
Apples & PearsNew York Apple Growers Partner With Grocer To Introduce SnapDragon Apples
October 22, 2014
First apples of this new variety to hit Wegmans stores as part of a marketing effort. Read More
CitrusAgriculture, Interior Departments Partner To Measure Conservation Impacts On Water Quality
October 22, 2014
Goal is to provide science-based information for watershed wellness. Read More
Stone FruitIntroducing A Gem Of A Peach For Central Florida
October 22, 2014
The newest variety to come from the UF Stone Fruit Breeding Program is another “gem,” UFGem! This peach variety is Read More

The Latest

Farm ManagementBill Gates Uploading Lots Of Florida Farmland
October 22, 2014
Recent acquisition for the Microsoft icon involves more than 4,500 acres for nearly $28 million. Read More
GenNext GrowersRegister Now For The GenNext Growers Webcast
October 22, 2014
In the first installment of the GenNext Webcast Series, you will learn how to engage with lawmakers and ag associations to help shape a better future for your business and for agriculture. Read More
FruitsAnnual Santa Maria Strawberry Meeting Approaching
October 22, 2014
The meeting will focus on IPM practices, weed management, updates on fumigation, and more. Read More
Apples & PearsUSApple’s AppletizeMe Personality Quiz Help Consumers…
October 22, 2014
Award-winning national apple month campaign returns with more featured varieties and informative videos. Read More
Farm ManagementUSDA Awards $18 Million To Stimulate Research And Devel…
October 22, 2014
The grants will provide resources to help small operations create breakthroughs, says USDA Secretary Tom Vilsack. Read More
CitrusEPA Launches Program To Reduce Pesticide Drift
October 22, 2014
The voluntary star-rating program aims to protect people, wildlife and the environment. Read More
CitrusMcCrometer Introduces New Ag Management Mobile App
October 22, 2014
The app displays data on weather, ET, soil moisture, water flow, and more. Read More
Apples & PearsNew York Apple Growers Partner With Grocer To Introduce…
October 22, 2014
First apples of this new variety to hit Wegmans stores as part of a marketing effort. Read More
CitrusAgriculture, Interior Departments Partner To Measure Co…
October 22, 2014
Goal is to provide science-based information for watershed wellness. Read More
Stone FruitIntroducing A Gem Of A Peach For Central Florida
October 22, 2014
The newest variety to come from the UF Stone Fruit Breeding Program is another “gem,” UFGem! This peach variety is Read More
CitrusIt’s Time For Young Farming Leaders To Engage [Op…
October 21, 2014
Florida Grower editor Frank Giles says there are capable and talented youths in the ag field ready to step up and take the reins. Read More
Farm ManagementResearchers Finding Biofuel Fits For Florida
October 21, 2014
Alternative crops identified that are suited for the state's climate and can easily be processed for power. Read More
CitrusBayer CropScience Launches New Award To Recognize Produ…
October 21, 2014
Award recognizes innovation that enhances the role of produce in creating better lives. Read More
FruitsCalifornia Department Of Food And Agriculture Awards $3…
October 21, 2014
State Water Efficiency And Enhancement Program highlights projects implement irrigation systems that reduce water and energy use. Read More
FruitsWashington State University Researchers See How Plants …
October 21, 2014
Results could lead to development of crops that recover from sun damage more easily. Read More
CitrusReport: Major Food And Agriculture Employers Can’t Fi…
October 21, 2014
Current shortage of young farming professionals means ample employment opportunity for the next generation. Read More
Apples & PearsInvasive Pest Found First Time In U.S. At Border Crossi…
October 20, 2014
Moth known to feed on apples, apple buds, leaves, and shoots, is found in a container full of pump valves originating in China. Read More
More Vegetables6 Savory Tomato And Pepper Varieties For Florida Grower…
October 20, 2014
Leading vegetable seed companies highlight a half dozen high-performance selections for producers in the Sunshine State. Read More