Apple Harvesters Use Vision To “See” Fruit

Editor’s Note: This past October, Tony Koselka, Jillian Cannons, and Ryan Carlon of Vision Robotics Corporation (VRC) returned to Washington for further field trials of the Newton Scout, a vision-based system for automatic crop load assessment. The following report highlights their results.

Developed in part through the USDA-funded project Comprehensive Automation for Specialty Crops (CASC), the Newton Scout will revolutionize orchard operations by providing growers with a map that accurately maps fruit placement, fruit size, and number throughout the growing season. Fruit maps will be used to determine crop load, which in turn will give growers the information to develop harvest and marketing strategies. And this is just the beginning. Developing reliable vision systems that are robust enough for use in orchards opens up all kinds of possibilities in the area of pest and disease detection, plant health assessment, and the use of robotics at harvest.

The Newton Scout integrates stereo cameras with a custom image processing algorithm. The algorithms use classifiers to identify the apples, where the simplest classifier uses color and shape. For red apples, color is the primary classifier. For detecting green fruit, the Scout detects using statistical classifiers generated by boosting algorithms. The most challenging aspects of detecting fruit are occlusions, shadows, shape and color coincidences, and ambient lighting conditions such as bright sunlight. Once an apple is identified, the stereo cameras enable the system to determine its location and size.

The Vision Robotics field trial was conducted at Allan Brothers Othello Orchard in Washington. Prior to VRC’s arrival, Washington State University (WSU) Extension Educator Karen Lewis, and interns with the Washington Tree Fruit Research Commission, hand tagged, mapped, and sized fruit on several rows of fruit. This data allows for the groundtruthing or validation component of the test. Koselka took time out to speak about the development of a similar unit that counts oranges with the interns.

On one of the field trials for that unit, Koselka asked the grower to estimate his crop. The farm manager responded with 1,200 bushels. For this test, Vision Robotics scouted and counted 1,900 — a sizable difference and one that would have a large financial impact.

As Koselka stated, “not only will it [Scout] be able to accurately count a crop and locate each piece of fruit, it will ultimately feed into a robot harvester that will pick all the apples.” And if that was not enough, Koselka added, “Oh, and a pruner is not far off either!”

Even More Potential

The Newton Scout utility will be enhanced in the future with the addition of other possible technologies, including near infrared (NIR). As Newton evolves, it will have multiple uses throughout the fruit growing cycle — for example, assessment of sugar and acid content and internal damage. CASC is also making progress in other automation fronts that bridge with Vision Robotics’ work.

Carnegie Mellon University (CMU) researchers, led by Sanjiv Singh, are developing an Autonomous Prime Mover (APM) capable of towing the Newton Scout safely through entire orchard blocks. In the future, they will also automate an agricultural harvesting platform so that workers no longer need to climb on ladders to pick fruit. The APM’s multifunctionality, autonomous drive, and wireless networking could move it into the center of precision management of specialty crops.

The CASC team, led by CMU, includes engineers, scientists, Extension educators, growers, equipment manufacturers, and industry representatives. This plurality of expertise has led to synergistic endeavors that push the envelope in both the engineering and the science components of the project. For example, Ben Grocholsky at CMU is developing a geospatial system to map and locate each tree in an orchard; in cooperation with the plant science group, he was able to augment this information with local temperature and NDVI data that has the future possibility of being incorporated to detect plant stress and disease.

In another example of the synergy of the CASC team, during a field trial of an automated caliper measurement device, growers told the CASC team that a simpler tree counter would also be very useful. Singh’s group returned to CMU and produced a prototype tree counter ready to be tested in weeks; they will return to nurseries and test the counter this fall. The total time from conception to development will be less than four months, and if successful, this device could go into production within a year.

It will only be a matter of time before specialty crop growers also start using vehicles, sensors, devices, and software that automate farm operations, increase farm efficiency, and meet the ultimate goal of delivering to the consumer a consistently high quality affordable eating experience while returning money to the farming operation.

For more information on Comprehensive Automation for Specialty Crops, please visit www.cascrop.com.

Leave a Reply

Featured Stories
Berries
May 26, 2017
Jury Sides with University in Strawberry Breeding Lawsuit
Decision says former breeders infringed on UC patents to develop material for corporate breeding firm. Read More
Managed bee hive boxes around a blueberry field
Nuts
May 25, 2017
Stolen Hives Recovered in Nearly $1M Heist
Man likely stole hives from across the state of California in the last three years. Read More
Panama Canal Expansion opening celebration
Farm Management
May 25, 2017
Plight of Trade Policy Grounding Many U.S. Growers
Much-needed investments in labor-saving technology could be key to long-term survival. Read More
Cannabis plant closeup
Farm Management
May 25, 2017
Florida Farm Winning Case for Medicinal Cannabis License
Judge rules in favor of 3 Boys Farm in the quest to expand its crop/business portfolio. Read More
hands holding a globe
Citrus
May 25, 2017
Are We Overstating the World’s Overpopulation Problem?
If growers can’t achieve a fair return on investment and stay in business, it won’t matter what the population is in 2050. We all will be trouble. Read More
Falcon-takes-flight-at-Duncan-Family-Farms
Citrus
May 24, 2017
Falcons Scare off Other Birds at Duncan Family Farms
It collaborates with a firm that uses trained falcons to intimidate and scare off nuisance birds such as the horned larks and pigeons that frequent the area. Read More
Organic
May 24, 2017
Organic Growers in Turmoil (Opinion)
Although consumers are demanding, and getting more organic produce options, growers are grappling with fundamental questions. Read More
Corn leaf aphids on sweet corn
Sweet Corn
May 24, 2017
Stay on Your A Game to Conquer Corn Leaf Aphid
Learn how to identify, the survival and spread, as well as management methods for this vegetable pest. Read More
Florida Ag Hall of Famer William Cook
Citrus
May 23, 2017
Search is on For Next Crop of Florida Agriculture Hall of Famers
Nomination period now open for longtime industry honor. Read More
Apples & Pears
May 23, 2017
Grower Shares Personal Impact of Labor Shortage
USApple leader Phil Glaize of Virginia stresses need for reform with current labor regulations, programs at Congressional briefing on agriculture labor crisis. Read More
Stone Fruit
May 23, 2017
Northeast Peach Crop Rebounds
After a mild winter and relatively uneventful bloom time, the crop is setting up nicely for the region. Read More
Berries
May 23, 2017
Researchers Examining Potential for Longer Berry Season in Northeast
Team at Penn State University using low and high tunnels and plastic coverings to extend raspberry and strawberry seasons. Read More
honeybees
Citrus
May 23, 2017
Webinar Series Highlights New Research on Honeybee Health
National Pollinator Week webinar showcases research to improve honeybee colony health by 2020. Read More
stack of lemons
Citrus
May 23, 2017
California Lemon Growers Sue USDA
Lemon industry files suit against rule allowing importation of Argentine lemons. Read More
Bee Vectoring Technologies experimental hives
Citrus
May 23, 2017
Insects (Bad and Good) Make for Interesting Field Trips
Disease-fighting bees and the deadly citrus psyllid were at the center of two recent, research-fueled out-of-office adventures. Read More
The Latest
Berries
May 26, 2017
Jury Sides with University in Strawberry…
Decision says former breeders infringed on UC patents to develop material for corporate breeding firm. Read More
Nuts
May 25, 2017
Stolen Hives Recovered in Nearly $1M Hei…
Man likely stole hives from across the state of California in the last three years. Read More
Farm Management
May 25, 2017
Plight of Trade Policy Grounding Many U.…
Much-needed investments in labor-saving technology could be key to long-term survival. Read More
Farm Management
May 25, 2017
Florida Farm Winning Case for Medicinal …
Judge rules in favor of 3 Boys Farm in the quest to expand its crop/business portfolio. Read More
Citrus
May 25, 2017
Are We Overstating the World’s Ove…
If growers can’t achieve a fair return on investment and stay in business, it won’t matter what the population is in 2050. We all will be trouble. Read More
Citrus
May 24, 2017
Falcons Scare off Other Birds at Duncan …
It collaborates with a firm that uses trained falcons to intimidate and scare off nuisance birds such as the horned larks and pigeons that frequent the area. Read More
Organic
May 24, 2017
Organic Growers in Turmoil (Opinion)
Although consumers are demanding, and getting more organic produce options, growers are grappling with fundamental questions. Read More
Sweet Corn
May 24, 2017
Stay on Your A Game to Conquer Corn Leaf…
Learn how to identify, the survival and spread, as well as management methods for this vegetable pest. Read More
Citrus
May 23, 2017
Search is on For Next Crop of Florida Ag…
Nomination period now open for longtime industry honor. Read More
Apples & Pears
May 23, 2017
Grower Shares Personal Impact of Labor S…
USApple leader Phil Glaize of Virginia stresses need for reform with current labor regulations, programs at Congressional briefing on agriculture labor crisis. Read More
Stone Fruit
May 23, 2017
Northeast Peach Crop Rebounds
After a mild winter and relatively uneventful bloom time, the crop is setting up nicely for the region. Read More
Berries
May 23, 2017
Researchers Examining Potential for Long…
Team at Penn State University using low and high tunnels and plastic coverings to extend raspberry and strawberry seasons. Read More
Citrus
May 23, 2017
Webinar Series Highlights New Research o…
National Pollinator Week webinar showcases research to improve honeybee colony health by 2020. Read More
Citrus
May 23, 2017
California Lemon Growers Sue USDA
Lemon industry files suit against rule allowing importation of Argentine lemons. Read More
Citrus
May 23, 2017
Insects (Bad and Good) Make for Interest…
Disease-fighting bees and the deadly citrus psyllid were at the center of two recent, research-fueled out-of-office adventures. Read More
Crop Protection
May 23, 2017
Valent Grants Organic Farming Scholarshi…
University of Florida Doctor of Plant Medicine student receives $1,500 toward the development of training materials. Read More
Varieties & Rootstocks
May 23, 2017
In Quest for Low-Seeded Citrus, It Takes…
Momentum for viable mandarin hybrids for Florida reaching a peak in the land down under. Read More
GenNext Growers
May 22, 2017
How to Make Dollars and Sense of Today&#…
Despite current challenges impacting the future of the North American fruit and vegetable industry, there are opportunities to be had. Read More