Apple Harvesters Use Vision To “See” Fruit

Editor’s Note: This past October, Tony Koselka, Jillian Cannons, and Ryan Carlon of Vision Robotics Corporation (VRC) returned to Washington for further field trials of the Newton Scout, a vision-based system for automatic crop load assessment. The following report highlights their results.

Developed in part through the USDA-funded project Comprehensive Automation for Specialty Crops (CASC), the Newton Scout will revolutionize orchard operations by providing growers with a map that accurately maps fruit placement, fruit size, and number throughout the growing season. Fruit maps will be used to determine crop load, which in turn will give growers the information to develop harvest and marketing strategies. And this is just the beginning. Developing reliable vision systems that are robust enough for use in orchards opens up all kinds of possibilities in the area of pest and disease detection, plant health assessment, and the use of robotics at harvest.

The Newton Scout integrates stereo cameras with a custom image processing algorithm. The algorithms use classifiers to identify the apples, where the simplest classifier uses color and shape. For red apples, color is the primary classifier. For detecting green fruit, the Scout detects using statistical classifiers generated by boosting algorithms. The most challenging aspects of detecting fruit are occlusions, shadows, shape and color coincidences, and ambient lighting conditions such as bright sunlight. Once an apple is identified, the stereo cameras enable the system to determine its location and size.

The Vision Robotics field trial was conducted at Allan Brothers Othello Orchard in Washington. Prior to VRC’s arrival, Washington State University (WSU) Extension Educator Karen Lewis, and interns with the Washington Tree Fruit Research Commission, hand tagged, mapped, and sized fruit on several rows of fruit. This data allows for the groundtruthing or validation component of the test. Koselka took time out to speak about the development of a similar unit that counts oranges with the interns.

On one of the field trials for that unit, Koselka asked the grower to estimate his crop. The farm manager responded with 1,200 bushels. For this test, Vision Robotics scouted and counted 1,900 — a sizable difference and one that would have a large financial impact.

As Koselka stated, “not only will it [Scout] be able to accurately count a crop and locate each piece of fruit, it will ultimately feed into a robot harvester that will pick all the apples.” And if that was not enough, Koselka added, “Oh, and a pruner is not far off either!”

Even More Potential

The Newton Scout utility will be enhanced in the future with the addition of other possible technologies, including near infrared (NIR). As Newton evolves, it will have multiple uses throughout the fruit growing cycle — for example, assessment of sugar and acid content and internal damage. CASC is also making progress in other automation fronts that bridge with Vision Robotics’ work.

Carnegie Mellon University (CMU) researchers, led by Sanjiv Singh, are developing an Autonomous Prime Mover (APM) capable of towing the Newton Scout safely through entire orchard blocks. In the future, they will also automate an agricultural harvesting platform so that workers no longer need to climb on ladders to pick fruit. The APM’s multifunctionality, autonomous drive, and wireless networking could move it into the center of precision management of specialty crops.

The CASC team, led by CMU, includes engineers, scientists, Extension educators, growers, equipment manufacturers, and industry representatives. This plurality of expertise has led to synergistic endeavors that push the envelope in both the engineering and the science components of the project. For example, Ben Grocholsky at CMU is developing a geospatial system to map and locate each tree in an orchard; in cooperation with the plant science group, he was able to augment this information with local temperature and NDVI data that has the future possibility of being incorporated to detect plant stress and disease.

In another example of the synergy of the CASC team, during a field trial of an automated caliper measurement device, growers told the CASC team that a simpler tree counter would also be very useful. Singh’s group returned to CMU and produced a prototype tree counter ready to be tested in weeks; they will return to nurseries and test the counter this fall. The total time from conception to development will be less than four months, and if successful, this device could go into production within a year.

It will only be a matter of time before specialty crop growers also start using vehicles, sensors, devices, and software that automate farm operations, increase farm efficiency, and meet the ultimate goal of delivering to the consumer a consistently high quality affordable eating experience while returning money to the farming operation.

For more information on Comprehensive Automation for Specialty Crops, please visit www.cascrop.com.

Leave a Reply

Featured Stories
Ready To Spring
Insect Control
May 26, 2016
Temperature, Location Key To Predicting Leaffooted Bug Pressure
This year, leaffooted bugs are expected to be a significant problem in almonds and pistachios, but watching temperature and the Read More
First-year impact of Prunus replant disease at the Firebaugh replant trial; stunted trees in the foreground row were planted in plot of non-fumigated replant soil. (Photo credit: University of California Agriculture)
Fruits
May 26, 2016
Consider Fumigating For Nematodes Before Replanting Almonds, Stone Fruit
Stone fruit and almond growers looking to replant orchards might want to invest in soil samples to assess nematode populations Read More
An open air farm dinner at Tangletown's farm FEATURE
Farm Marketing
May 26, 2016
Farm Dinners: Your Most Powerful Marketing Tool?
Farm dinners are a hassle. They’re expensive. And ridiculously effective. Read More
Whiteflies on a leaf
Cucurbits
May 25, 2016
Whitefly Threat Has Florida Growers On High Alert
Researchers, state agencies working together to prevent a potential outbreak. Read More
The MyIPM series of apps are available for Android and IOS devices. (Photo Credit: Scott Miller / Clemson University)
Crop Protection
May 25, 2016
More Apps Help Growers Identify Insects And Diseases
Berries, apples, pears, and cherries now rolled into new app series from Clemson University. Read More
close-up of stop light
Citrus
May 25, 2016
Monsanto Says Bayer Bid ‘Financially Inadequate’
Proposal cited as undervalued, not able to address financial, regulatory risks. Read More
cherry cracking
Stone Fruit
May 25, 2016
Researchers Study Why Cherry Cracking Affects Different Varieties
German researchers studied how water uptake and fruit skin determined a cultivar’s susceptibility to cherry cracking. Read More
Port Of Miami shipyard
Farm Management
May 25, 2016
Report Highlights Benefits Of Trans-Pacific Partnership
National Potato Council says report from the International Trade Commission offers the benefits the free trade agreement would offer growers. Read More
Wetland and streamside vegetation serves as a buffer to filter excess nutrients from water running off agricultural land. Photo by Scott Bauer.
Fruits
May 25, 2016
Farm Bureau Says EPA, Army Corps Of Engineers Violate Clean Water Act
AFBF told Congress that the Army Corps' novel interpretations of environmental law are threatening farmers in California and other areas of the country. Read More
The Latest
Insect Control
May 26, 2016
Temperature, Location Key To Predicting …
This year, leaffooted bugs are expected to be a significant problem in almonds and pistachios, but watching temperature and the Read More
Fruits
May 26, 2016
Consider Fumigating For Nematodes Before…
Stone fruit and almond growers looking to replant orchards might want to invest in soil samples to assess nematode populations Read More
Farm Marketing
May 26, 2016
Who Grows Organically — And Who Doesn…
We surveyed 816 fruit and vegetable growers and found that farm marketers and vegetable growers are much more likely than their peers to embrace the practice. Read More
Farm Marketing
May 26, 2016
Farm Dinners: Your Most Powerful Marketi…
Farm dinners are a hassle. They’re expensive. And ridiculously effective. Read More
Cucurbits
May 25, 2016
Whitefly Threat Has Florida Growers On H…
Researchers, state agencies working together to prevent a potential outbreak. Read More
Crop Protection
May 25, 2016
More Apps Help Growers Identify Insects …
Berries, apples, pears, and cherries now rolled into new app series from Clemson University. Read More
Citrus
May 25, 2016
Monsanto Says Bayer Bid ‘Financially Ina…
Proposal cited as undervalued, not able to address financial, regulatory risks. Read More
Stone Fruit
May 25, 2016
Researchers Study Why Cherry Cracking Af…
German researchers studied how water uptake and fruit skin determined a cultivar’s susceptibility to cherry cracking. Read More
Farm Management
May 25, 2016
Report Highlights Benefits Of Trans-Paci…
National Potato Council says report from the International Trade Commission offers the benefits the free trade agreement would offer growers. Read More
Fruits
May 25, 2016
Farm Bureau Says EPA, Army Corps Of Engi…
AFBF told Congress that the Army Corps' novel interpretations of environmental law are threatening farmers in California and other areas of the country. Read More
Citrus
May 25, 2016
NRCS Invests $4.3 Million To Combat Clim…
The Natural Resources Conservation Service in California is committing funds to help farmers and ranchers mitigate the effects of climate change and help to reduce greenhouse gas emissions. Read More
Citrus
May 24, 2016
Panama Canal Expansion To Bring Trade Op…
Agriculture Commissioner Adam Putnam says a confluence of events is putting the state's producers in a good spot to open new markets. Read More
Citrus
May 24, 2016
Social Media Posts On GMOs Falling Flat …
Hopefully, the hysteria the West has perpetuated on genetic engineering will not stifle the potential of moving our production forward enough to help feed a growing global population. Read More
Fruits
May 24, 2016
Organic Food Sales Hit Record $43.3 Bill…
Organic Trade Association says organic fruits and vegetables logged sales of $14.4 billion. Read More
Grapes
May 24, 2016
Sea Change Coming To California Winegrap…
Vines continue to be pulled from the state’s interior, while planting is expected to increase along coast and in Delta area. Read More
Citrus
May 24, 2016
Report Says GMOs Offer No Risk To Human …
The study stresses the need for proper resistance management, the need for a different ways to evaluate all new crop varieties, regardless of process in which they were developed. Read More
Citrus
May 24, 2016
Full-Time Effort Required To Feed Farm-T…
Florida Commissioner of Agriculture Adam Putnam gives a shout-out to those providing fuel to kids during the school year as well as the summer months. Read More
Citrus
May 23, 2016
World Health Organization Experts: Glyph…
Risk unlikely when consuming crops treated with herbicide. Read More
[gravityform id="62" title="false" description="false"]