Apple Harvesters Use Vision To “See” Fruit

Editor’s Note: This past October, Tony Koselka, Jillian Cannons, and Ryan Carlon of Vision Robotics Corporation (VRC) returned to Washington for further field trials of the Newton Scout, a vision-based system for automatic crop load assessment. The following report highlights their results.

Developed in part through the USDA-funded project Comprehensive Automation for Specialty Crops (CASC), the Newton Scout will revolutionize orchard operations by providing growers with a map that accurately maps fruit placement, fruit size, and number throughout the growing season. Fruit maps will be used to determine crop load, which in turn will give growers the information to develop harvest and marketing strategies. And this is just the beginning. Developing reliable vision systems that are robust enough for use in orchards opens up all kinds of possibilities in the area of pest and disease detection, plant health assessment, and the use of robotics at harvest.

The Newton Scout integrates stereo cameras with a custom image processing algorithm. The algorithms use classifiers to identify the apples, where the simplest classifier uses color and shape. For red apples, color is the primary classifier. For detecting green fruit, the Scout detects using statistical classifiers generated by boosting algorithms. The most challenging aspects of detecting fruit are occlusions, shadows, shape and color coincidences, and ambient lighting conditions such as bright sunlight. Once an apple is identified, the stereo cameras enable the system to determine its location and size.

The Vision Robotics field trial was conducted at Allan Brothers Othello Orchard in Washington. Prior to VRC’s arrival, Washington State University (WSU) Extension Educator Karen Lewis, and interns with the Washington Tree Fruit Research Commission, hand tagged, mapped, and sized fruit on several rows of fruit. This data allows for the groundtruthing or validation component of the test. Koselka took time out to speak about the development of a similar unit that counts oranges with the interns.

On one of the field trials for that unit, Koselka asked the grower to estimate his crop. The farm manager responded with 1,200 bushels. For this test, Vision Robotics scouted and counted 1,900 — a sizable difference and one that would have a large financial impact.

As Koselka stated, “not only will it [Scout] be able to accurately count a crop and locate each piece of fruit, it will ultimately feed into a robot harvester that will pick all the apples.” And if that was not enough, Koselka added, “Oh, and a pruner is not far off either!”

Even More Potential

The Newton Scout utility will be enhanced in the future with the addition of other possible technologies, including near infrared (NIR). As Newton evolves, it will have multiple uses throughout the fruit growing cycle — for example, assessment of sugar and acid content and internal damage. CASC is also making progress in other automation fronts that bridge with Vision Robotics’ work.

Carnegie Mellon University (CMU) researchers, led by Sanjiv Singh, are developing an Autonomous Prime Mover (APM) capable of towing the Newton Scout safely through entire orchard blocks. In the future, they will also automate an agricultural harvesting platform so that workers no longer need to climb on ladders to pick fruit. The APM’s multifunctionality, autonomous drive, and wireless networking could move it into the center of precision management of specialty crops.

The CASC team, led by CMU, includes engineers, scientists, Extension educators, growers, equipment manufacturers, and industry representatives. This plurality of expertise has led to synergistic endeavors that push the envelope in both the engineering and the science components of the project. For example, Ben Grocholsky at CMU is developing a geospatial system to map and locate each tree in an orchard; in cooperation with the plant science group, he was able to augment this information with local temperature and NDVI data that has the future possibility of being incorporated to detect plant stress and disease.

In another example of the synergy of the CASC team, during a field trial of an automated caliper measurement device, growers told the CASC team that a simpler tree counter would also be very useful. Singh’s group returned to CMU and produced a prototype tree counter ready to be tested in weeks; they will return to nurseries and test the counter this fall. The total time from conception to development will be less than four months, and if successful, this device could go into production within a year.

It will only be a matter of time before specialty crop growers also start using vehicles, sensors, devices, and software that automate farm operations, increase farm efficiency, and meet the ultimate goal of delivering to the consumer a consistently high quality affordable eating experience while returning money to the farming operation.

For more information on Comprehensive Automation for Specialty Crops, please visit www.cascrop.com.

Leave a Reply

Featured Stories
Greg Lang of Michigan State discusses cherry tree pruning at IFTA 2016.
Fruits
February 8, 2016
Precision Fruit Growing, Business Management Top Agenda At IFTA Conference Kickoff
American Fruit Grower magazine managing editor Christina Herrick posts her updates from the 59th annual gathering. Read More
U.S. Capitol Building FEATURE
Farm Marketing
February 8, 2016
Food Policy: If You Don’t Speak, Others Will Speak For You
British Columbia’s former Minister of Agriculture, Corky Evans, spoke with attendees at 2016 North American Farmers’ Direct Marketing Association’s Convention about Read More
Winery at Krause Berry Farm FEATURE
Farm Marketing
February 8, 2016
How British Columbia And U.S. Regs Are Shaping Farm Markets [Opinion]
Laws passed decades ago to protect small farms aren't keeping up with the realities of modern farmers and their marketing efforts. Restrictions can shut down hosting weddings, but offer incentives to open wineries. Read More
Table with cut flowers at Krause Berry Farm
Farm Marketing
February 8, 2016
3 Tips On How Not To Lose Your Shirt When Adding A Restaurant To Your Farm
Chef and consultant Chuck Currie told NAFDMA Convention attendees the basic principles of what makes a farm market eatery a success or a flop. Read More
field plot
Crop Protection
February 8, 2016
7 Steps To See If Biocontrols Pencil Out For Your Operation
Have you considered adding biocontrols to your operation? If you have, there’s no doubt you’ve already weighed the potential costs, Read More
The family at the helm of Sterman Masser Potato Farms includes (from left) David, Keith, Helen, and Julie.
Photo credit: Rosemary Gordon
Potatoes
February 7, 2016
A Lesson In Marketing From Sterman Masser Potato Farms
Always looking for new product offerings and ways to expand, Sterman Masser Potato Farms caters to consumers. Read More
Thomas Björkman, the director of the Eastern Broccoli Project, discusses details on the challenges of broccoli production in the East.
Vegetables
February 7, 2016
The Eastern Broccoli Project Moves Forward
As acreage increases in the East, mid-sized growers seek channels to distribute their product and continue the search for varieties. Read More
Apples On Display
Fruits
February 6, 2016
Best Research Is Industry Driven [Opinion]
Like many of you, I’m not crazy about a lot of government programs. It’s not that their genesis isn’t noble; God Read More
RosBREED will have an impact on nearly all the major U.S. rosaceous crop production areas.
Fruits
February 6, 2016
RosBREED: A National Effort To Improve The Fruit Industry
The project focuses on improving disease resistance and fruit quality through better genetics and breeding. Read More
The Latest
Fruits
February 8, 2016
Precision Fruit Growing, Business Manage…
American Fruit Grower magazine managing editor Christina Herrick posts her updates from the 59th annual gathering. Read More
Farm Marketing
February 8, 2016
Food Policy: If You Don’t Speak, O…
British Columbia’s former Minister of Agriculture, Corky Evans, spoke with attendees at 2016 North American Farmers’ Direct Marketing Association’s Convention about Read More
Farm Marketing
February 8, 2016
How British Columbia And U.S. Regs Are S…
Laws passed decades ago to protect small farms aren't keeping up with the realities of modern farmers and their marketing efforts. Restrictions can shut down hosting weddings, but offer incentives to open wineries. Read More
Farm Marketing
February 8, 2016
3 Tips On How Not To Lose Your Shirt Whe…
Chef and consultant Chuck Currie told NAFDMA Convention attendees the basic principles of what makes a farm market eatery a success or a flop. Read More
Farm Marketing
February 8, 2016
10 Ideas You Can Steal From NAFDMA’…
We visited several farm markets, as well as poultry and cheese farms and garden shops, this past week. Here are just a few of the best ideas we spied. Read More
Crop Protection
February 8, 2016
7 Steps To See If Biocontrols Pencil Out…
Have you considered adding biocontrols to your operation? If you have, there’s no doubt you’ve already weighed the potential costs, Read More
Potatoes
February 7, 2016
A Lesson In Marketing From Sterman Masse…
Always looking for new product offerings and ways to expand, Sterman Masser Potato Farms caters to consumers. Read More
Vegetables
February 7, 2016
The Eastern Broccoli Project Moves Forwa…
As acreage increases in the East, mid-sized growers seek channels to distribute their product and continue the search for varieties. Read More
Fruits
February 6, 2016
Best Research Is Industry Driven [Opinio…
Like many of you, I’m not crazy about a lot of government programs. It’s not that their genesis isn’t noble; God Read More
Fruits
February 6, 2016
RosBREED: A National Effort To Improve T…
The project focuses on improving disease resistance and fruit quality through better genetics and breeding. Read More
Berries
February 5, 2016
Moving Beyond Methyl Bromide With Biofun…
Editor’s note: University of California Farm Advisor Mark Bolda will present much more information on this topic at the Biocontrols Read More
Grapes
February 5, 2016
Geotextiles Can Help Prevent Winter Grap…
Although Northeast and Midwest grape growing regions have so far been spared from extreme cold this year, the winters of Read More
Grapes
February 5, 2016
Researchers Test New Method To Mitigate …
The use of evaporative cooling in vineyards during hot weather isn’t a new concept, but researchers in Australia are testing Read More
Crop Protection
February 5, 2016
Register Now For Biocontrols Conference …
Space is limited, so register now to attend the first-ever Biocontrols USA 2016 Conference Field and Greenhouse Tour. Following the Read More
Grapes
February 4, 2016
Northern Grapes Project Fuels A Market F…
 There’s a burgeoning market for cold-weather grapes. The Northern Grapes Project, funded in 2011 by a USDA National Institute of Read More
Citrus
February 4, 2016
Can Olive Oil Grease Skids For Florida’s…
Researchers, interested growers help launch new industry. Read More
Citrus
February 3, 2016
Sunkist Expands Organic Portfolio
With the organic citrus sector growing three times as fast as conventional, the 123-year-old cooperative is keeping up with the times. Read More
Citrus
February 3, 2016
Syngenta OKs Buyout By ChemChina
Deal valued at more than $43 billion; Syngenta management team to stay intact. Read More
[gravityform id="62" title="false" description="false"]