Apple Harvesters Use Vision To “See” Fruit

Editor’s Note: This past October, Tony Koselka, Jillian Cannons, and Ryan Carlon of Vision Robotics Corporation (VRC) returned to Washington for further field trials of the Newton Scout, a vision-based system for automatic crop load assessment. The following report highlights their results.

Developed in part through the USDA-funded project Comprehensive Automation for Specialty Crops (CASC), the Newton Scout will revolutionize orchard operations by providing growers with a map that accurately maps fruit placement, fruit size, and number throughout the growing season. Fruit maps will be used to determine crop load, which in turn will give growers the information to develop harvest and marketing strategies. And this is just the beginning. Developing reliable vision systems that are robust enough for use in orchards opens up all kinds of possibilities in the area of pest and disease detection, plant health assessment, and the use of robotics at harvest.

The Newton Scout integrates stereo cameras with a custom image processing algorithm. The algorithms use classifiers to identify the apples, where the simplest classifier uses color and shape. For red apples, color is the primary classifier. For detecting green fruit, the Scout detects using statistical classifiers generated by boosting algorithms. The most challenging aspects of detecting fruit are occlusions, shadows, shape and color coincidences, and ambient lighting conditions such as bright sunlight. Once an apple is identified, the stereo cameras enable the system to determine its location and size.

The Vision Robotics field trial was conducted at Allan Brothers Othello Orchard in Washington. Prior to VRC’s arrival, Washington State University (WSU) Extension Educator Karen Lewis, and interns with the Washington Tree Fruit Research Commission, hand tagged, mapped, and sized fruit on several rows of fruit. This data allows for the groundtruthing or validation component of the test. Koselka took time out to speak about the development of a similar unit that counts oranges with the interns.

On one of the field trials for that unit, Koselka asked the grower to estimate his crop. The farm manager responded with 1,200 bushels. For this test, Vision Robotics scouted and counted 1,900 — a sizable difference and one that would have a large financial impact.

As Koselka stated, “not only will it [Scout] be able to accurately count a crop and locate each piece of fruit, it will ultimately feed into a robot harvester that will pick all the apples.” And if that was not enough, Koselka added, “Oh, and a pruner is not far off either!”

Even More Potential

The Newton Scout utility will be enhanced in the future with the addition of other possible technologies, including near infrared (NIR). As Newton evolves, it will have multiple uses throughout the fruit growing cycle — for example, assessment of sugar and acid content and internal damage. CASC is also making progress in other automation fronts that bridge with Vision Robotics’ work.

Carnegie Mellon University (CMU) researchers, led by Sanjiv Singh, are developing an Autonomous Prime Mover (APM) capable of towing the Newton Scout safely through entire orchard blocks. In the future, they will also automate an agricultural harvesting platform so that workers no longer need to climb on ladders to pick fruit. The APM’s multifunctionality, autonomous drive, and wireless networking could move it into the center of precision management of specialty crops.

The CASC team, led by CMU, includes engineers, scientists, Extension educators, growers, equipment manufacturers, and industry representatives. This plurality of expertise has led to synergistic endeavors that push the envelope in both the engineering and the science components of the project. For example, Ben Grocholsky at CMU is developing a geospatial system to map and locate each tree in an orchard; in cooperation with the plant science group, he was able to augment this information with local temperature and NDVI data that has the future possibility of being incorporated to detect plant stress and disease.

In another example of the synergy of the CASC team, during a field trial of an automated caliper measurement device, growers told the CASC team that a simpler tree counter would also be very useful. Singh’s group returned to CMU and produced a prototype tree counter ready to be tested in weeks; they will return to nurseries and test the counter this fall. The total time from conception to development will be less than four months, and if successful, this device could go into production within a year.

It will only be a matter of time before specialty crop growers also start using vehicles, sensors, devices, and software that automate farm operations, increase farm efficiency, and meet the ultimate goal of delivering to the consumer a consistently high quality affordable eating experience while returning money to the farming operation.

For more information on Comprehensive Automation for Specialty Crops, please visit www.cascrop.com.

Leave a Reply

Featured Stories
Citrus
September 21, 2017
Hurricane Relief Fund Launched for Florida Farmworkers
Money raised will benefit Redlands Christian Migrant Association’s efforts to aid families in need. Read More
Paul and Derek Orsenigo
GenNext Growers
September 21, 2017
Advice for Taking Over the Family Farm
If you are considering joining and running your family business, here are some tips to ensure success. Read More
Picking fresh Florida blueberries
Berries
September 21, 2017
Ways Florida Blueberry Growers Can Elevate Their Message
Promotional efforts seek to harness power that drives the Sunshine State's ‘little blue dynamos.’ Read More
Inside the Horticulture field tour wagon at the 2016 Florida Ag Expo
Florida Ag Expo
September 21, 2017
Everything You Need to Know About the 2017 Florida Ag Expo
More education choices, flexible field tours, and networking opportunities to provide an enhanced attendee experience. Read More
Whiteflies on a leaf
Insect Control
September 20, 2017
Whitefly Troubles Go From Bad to Worse in Florida
Growers should have a plan for managing the pest from beginning to end of season. Read More
Grower Achievement Award
September 20, 2017
Ask Yourself: What Can I Do to Make Things Better?
Alan Jones, this year's recipient of American Vegetable Grower magazine's Grower Achievement Award, invites all growers to do what he does: Look in the mirror and ask," What can I do to make a positive impact on our industry?" Read More
Bacterial wilt symptoms on blueberry leaves
Berries
September 20, 2017
Getting to the Bottom of Bacterial Wilt in Blueberries
Prolific plant pathogen now presenting a new challenge for Florida producers and researchers. Read More
Citrus
September 19, 2017
Precision in Specialty Crops Gains Momentum
Here are some of the key drivers that may transform your farm in the years ahead. Read More
Fruits
September 19, 2017
EPA Mulls Restricting Dicamba Use
Proposal is a part of agency’s focus on reducing herbicide injury in vulnerable crops. Read More
Fruits
September 19, 2017
Washington State University Creates BMSB Management Resource Page
As populations grow in the state, university offers growers pest management advice. Read More
Berries
September 19, 2017
UC Davis Strawberry Lawuit Settled
Former university breeders to return any crosses that used unpatented varieties and agree the school owns rights to them. Read More
Neo 1 hops at UF/IFAS MREC
Citrus
September 19, 2017
Hopes High for Hops Fresh From Florida
When it comes to alternative crop solutions, something good is brewing in the Sunshine State. Do you have what it takes to tap in? Read More
Apples & Pears
September 16, 2017
Researcher Finds Bacteria Can Stop Blue Mold on Apples
The natural bacterium found in soil can survive in cold storage; this promising biocontrol offers an alternative to chemical treatments. Read More
Laurel wilt-damaged avocado tree leaves
Disease Control
September 15, 2017
Scientist Uncovers Chilling Side of Deadly Avocado Disease
Lab tests indicate laurel wilt pathogen grows faster in the fall and winter. Read More
Winter melon damage from Hurricane Irma in Elkton, FL
Citrus
September 15, 2017
Help Extended to Hurricane-Weary Farmers Dealing With Disaster
USDA chief grants extra time and special procedures to document and file claims. Read More
The Latest
Citrus
September 21, 2017
Hurricane Relief Fund Launched for Flori…
Money raised will benefit Redlands Christian Migrant Association’s efforts to aid families in need. Read More
GenNext Growers
September 21, 2017
Advice for Taking Over the Family Farm
If you are considering joining and running your family business, here are some tips to ensure success. Read More
Berries
September 21, 2017
Ways Florida Blueberry Growers Can Eleva…
Promotional efforts seek to harness power that drives the Sunshine State's ‘little blue dynamos.’ Read More
Florida Ag Expo
September 21, 2017
Everything You Need to Know About the 20…
More education choices, flexible field tours, and networking opportunities to provide an enhanced attendee experience. Read More
Insect Control
September 20, 2017
Whitefly Troubles Go From Bad to Worse i…
Growers should have a plan for managing the pest from beginning to end of season. Read More
Grower Achievement Award
September 20, 2017
Ask Yourself: What Can I Do to Make Thin…
Alan Jones, this year's recipient of American Vegetable Grower magazine's Grower Achievement Award, invites all growers to do what he does: Look in the mirror and ask," What can I do to make a positive impact on our industry?" Read More
Berries
September 20, 2017
Getting to the Bottom of Bacterial Wilt …
Prolific plant pathogen now presenting a new challenge for Florida producers and researchers. Read More
Citrus
September 19, 2017
Precision in Specialty Crops Gains Momen…
Here are some of the key drivers that may transform your farm in the years ahead. Read More
Fruits
September 19, 2017
EPA Mulls Restricting Dicamba Use
Proposal is a part of agency’s focus on reducing herbicide injury in vulnerable crops. Read More
Fruits
September 19, 2017
Washington State University Creates BMSB…
As populations grow in the state, university offers growers pest management advice. Read More
Berries
September 19, 2017
UC Davis Strawberry Lawuit Settled
Former university breeders to return any crosses that used unpatented varieties and agree the school owns rights to them. Read More
Citrus
September 19, 2017
Hopes High for Hops Fresh From Florida
When it comes to alternative crop solutions, something good is brewing in the Sunshine State. Do you have what it takes to tap in? Read More
Apples & Pears
September 16, 2017
Researcher Finds Bacteria Can Stop Blue …
The natural bacterium found in soil can survive in cold storage; this promising biocontrol offers an alternative to chemical treatments. Read More
Disease Control
September 15, 2017
Scientist Uncovers Chilling Side of Dead…
Lab tests indicate laurel wilt pathogen grows faster in the fall and winter. Read More
Citrus
September 15, 2017
Help Extended to Hurricane-Weary Farmers…
USDA chief grants extra time and special procedures to document and file claims. Read More
Citrus
September 13, 2017
New Guidance on FSMA Ag Water Compliance…
FDA issued a rule to extend compliance dates for agricultural water and announced a start date for farm inspections. Read More
Biocontrols Conference
September 13, 2017
Get Better Results From Your Biopesticid…
Softer crop protection programs are becoming a must in specialty crop production. Growers who aren’t learning more about their options Read More
Citrus
September 13, 2017
Florida Farmers Digging out From Impacts…
Damage reports starting to flood in from the field following monster storm, and it's not pretty. Read More