Apple Harvesters Use Vision To “See” Fruit

Editor’s Note: This past October, Tony Koselka, Jillian Cannons, and Ryan Carlon of Vision Robotics Corporation (VRC) returned to Washington for further field trials of the Newton Scout, a vision-based system for automatic crop load assessment. The following report highlights their results.

Developed in part through the USDA-funded project Comprehensive Automation for Specialty Crops (CASC), the Newton Scout will revolutionize orchard operations by providing growers with a map that accurately maps fruit placement, fruit size, and number throughout the growing season. Fruit maps will be used to determine crop load, which in turn will give growers the information to develop harvest and marketing strategies. And this is just the beginning. Developing reliable vision systems that are robust enough for use in orchards opens up all kinds of possibilities in the area of pest and disease detection, plant health assessment, and the use of robotics at harvest.

The Newton Scout integrates stereo cameras with a custom image processing algorithm. The algorithms use classifiers to identify the apples, where the simplest classifier uses color and shape. For red apples, color is the primary classifier. For detecting green fruit, the Scout detects using statistical classifiers generated by boosting algorithms. The most challenging aspects of detecting fruit are occlusions, shadows, shape and color coincidences, and ambient lighting conditions such as bright sunlight. Once an apple is identified, the stereo cameras enable the system to determine its location and size.

The Vision Robotics field trial was conducted at Allan Brothers Othello Orchard in Washington. Prior to VRC’s arrival, Washington State University (WSU) Extension Educator Karen Lewis, and interns with the Washington Tree Fruit Research Commission, hand tagged, mapped, and sized fruit on several rows of fruit. This data allows for the groundtruthing or validation component of the test. Koselka took time out to speak about the development of a similar unit that counts oranges with the interns.

On one of the field trials for that unit, Koselka asked the grower to estimate his crop. The farm manager responded with 1,200 bushels. For this test, Vision Robotics scouted and counted 1,900 — a sizable difference and one that would have a large financial impact.

As Koselka stated, “not only will it [Scout] be able to accurately count a crop and locate each piece of fruit, it will ultimately feed into a robot harvester that will pick all the apples.” And if that was not enough, Koselka added, “Oh, and a pruner is not far off either!”

Even More Potential

The Newton Scout utility will be enhanced in the future with the addition of other possible technologies, including near infrared (NIR). As Newton evolves, it will have multiple uses throughout the fruit growing cycle — for example, assessment of sugar and acid content and internal damage. CASC is also making progress in other automation fronts that bridge with Vision Robotics’ work.

Carnegie Mellon University (CMU) researchers, led by Sanjiv Singh, are developing an Autonomous Prime Mover (APM) capable of towing the Newton Scout safely through entire orchard blocks. In the future, they will also automate an agricultural harvesting platform so that workers no longer need to climb on ladders to pick fruit. The APM’s multifunctionality, autonomous drive, and wireless networking could move it into the center of precision management of specialty crops.

The CASC team, led by CMU, includes engineers, scientists, Extension educators, growers, equipment manufacturers, and industry representatives. This plurality of expertise has led to synergistic endeavors that push the envelope in both the engineering and the science components of the project. For example, Ben Grocholsky at CMU is developing a geospatial system to map and locate each tree in an orchard; in cooperation with the plant science group, he was able to augment this information with local temperature and NDVI data that has the future possibility of being incorporated to detect plant stress and disease.

In another example of the synergy of the CASC team, during a field trial of an automated caliper measurement device, growers told the CASC team that a simpler tree counter would also be very useful. Singh’s group returned to CMU and produced a prototype tree counter ready to be tested in weeks; they will return to nurseries and test the counter this fall. The total time from conception to development will be less than four months, and if successful, this device could go into production within a year.

It will only be a matter of time before specialty crop growers also start using vehicles, sensors, devices, and software that automate farm operations, increase farm efficiency, and meet the ultimate goal of delivering to the consumer a consistently high quality affordable eating experience while returning money to the farming operation.

For more information on Comprehensive Automation for Specialty Crops, please visit www.cascrop.com.

Leave a Reply

Featured Stories
Agritainment
CitrusAgritourism Gaining Lots Of Ground In Sunshine State
July 6, 2015
Researchers report more farming operations are growing revenue by hosting festivals, tours, wine tastings, and more. Read More
Photo of New Guinea flatworm found near Coral Gables, FL
CitrusSouth Florida’s Newest Pest Has Significant Ooze Factor
July 2, 2015
The infamous New Guinea flatworm, one of the ‘world's worst invasive alien species,’ might have found a new home. Read More
Farm ManagementPMA Advocates For Improvements To Proposed FSMA Mandatory Food Recall Guidance
July 2, 2015
This week the Produce Marketing Association (PMA) submitted comments on behalf of its membership to FDA regarding the proposed Mandatory Read More
VegetablesUnited Fresh Leadership Program’s Class 20 Donates 28 Salad Bars To Schools
July 2, 2015
Over the past year, the twelve members of United Fresh Produce Industry Leadership Program’s Class 20 raised funds to support Read More
CitrusFlorida’s Young Farmers Working To Bridge Generation Gap
July 2, 2015
Leadership conference to give GenNext Growers tools to become better agvocates. Read More
ProductionUse Broccoli Rotations To Reduce Disease And Minimize Leaching
July 1, 2015
Crop rotations are a key production practice utilized by every grower. You recognize the importance of rotations to break insect Read More
CitrusIndustry Stakeholders Rip Report Linking Citrus Consumption With Skin Cancer
July 1, 2015
Associations react swiftly; call out flaws in study released by the Journal of Clinical Oncology. Read More
Cucurbits11 Pumpkin And Squash Varieties
July 1, 2015
This month’s variety specs feature focuses on pumpkin and squash varieties. Features highlighted include great disease resistance packages, excellent yield potential, Read More
VegetablesInterview With Hazera Global Onion Product Manager
July 1, 2015
Onion is one of the most produced vegetable crops in the world. A lot of expertise is required to develop Read More
EquipmentHarvest Trends: More Mechanization Needed For Vegetable Growers
July 1, 2015
As the labor supply declines, industry experts and growers say investing in the latest harvesting technology will be necessary. Read More
The Latest
Agritainment
CitrusAgritourism Gaining Lots Of Ground In Sunshine State
July 6, 2015
Researchers report more farming operations are growing revenue by hosting festivals, tours, wine tastings, and more. Read More
Photo of New Guinea flatworm found near Coral Gables, FL
CitrusSouth Florida’s Newest Pest Has Significant Ooze Factor
July 2, 2015
The infamous New Guinea flatworm, one of the ‘world's worst invasive alien species,’ might have found a new home. Read More
Farm ManagementPMA Advocates For Improvements To Proposed FSMA Mandato…
July 2, 2015
This week the Produce Marketing Association (PMA) submitted comments on behalf of its membership to FDA regarding the proposed Mandatory Read More
VegetablesUnited Fresh Leadership Program’s Class 20 Donates 28 S…
July 2, 2015
Over the past year, the twelve members of United Fresh Produce Industry Leadership Program’s Class 20 raised funds to support Read More
CitrusFlorida’s Young Farmers Working To Bridge Generation Ga…
July 2, 2015
Leadership conference to give GenNext Growers tools to become better agvocates. Read More
ProductionUse Broccoli Rotations To Reduce Disease And Minimize L…
July 1, 2015
Crop rotations are a key production practice utilized by every grower. You recognize the importance of rotations to break insect Read More
CitrusIndustry Stakeholders Rip Report Linking Citrus Consump…
July 1, 2015
Associations react swiftly; call out flaws in study released by the Journal of Clinical Oncology. Read More
Cucurbits11 Pumpkin And Squash Varieties
July 1, 2015
This month’s variety specs feature focuses on pumpkin and squash varieties. Features highlighted include great disease resistance packages, excellent yield potential, Read More
VegetablesInterview With Hazera Global Onion Product Manager
July 1, 2015
Onion is one of the most produced vegetable crops in the world. A lot of expertise is required to develop Read More
EquipmentHarvest Trends: More Mechanization Needed For Vegetable…
July 1, 2015
As the labor supply declines, industry experts and growers say investing in the latest harvesting technology will be necessary. Read More
healthy piece of citrus
Citrus Achievement AwardNeglected Research, Future Growers Top Of Mind For 2015…
July 1, 2015
Orie Lee advises young up-and-comers to supplement production knowledge with business acumen; calls for more focus on blight. Read More
Business PlanningTake Climate Change — Please [Opinion]
June 30, 2015
It’s a difficult issue because people tend to think of it in terms of something tangible — like the weather. Read More
NutsWalnut Nitrogen Budgeting Critical
June 30, 2015
Applying the right amount of N is not easy because walnuts take a lot, but these days you don’t want to apply too much. Read More
FruitsHighlights From The Ohio Produce Growers And Marketers …
June 30, 2015
American Vegetable Grower magazine made its way to the Ohio Produce Growers and Marketers Association (OPGMA) Summer Tour & Field Read More
Insect ControlAlabama Insect Scouting Report Indicates Armyworm Numbe…
June 30, 2015
Report discusses caterpillars, sucking pests, and cowpea curculio activity and management. Read More
Nutrient ManagementResearchers Say Cover Crops May Help Save Soil In Flood…
June 30, 2015
According to experts, planting cover crops may help preserve healthy soil in fields in fields left empty this year thanks to flooding. Read More
TomatoesGSPP Accreditation Enables Sakata Seed America To Offer…
June 30, 2015
The seed company is now authorized to store, treat, process, package, and distribute GSPP-certified tomato seed for the NAFTA market. Read More
MarketingThe Power Of Local Food [Opinion]
June 30, 2015
Summer is now in full swing and we have entered what I like to call “the season of farmers markets.” Read More