Robotic Harvesting: Challenges and Emerging Solutions

Robotic harvesters have appeared in fields, orchards and greenhouses. By reading this blog article, discover the main challenges they are facing.

Agriculture Agri-food
Donald Prévost
Date  December 2023

The agricultural industry has faced a shortage of skilled labour and rising labour costs for the past several years. The industry has automated many of its operations to optimize the use of available resources, including harvesting. In this context, autonomous or semi-autonomous mechanical harvesters have taken the fields by storm.

However, in vegetable production, many crops are still harvested by hand. This is particularly true for lettuce, cauliflower, broccoli, peppers, eggplants, asparagus, cucumbers, and tomatoes. Harvesting these crops requires significant precision. Fruit and vegetables are often fragile and require handling with care to avoid damage, which directly affects farmers’ incomes. A specialized workforce usually harvests these crops manually by locating mature fruits and vegetables under the foliage and harvesting them over a short period for optimum yield.

Robotic harvesters have appeared in fields, orchards and greenhouses alike to meet these specific challenges.

The challenges of automated harvesting

The main challenges facing harvesting robots include:

  • Identifying mature fruits and vegetables through foliage and in all environmental conditions (rain, low light, mud, dust, etc.);
  • Characterizing and positioning fruits and vegetables in space to guide robotic arms for gripping and cutting;
  • Selective or non-selective cutting or picking of fruit and vegetables;
  • Handling ripe fruits and vegetables without damaging them;
  • Moving around fields, orchards or greenhouses without damaging crops or infrastructure.

What distinguishes harvesting robots from mechanized harvesters is the addition of several technologies, such as cameras, sensors, robotic arms and the use of artificial intelligence algorithms. Discover the main optical sensors used in agriculture by reading this blog article.

Many agricultural machinery integrators and equipment manufacturers are taking the plunge and working to integrate vision systems and robotic arms into their farm equipment to turn it into harvesting robots.

Robots that harvest like experienced pickers

Robotic harvesters are autonomous machines designed to harvest fruits and vegetables in fields, orchards and greenhouses. They are equipped with a vision system consisting of optical sensors and cameras, which detect fruits and vegetables partially or almost completely hidden under foliage and evaluate their position, size, volume, shape and colour, and determine the optimal time for harvesting regardless of the environmental conditions. Optical vision systems are integrated into the eye of the robot and can be installed on motorized vehicles equipped with robotic arms fitted with grippers and cutting systems.

A broccoli-picking robot in the field

To meet the needs of equipment manufacturers and farmers, INO developed AxomAcu, an intelligent and versatile vision module used for robotic harvesting and precision agriculture. This solution can detect and track the position and size of fruit or vegetables and guide the harvesting system in real-time. Contact us to find out more about the AxomAcu solution.

Univerco developed equipment to automate broccoli harvesting in the field using AxomAcu. Univerco’s challenge was integrating advanced technology into its equipment to harvest several rows at a time and detect broccoli heads.

Detecting broccoli in the field is no easy task. Size, shape, and volume vary from one head to another. In this context, the systems and algorithms that detect these vegetables must consider a wide spectrum of specifications. In the field, foliage often hides broccoli, allowing only a partial view of the vegetable. Therefore, vision systems and AI algorithms need to spot this hidden broccoli or detect clues to their presence to guide the robotic arms to the broccoli no matter how high they grow.

A cucumber-picking robot for greenhouses

In Canadian greenhouses, it is common to grow cucumbers using the high-wire method. In other words, the cucumbers grow in a vertical, hanging configuration. Visibility of cucumbers on the vine is a challenge for the harvesting robot, as it must detect and differentiate between a cucumber, a vine, a leaf and a tendril of the cucumber plant.

It can be difficult to properly locate cucumbers on the vine because they grow in different locations and at different heights along the row, with some located further back in the foliage and others toward the front. The harvesting robot must locate the cucumber, move through the vines to pick it and make sure that, as it pulls the cucumber, it does not snag a vine and knock over the row of plants.

To meet the challenges, INO and Vineland Research and Innovation Centre developed an autonomous cucumber harvesting robot. Learn more by reading the case study A Partnership to Solve Challenges in the Agri-Food Industry.


In conclusion, harvesting automation and robotization using vision systems equipped with artificial intelligence help meet the challenges facing the agricultural industry and contribute to increasing productivity and profitability for farmers. Using robotic harvesters in fields, orchards and greenhouses not only reduces labour costs, but also increases the quality and quantity of harvested crops and improves crop management by notably reducing crop losses.

About the author

Donald Prévost

Solutions Manager

Donald Prévost, PhD, graduated from Laval University with a master degree in physics and from the Paris-Sud University with a PhD in imaging physics. 

He is a physicist with 25 years of experience in applied research and research management in the industry. Throughout his career, he has been leading several custom vision system and video analytics developments for clients in manufacturing, surveillance, aerospace, transportation, low-vision and agro-food industries.  Dr. Prévost has been co-directing three doctoral-level and five Master’s level graduated students, and is currently associate professor at Laval University.

View LinkedIn Profile

Subscribe to the blog

Stay tuned for our latest articles.