Wang, Fuli and Cuan Urquizo, Rodolfo and Roberts, Penelope and Mohan, Vishwanathan and Newenham, Chris and Ivanov, Andrey and Dowling, Robin (2023) Biologically inspired robotic perception-action for soft fruit harvesting in vertical growing environments. Precision Agriculture, 24 (3). pp. 1072-1096. DOI https://doi.org/10.1007/s11119-023-10000-4
Wang, Fuli and Cuan Urquizo, Rodolfo and Roberts, Penelope and Mohan, Vishwanathan and Newenham, Chris and Ivanov, Andrey and Dowling, Robin (2023) Biologically inspired robotic perception-action for soft fruit harvesting in vertical growing environments. Precision Agriculture, 24 (3). pp. 1072-1096. DOI https://doi.org/10.1007/s11119-023-10000-4
Wang, Fuli and Cuan Urquizo, Rodolfo and Roberts, Penelope and Mohan, Vishwanathan and Newenham, Chris and Ivanov, Andrey and Dowling, Robin (2023) Biologically inspired robotic perception-action for soft fruit harvesting in vertical growing environments. Precision Agriculture, 24 (3). pp. 1072-1096. DOI https://doi.org/10.1007/s11119-023-10000-4
Abstract
Multiple interlinked factors like demographics, migration patterns, and economics are presently leading to the critical shortage of labour available for low-skilled, physically demanding tasks like soft fruit harvesting. This paper presents a biomimetic robotic solution covering the full ‘Perception-Action’ loop targeting harvesting of strawberries in a state-of-the-art vertical growing environment. The novelty emerges from both dealing with crop/environment variance as well as configuring the robot action system to deal with a range of runtime task constraints. Unlike the commonly used deep neural networks, the proposed perception system uses conditional Generative Adversarial Networks to identify the ripe fruit using synthetic data. The network can effectively train the synthetic data using the image-to-image translation concept, thereby avoiding the tedious work of collecting and labelling the real dataset. Once the harvest-ready fruit is localised using point cloud data generated by a stereo camera, our platform’s action system can coordinate the arm to reach/cut the stem using the Passive Motion Paradigm framework inspired by studies on neural control of movement in the brain. Results from field trials for strawberry detection, reaching/cutting the stem of the fruit, and extension to analysing complex canopy structures/bimanual coordination (searching/picking) are presented. While this article focuses on strawberry harvesting, ongoing research towards adaptation of the architecture to other crops such as tomatoes and sweet peppers is briefly described.
Item Type: | Article |
---|---|
Uncontrolled Keywords: | Soft fruit harvesting; Generative adversarial networks; Crop detection/localization; Dexterous manipulation |
Divisions: | Faculty of Science and Health Faculty of Science and Health > Computer Science and Electronic Engineering, School of |
SWORD Depositor: | Unnamed user with email elements@essex.ac.uk |
Depositing User: | Unnamed user with email elements@essex.ac.uk |
Date Deposited: | 06 Jun 2023 13:31 |
Last Modified: | 07 Aug 2024 20:14 |
URI: | http://repository.essex.ac.uk/id/eprint/35218 |
Available files
Filename: s11119-023-10000-4.pdf
Licence: Creative Commons: Attribution 4.0