PhenoRob – Robotics and Phenotyping for Sustainable Crop Production (DFG Cluster of Excellence)
One of the greatest challenges for humanity is to produce sufficient food, feed, fiber, and fuel for an ever-growing world population while simultaneously reducing the environmental footprint of agricultural production. x arable land is limited, and the input of agro-chemicals needs to be reduced to curb environmental pollution and halt the decline in biodiversity. Climate change poses additional constraints on crop farming. Achieving sustainable crop production with limited resources is, thus, a task of immense proportions.
Our main hypothesis is that a major shift toward sustainable crop production can be achieved via two approaches: (1) multi-scale monitoring of plants and their environment using autonomous robots with automated and individualized intervention and big data analytics combined with machine learning to improve our understanding of the relation between input and output parameters of crop production, and (2) assessing, modeling, and optimizing the implications of the developed technical innovations in a systemic manner.
To realize our vision, we will take a technology-driven approach to address the challenging scientific objectives. We foresee novel ways of growing crops and managing fields, and aim at reducing the environmental footprint of crop production, maintaining the quality of soil and arable land, and analyzing the best routes to improve the adoption of technology.
The novel approach of PhenoRob is characterized by the integration of robotics, digitalization, and machine learning on one hand, and modern phenotyping, modeling, and crop production on the other. First, we will systematically monitor all essential aspects of crop production using sensor networks as well as ground and aerial robots. This is expected to provide detailed spatially and temporally aligned information at the level of individual plants, nutrient and disease status, soil information as well as ecosystem parameters, such as vegetation diversity. This will enable a more targeted management of inputs (genetic resources, crop protection, fertilization) for optimizing outputs (yield, growth, environmental impact). Second, we will develop novel technologies to enable real-time control of weeds and selective spraying and fertilization of individual plants in field stands. This will help reduce the environmental footprint by reducing chemical input. Third, machine learning applied to crop data will improve our understanding and modeling of plant growth and resource efficiencies and will further assist in the identification of correlations. Furthermore, we will develop integrated multi-scale models for the soil-crop-atmosphere system. These technologies and the gained knowledge will change crop production on all levels. Fourth, in addition to the impact on management decisions at the farm level, we will investigate the requirements for technology adoption as well as socioeconomic and environmental impact of the innovations resulting from upscaling.
Pheno-Inspect (EFRE-funded Start-Up, TRANSFER.NRW)
Accelerating and improving breeding towards more efficient crops and varieties is key for increasing yield and improving resilience of plants. For plant breeders, it is important to observe and document phenotypic traits describing the appearance of plants in the field to evaluate the quality and success of the breeding process. With Pheno-Inspect, we aim at offering growers and farmers novel software solutions for automated high-throughput phenotyping in the fields. For sensing, we reply on small and lightweight aerial platforms, which enable a flexible, large-area and time-efficient survey of fields and plot experiments. With our software toolchain, we provide breeders and farmers with a tool to gain precise knowledge of the crop and individual plants. We automatically detect phenotypic traits of crop plants, the species of plants and weeds in the field and derive site- or plot-specific statistics. Our approach relies on state-of-the-art machine learning methods optimized for the agricultural domain to semantically interpret the captured image data and to extract the desired parameters about the plants. The learning procedures developed by us use expert knowledge inserted by the user to adapt efficiently and thus deliver the desired results quickly and effectively with regard to individual problems and local characteristics of the environment.
DeepUp (EXIST-funded Start-Up, EXIST Host: Dr. Lasse Klingbeil)
DeepUp is an EXIST-funded tech startup that develops novel solutions for geolocation and intelligent data processing.
Escarda Technologies GmbH Spinoff (BMBF, EXISTS)
Applying herbicides is the main means of weed control. The extensive use of chemical substances in agriculture can generate negative effects on our environment and biodiversity. Additionally, several weed varieties have been developing a natural resistance to the applied chemicals, hence new and more potent herbicides have to be developed, which have a higher toxicity with potentially dire consequences for the environment and organisms that come into contact with it. Escarda Technologies is a BMBF-funded EXIST project that aims at combining modern computer vision technology with lasers to develop a chemical-free weeding alternative for our fields. Escarda received funding in 2018 and has turned into a spin-off from the University of Bonn in 2019 with investment from the Berlin Industrial Group. See the Escarda Technologies GmbH website for more information.
MoD – Mapping on Demand (DFG Research Unit)
Sub-project P4 ‘Incremental Mapping from Image Sequences’
Sub-project P8 ‘Exploration for UAVs’
The goal of the project is the development and testing of procedures and algorithms for the fast three-dimensional identification and mensuration of inaccessible objects on the basis of a semantically specified user inquiry. The sensor platform is a lightweight autonomously flying drone. It uses the visual information from cameras for navigation, obstacle detection, exploration and object acquisition. The methods to be develop focus on the autonomous acquisition of probabilistic models to capture spatial-temporal patterns including semantics. The ability to cope with noisy sensor data and to explicitly represent the uncertainty is a central design element. The targeted approaches can be summarized with the term Mapping on Demand. This includes all processes, techniques, and tools to acquire the sensor data, to process and to interpret it with the goal of building a semantically annotated 3D model for the user. The user obtains the model on time and it supports his decision making process. Within the second phase, the research unit focuses on techniques for onboard execution and that are of incremental nature.
Flourish (EC, H2020)
To feed a growing world population with the given amount of available farm land, we must develop new methods of sustainable farming that increase yield while reducing reliance on herbicides and pesticides. Precision agricultural techniques seek to address this challenge by monitoring key indicators of crop health and targeting treatment only to plants that need it. This is a time consuming and expensive activity and while there has been great progress on autonomous farm robots, most systems have been developed to solve only specialized tasks. This lack of flexibility poses a high risk of no return on investment for farmers. The goal of the Flourish project is to bridge the gap between the current and desired capabilities of agricultural robots by developing an adaptable robotic solution for precision farming. By combining the aerial survey capabilities of a small autonomous multi-copter Unmanned Aerial Vehicle (UAV) with a multi-purpose agricultural Unmanned Ground Vehicle, the system will be able to survey a field from the air, perform targeted intervention on the ground, and provide detailed information for decision support, all with minimal user intervention. The system can be adapted to a wide range of crops by choosing different sensors and ground treatment packages. This development requires improvements in technological abilities for safe accurate navigation within farms, coordinated multi-robot mission planning that enables large field survey even with short UAV flight times, multispectral three-dimensional mapping with high temporal and spatial resolution, ground intervention tools and techniques, data analysis tools for crop monitoring and weed detection, and user interface design to support agricultural decision making. As these aspects are addressed in Flourish, the project will unlock new prospects for commercial agricultural robotics in the near future.
RobDREAM (EC, H2020)
Sleep! For hominids and most other mammals sleep means more than regeneration. Sleep positively affects working memory, which in turn improves higher-level cognitive functions such as decision making and reasoning. This is the inspiration of RobDREAM! What if robots could also improve their capabilities in their inactive phases – by processing experiences made during the working day and by exploring – or “dreaming” of – possible future situations and how to solve them best? In RobDREAM we will improve industrial mobile manipulators’ perception, navigation and manipulation and grasping capabilities by automatic optimization of parameters, strategies and selection of tools within a portfolio of key algorithms for perception, navigation and manipulation and grasping, by means of learning and simulation, and through use case driven evaluation. As a result, mobile manipulation systems will adapt more quickly to new tasks, jobs, parts, areas of operation and various other constraints. From a scientific perspective the RobDREAM robots will feature increased adaptability, dependability, flexibility, configurability, decisional autonomy, as well as improved abilities in perception, interaction manipulation and motion. The technology readiness level (TRL) of the related key technologies will be increased by means of frequent and iterative real-world testing, validation and improvement phases from the very beginning of the project.From an economic perspective the Quality of Service and the Overall Equipment Efficiency will increase, while at the same time the Total Cost of Ownership for setup, programming and parameter tuning will decrease. These advantages will support the competitiveness of Europe’s manufacturing sector, in particular in SME-like settings with higher product variety and smaller lot-sizes. They also support the head start of technology providers adopting RobDREAM’s technologies to conquer market shares in industrial and professional service robotics.
Rovina – Robots for Exploration, Digital Preservation and Visualization of Archeological Sites (EC, FP7, 2013-2016)
Mapping and digitizing archeological sites is an important task to preserve cultural heritage and to make it accessible to the public. Current systems for digitizing sites typically build upon static 3D laser scanning technology that is brought into archeological sites by humans. This is acceptable in general, but prevents the digitization of sites that are inaccessible by humans. In the field of robotics, however, there has recently been a tremendous progress in the development of autonomous robots that can access hazardous areas. ROVINA aims at extending this line of research with respect to reliability, accuracy and autonomy to enable the novel application scenario of autonomously mapping of areas of high archeological value that are hardly accessible. ROVINA will develop methods for building accurate, textured 3D models of large sites including annotations and semantic information. To construct the detailed model, it will combine innovative techniques to interpret vision and depth data. ROVINA will furthermore develop advanced techniques for the safe navigation in the cultural heritage site. To actively control the robot, ROVINA will provide interfaces with different levels of robot autonomy. Already during the exploration mission, we will visualize relevant environmental aspects to the end-users so that they can appropriately interact and provide direct feedback.
EUROPA2 (EC, FP7, 2013-2016)
The goal of the EUROPA2 project, which builds on top of the results of the successfully completed FP7 project EUROPA (see below), is to bridge this gap and to develop the foundations for robots designed to autonomously navigate in urban environments outdoors as well as in shopping malls and shops, for example, to provide various services to humans. Based on the combination of publicly available maps and the data gathered with the robot’s sensors, it will acquire, maintain, and revise a detailed model of the environment including semantic information, detect and track moving objects in the environment, adapt its navigation behavior according to the current situation and anticipate interactions with users during navigation. A central aspect in the project is life-long operation and reduced deployment efforts by avoiding to build maps with the robot before it can operate. EUROPA2 is targeted at developing novel technologies that will open new perspectives for commercial applications of service robots in the future.
Previous Projects by C. Stachniss as a PI conducted at the University of Freiburg
- STAMINA – Sustainable and Reliable Robotics for Part Handling in Manufacturing Automation
- AdvancedEDC – Advanced Intracortical Neural Probes with Electronic Depth Control (within EXC-BrainLinks-BrainTools)
- SFB/TR-8 – Spatial Congnition
- TAPAS – Robotics-enabled Logistics and Assistive Services for the Transformable Factory of the Future
- First-MM – Flexible Skill Acquisition and Intuitive Robot Tasking for Mobile Manipulation in the Real World
- EUROPA – European Robotic Pedestrian Assistant
Previous Projects by W. Förster and B. Waske
- Modelling the spatio-temporal variability of crop and cropping system processes under heterogenous field conditions
- Structural-ecolgical mapping of river courses, using TerraSAR-X and RaipdEye data
- Remote sensing based retrieval of biomethane potential (BMP) of crops, with regard to the EnMAP mission
- Monitoring Farmland Abandonment by multitemporal and multisensor remote sensing imagery (MOFA) (transfered to FU Berlin 01.01.2014)
- Semi-automatic Generation of Highly Detailed Textured Building Models
- eTRIMS- E-Training for Interpreting Images of Man-Made Scenes (2006-2009)
- Ontological Scales
- A Control Point Model Database for Automatic Exterior Orientation (together with Survey Department NRW) (1994-2000)
- Semantic Modeling and Extraction of Spatial Objects from Images and Maps (SM) (1993-1999)
- Image Processing for Automatic Carthographic Tools (IMPACT)
- Photogrammetric investigations with MOMS-02 imagery
- Calibration of a fringe projection (structured light) sensor system (in german)
- Automatic Geometric and Semantic Reconstruction of Buildings from Images by Extraction of 3D-Corners and their 3D-Aggregation
- Photogrammetric Eye
- A Generic Adjustment Module
- Photogrammetric Observation and Reconstruction of the Development of a Fluvial Sediment Surface (in german)
- Semi-automatic Building Acquisition
- A Photogrammetric Scanner for the Analytical Plotter P3