IC-EO
This page presents the project Interpretable, Code-based assistant for Earth Observation (IC-EO) funded by ESA
The objective of the IC-EO project is to propose a conversational assistant that can interact with EO data, such as images. In this project, we will take advantage of the fact that EO tools can be interfaced via code. This will allow us to propose a model that can integrate current state of the art research in an extensible and straightforward manner. The objective of the IC-EO assistant is to be able to answer such questions as: “How much of the field on the left of this area burned down during the fires last summer?”, and “How many buildings are less than 5 minutes walk from the park?”, without restriction to specific formats. Uniquely, the answer to such questions will be code-based and hence can be reviewed and made comprehensible, as opposed to the black-box nature of current approaches. The model can be decomposed in two parts: first, we propose an application programming interface (API) that encompasses elementary tasks, such as classification, object detection, visual grounding and semantic segmentation. Second, we will train a large-language model to use this API, given as a context, to predict a code-based solution to the user’s query.
This work is funded by ESA
