Hybrid Human Artificial Collective Intelligence in Open-Ended Decision Making

Digital, Industry & SpaceHORIZON-IAID: 101070588
EC Contribution
€18,773
Consortium Size
5 orgs
Start Year
2022
Summary

HACID develops a novel hybrid collective intelligence for decision support to professionals facing complex open-ended problems, promoting engagement, fairness and trust. A decision support system (HACID-DSS) is proposed that is based on structured domain knowledge, semi-automatically assembled in a domain knowledge graph (DKG) from available data sources, such as scientific and gray literature. Given a specific case within the addressed domain, a pool of experts is consulted to (i) extract supporting evidence and enrich it, generating a case knowledge graph (CKG) as a subset of the DKG, and (ii) provide one or more solutions to the problem. Exploiting the CKG, the HACID-DSS gathers the expert advice in a collective solution that aggregates the individual opinions and expands them with machine-generated suggestions. In this way, HACID harnesses the wisdom of the crowd in open-ended problems, relying on a traceable process based on supporting evidence for better explainability. A set of evaluation methods is proposed to deal with domains where ground truth is not available, demonstrating the suitability of the proposed approach in a wide range of application domains. Demonstrations are provided in two compelling case studies contributing to the UN Sustainable Development Goals: crowd-sourcing medical diagnostics and climate services for urban adaptation.

Consortium (5)

Project Results (28)

Source: CORDIS, the EU research results database.

Publications (16)
Collective Intelligence Increases Diagnostic Accuracy in a General Practice Setting
Medical Decision Making· 2025DOI
Matthew D. Blanchard, Stefan M. Herzog, Juliane E. Kämmer, Nikolas Zöller, Olga Kostopoulou, Ralf H. J. M. Kurvers
Human–AI collectives most accurately diagnose clinical vignettes
Proceedings of the National Academy of Sciences· 2025DOI
Nikolas Zöller, Julian Berger, Irving Lin, Nathan Fu, Jayanth Komarneni, Gioele Barabucci, Kyle Laskowski, Victor Shia, Benjamin Harack, Eugene A. Chu, Vito Trianni, Ralf H. J. M. Kurvers, Stefan M. Herzog
Large Language Models Assisting Ontology Evaluation
Lecture Notes in Computer Science, The Semantic Web – ISWC 2025· 2025DOI
Anna Sofia Lippolis, Mohammad Javad Saeedizade, Robin Keskisärkkä, Aldo Gangemi, Eva Blomqvist, Andrea Giovanni Nuzzolese
Logic Augmented Generation
Journal of Web Semantics· 2025DOI
Aldo Gangemi, Andrea Giovanni Nuzzolese
Ontogenia: Ontology Generation with Metacognitive Prompting in Large Language Models
Lecture Notes in Computer Science ISBN: 9783031789519· 2025DOI
Lippolis, Anna Sofia; Ceriani, Miguel; Zuppiroli, Sara; Nuzzolese, Andrea Giovanni
Ontology Generation Using Large Language Models
Lecture Notes in Computer Science, The Semantic Web· 2025DOI
Anna Sofia Lippolis, Mohammad Javad Saeedizade, Robin Keskisärkkä, Sara Zuppiroli, Miguel Ceriani, Aldo Gangemi, Eva Blomqvist, Andrea Giovanni Nuzzolese
Proceedings of the National Academy of Sciences of the United States of America
Proceedings of the National Academy of Sciences· 2025DOI
Zöller, N; Berger, J; Lin, I; Fu, N; Komarneni, J; Barabucci, G; Laskowski, K; Shia, V; Harack, B; Chu, EA; Trianni, V; Kurvers, RHJM; Herzog, SM
Automating hybrid collective intelligence in open-ended medical diagnostics
Proceedings of the National Academy of Sciences· 2024DOI
Ralf H. J. M. Kurvers, Andrea Giovanni Nuzzolese, Alessandro Russo, Gioele Barabucci, Stefan M. Herzog, Vito Trianni
Combining Multiple Large Language Models Improves Diagnostic Accuracy
NEJM AI· 2024DOI
Gioele Barabucci, Victor Shia, Eugene Chu, Benjamin Harack, Kyle Laskowski, Nathan Fu
Does AI mean we need to do climate services differently?
· 2024DOI
Fai Fung, Neha Mittal
How large language models can reshape collective intelligence
Nature Human Behaviour· 2024DOI
Jason W. Burton; Ezequiel Lopez-Lopez; Shahar Hechtlinger; Zoe Rahwan; Samuel Aeschbach; Michiel A. Bakker; Joshua A. Becker; Aleks Berditchevskaia; Julian Berger; Levin Brinkmann; Lucie Flek; Stefan M. Herzog; Saffron Huang; Sayash Kapoor; Arvind Narayanan; Anne-Marie Nussberger; Taha Yasseri; Pietro Nickl; Abdullah Almaatouq; Ulrike Hahn; Ralf H. J. M. Kurvers; Susan Leavy; Iyad Rahwan; Divya Siddarth; Alice Siu; Anita W. Woolley; Dirk U. Wulff; Ralph Hertwig
Boosting collective intelligence in medical diagnostics: Leveraging decision similarity as a predictor of accuracy when answers are open-ended rankings
HCOMP-CI 2023 Works-in-Progress and Demonstrations· 2023
Nikolas Zöller, Stefan M. Herzog, RalfH.J.M. Kurvers
Expertise Trees Resolve Knowledge Limitations in Collective Decision-Making
Proceedings of the 40th International Conference on Machine Learning· 2023
Abels A.; Lenaerts T.; Trianni V.; Nowe A.
Hybrid Collective Intelligence for Decision Support in Complex Open-Ended Domains
Frontiers in Artificial Intelligence and Applications· 2023DOI
Vito Trianni; Andrea Giovanni Nuzzolese; Jaron Porciello; Ralf H. J. M. Kurvers; Stefan M. Herzog; Gioele Barabucci; Aleksandra Berditchevskaia; Fai Fung
Proceedings of the National Academy of Sciences of the United States of America
Proceedings of the National Academy of Sciences of the United States of America· 2023DOI
Ralf H. J. M. Kurvers; Andrea Giovanni Nuzzolese; Alessandro Russo; Gioele Barabucci; Stefan M. Herzog; Vito Trianni
Assessing the Capability of Large Language Models for Domain-Specific Ontology Generation
Anna Sofia Lippolis, Mohammad Javad Saeedizade, Robin Keskisarkka, Aldo Gangemi, Eva Blomqvist, Andrea Giovanni Nuzzolese
Deliverables (11)
Other Results (1)
Periodic Reporting for period 1 - HACID (Hybrid Human Artificial Collective Intelligence in Open-Ended Decision Making)