European Lighthouse on Secure and Safe AI
Digital, Industry & SpaceHORIZON-RIAID: 101070617
EC Contribution
€74,341
Consortium Size
26 orgs
Start Year
2022
▶Summary
In order to reinforce European leadership in safe and secure AI technology, we are proposing a virtual center of excellence on safe and secure AI that will address major challenges hampering the deployment of AI technology. These grand challenges are fundamental in nature. Addressing them in a sustainable manner requires a lighthouse rooted in scientific excellence and rigorous methods. We will develop a strategic research agenda which is supported by research programmes that focus on “technical robustness and safety”, “privacy preserving techniques and infrastructures” and “human agency and oversight”. Furthermore, we focus our efforts to detect, prevent and mitigate threats and enable recovery from harm by 3 grand challenges: “Robustness guarantees and certification”, “Private and robust collaborative learning at scale” and “Human-in-the-loop decision making: Integrated governance to ensure meaningful oversight” that cut across 6 use cases: health, autonomous driving, robotics, cybersecurity, multi-media, and document intelligence. Throughout our project, we seek to integrate robust technical approaches with legal and ethical principles supported by meaningful and effective governance architectures to nurture and sustain the development and deployment of AI technology that serves and promotes foundational European values. Our initiative builds on and expands the internationally recognized, highly successful and fully operational network of excellence ELLIS (European Laboratory for Learning and Intelligent Systems). We build ELSA on its 3 pillars: research programmes, a set of research units, and a PhD/postdoc programme, thereby connecting a network of over 100 organizations and more than 337 ELLIS fellows and scholars (113 ERC grants) committed to shared standards of excellence. We will not only establish a virtual center of excellence, but all our activities will be also inclusive and open to input, interactions and collaboration of AI researchers and industrial partners in order to drive the entire field forward.
Consortium (26)
🇩🇪 CISPA - HELMHOLTZ-ZENTRUM FUR INFORMATIONSSICHERHEIT GGMBHDE
coordinator🇪🇸 CENTRE DE VISIO PER COMPUTADORES
partner🇮🇹 CONSORZIO INTERUNIVERSITARIO NAZIONALE PER L'INFORMATICAIT
partner🇨🇭 ECOLE POLYTECHNIQUE FEDERALE DE LAUSANNECH
partner🇨🇭 EIDGENOESSISCHE TECHNISCHE HOCHSCHULE ZUERICHCH
partner🇩🇪 EUROPEAN MOLECULAR BIOLOGY LABORATORYDE
partner🇮🇹 FONDAZIONE ISTITUTO ITALIANO DI TECNOLOGIAIT
partner🇫🇮 HELSINGIN YLIOPISTOFI
partner🇫🇷 INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET AUTOMATIQUEFR
partner🇸🇪 KUNGLIGA TEKNISKA HOEGSKOLANSE
partner🇮🇹 LEONARDO - SOCIETA PER AZIONIIT
partner MAX-PLANCK-GESELLSCHAFT ZUR FORDERUNG DER WISSENSCHAFTEN EV
partner🇨🇭 NVIDIA SWITZERLAND AGCH
partner🇪🇸 PAL ROBOTICS SLUES
partner🇮🇹 PLURIBUS ONE SRLIT
partner🇮🇹 POLITECNICO DI TORINOIT
partner🇬🇧 THE ALAN TURING INSTITUTEGB
partner🇬🇧 THE CHANCELLOR, MASTERS AND SCHOLARS OF THE UNIVERSITY OF OXFORDGB
partner🇬🇧 THE UNIVERSITY OF BIRMINGHAMGB
partner🇮🇹 UNIVERSITA DEGLI STUDI DI CAGLIARIIT
partner🇮🇹 UNIVERSITA DEGLI STUDI DI GENOVAIT
partner🇮🇹 UNIVERSITA DEGLI STUDI DI MILANOIT
partner🇮🇹 UNIVERSITA DEGLI STUDI DI MODENA E REGGIO EMILIAIT
partner🇬🇧 UNIVERSITY OF LANCASTERGB
partner🇫🇷 VALEO COMFORT AND DRIVING ASSISTANCEFR
partner🇫🇷 YOOZFR
partnerProject Results (200)
Source: CORDIS, the EU research results database.
▶Publications (194)
A Bias-Variance Decomposition for Ensembles over Multiple Synthetic Datasets
Ossi Räisä, Antti Honkela
Adversarial pruning: A survey and benchmark of pruning methods for adversarial robustness
Giorgio Piras, Maura Pintor, Ambra Demontis, Battista Biggio, Giorgio Giacinto, Fabio Roli
Algorithmic loafing and mitigation strategies in Human-AI teams
Isa Inuwa-Dutse, Alice Toniolo, Adrian Weller, Umang Bhatt
AttackBench: Evaluating Gradient-based Attacks for Adversarial Examples
Antonio Emanuele Cinà, Jérôme Rony, Maura Pintor, Luca Demetrio, Ambra Demontis, Battista Biggio, Ismail Ben Ayed, Fabio Roli
Certified Robustness of Static Deep Learning-based Malware Detectors against Patch and Append Attacks
Daniel Gibert, Giulio Zizzo, Quan Le
Data Drift in Android Malware Detection
Luca Minnei, Hicham Eddoubi, Angelo Sotgiu, Maura Pintor, Ambra Demontis, Battista Biggio
Fairness Without Demographic Data: A Survey of Approaches
Carolyn Ashurst, Adrian Weller
FLoRA: Sample-Efficient Preference-based RL via Low-Rank Style Adaptation of Reward Functions
2025 International Conference on Robotics and Automation· 2025
Daniel Marta, Simon Holk, Miguel Vasco, Jens Lundell, Timon Homberger, Finn Busch, Olov Andersson, Danica Kragic, Iolanda Leite
Foundation Models and Fine-Tuning: A Benchmark for Out of Distribution Detection
Francesco Cappio Borlino, Lorenzo Lu, Tatiana Tommasi
From ‘wild west’ to ‘responsible’ AI testing ‘in-the-wild’: lessons from live facial recognition testing by law enforcement authorities in Europe
Karen Yeung, Wenlong Li
Harms from Increasingly Agentic Algorithmic Systems
Alan Chan, Rebecca Salganik, Alva Markelius, Chris Pang, Nitarshan Rajkumar, Dmitrii Krasheninnikov, Lauro Langosco, Zhonghao He, Yawen Duan, Micah Carroll, Michelle Lin, Alex Mayhew, Katherine Collins, Maryam Molamohammadi, John Burden, Wanru Zhao, Shalaleh Rismani, Konstantinos Voudouris, Umang Bhatt, Adrian Weller, David Krueger, Tegan Maharaj
Human-centered AI Technologies in Human-robot Interaction for Social Settings
Yuchong Zhang, Khaled Kassem, Zhengya Gong, Fan Mo, Yong Ma, Emma Kirjavainen, Jonna Häkkilä
Hyperparameters in Score-Based Membership Inference Attacks
Gauri Pradhan, Joonas Jälkö, Marlon Tobaben, Antti Honkela
IDEAL: Interpretable-by-Design ALgorithms for learning from foundation feature spaces
Plamen Angelov, Dmitry Kangin, Ziyang Zhang
IMAFD: An Interpretable Multi-stage Approach to Flood Detection from time series Multispectral Data
Ziyang Zhang, Plamen Angelov, Dmitry Kangin, Nicolas Longépé
Imitation or Innovation? Translating Features of Expressive Motion from Humans to Robots
Benedikte Wallace, Marieke van Otterdijk, Yuchong Zhang, Nona Rajabi, Diego Marin-Bucio, Danica Kragic, Jim Torresen
Informed Machine Learning: Excess risk and generalization
Luca Oneto, Sandro Ridella, Davide Anguita
Learning to mask and permute visual tokens for Vision Transformer pre-training
Lorenzo Baraldi, Roberto Amoroso, Marcella Cornia, Lorenzo Baraldi, Andrea Pilzer, Rita Cucchiara
LFPD: Local-Feature-Powered Defense Against Adaptive Backdoor Attacks
Wei Guo, Ambra Demontis, Maura Plntor, Patrick P.K. Chan, Battista Biggio
Media Coverage of Predictive Policing: Bias, Police Engagement, and the Future of Transparency
Harry Camilleri, Carolyn Ashurst, Nithya Jaisankar, Adrian Weller, Miri Zilka
Mitigating Unfair Regression in Machine Learning Model Updates
Irene Buselli, Anna Pallarès López, Eduard Martín Jiménez, Davide Anguita, Fabio Roli, Luca Oneto
Modeling Brain Aging With Explainable Triamese ViT: Towards Deeper Insights Into Autism Disorder
Zhaonian Zhang, Vaneet Aggarwal, Plamen Angelov, Richard Jiang
ModSec-AdvLearn: Countering Adversarial SQL Injections With Robust Machine Learning
Giuseppe Floris, Christian Scano, Biagio Montaruli, Luca Demetrio, Andrea Valenza, Luca Compagna, Davide Ariu, Luca Piras, Davide Balzarotti, Battista Biggio
ModSec-Learn: Boosting ModSecurity with Machine Learning
Lecture Notes in Networks and Systems, Distributed Computing and Artificial Intelligence, Special Sessions I, 21st International Conference· 2025DOI
Christian Scano, Giuseppe Floris, Biagio Montaruli, Luca Demetrio, Andrea Valenza, Luca Compagna, Davide Ariu, Luca Piras, Davide Balzarotti, Battista Biggio
Nebula: Self-Attention for Dynamic Malware Analysis
Dmitrijs Trizna, Luca Demetrio, Battista Biggio, Fabio Roli
Noise-Aware Differentially Private Variational Inference
Talal Alrawajfeh, Joonas Jälkö, Antti Honkela
On the robustness of adversarial training against uncertainty attacks
Emanuele Ledda, Giovanni Scodeller, Daniele Angioni, Giorgio Piras, Antonio Emanuele Cinà, Giorgio Fumera, Battista Biggio, Fabio Roli
Poster: Protection against Source Inference Attacks in Federated Learning using Unary Encoding and Shuffling
Andreas Athanasiou, Kangsoo Jung, Catuscia Palamidessi
Robustness-Congruent Adversarial Training for Secure Machine Learning Model Updates
Daniele Angioni, Luca Demetrio, Maura Pintor, Luca Oneto, Davide Anguita, Battista Biggio, Fabio Roli
Runtime Backdoor Detection for Federated Learning via Representational Dissimilarity Analysis
Xiyue Zhang, Xiaoyong Xue, Xiaoning Du, Xiaofei Xie, Yang Liu, Meng Sun
TransferBench: Benchmarking Ensemble-based Black-box Transfer Attacks
The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track· 2025
Brau, Fabio; Pintor, Maura; Cinà, Antonio Emanuele; Mura, Raffaele; Scionis, Luca; Oneto, Luca; Roli, Fabio; Biggio,Battista
Vision-Based Landing Guidance Through Tracking and Orientation Estimation
João P. K. Ferreira, João P. Pinto, Júlia Moura, Yi Li, Cristiano L. Castro, Plamen Angelov
Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning
Antonio Emanuele Cinà, Kathrin Grosse, Ambra Demontis, Sebastiano Vascon, Werner Zellinger, Bernhard A. Moser, Alina Oprea, Battista Biggio, Marcello Pelillo, Fabio Roli
"""Reliability in Semantic Segmentation: Can We Use Synthetic Data? """
Thibaut Loiseau, Tuan-Hung Vu, Mickael Chen, Patrick Pérez, Matthieu Cord
A Simple Recipe for Language-guided Domain Generalized Segmentation
Mohammad Fahes, Tuan-Hung Vu, Andrei Bursuc, Patrick Pérez, Raoul de Charette
Adapt to Scarcity: Few-Shot Deepfake Detection via Low-Rank Adaptation
Silvia Cappelletti, Lorenzo Baraldi, Federico Cocchi, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara
Building machines that learn and think with people
Katherine M. Collins, Ilia Sucholutsky, Umang Bhatt, Kartik Chandra, Lionel Wong, Mina Lee, Cedegao E. Zhang, Tan Zhi-Xuan, Mark Ho, Vikash Mansinghka, Adrian Weller, Joshua B. Tenenbaum, Thomas L. Griffiths
Collaborative learning from distributed data with differentially private synthetic data
Lukas Prediger, Joonas Jälkö, Antti Honkela, Samuel Kaski
Contrasting Deepfakes Diffusion via Contrastive Learning and Global-Local Similarities
Lorenzo Baraldi, Federico Cocchi, Marcella Cornia, Lorenzo Baraldi, Alessandro Nicolosi, Rita Cucchiara
Cooperative online learning with feedback graphs
Transactions on Machine Learning Research (06/2024)· 2024
Nicolò Cesa-Bianchi, Tommaso Cesari, and Riccardo Della Vecchia
Dataset and Lessons Learned from the 2024 SaTML LLM Capture-the-Flag Competition
· 2024DOI
Edoardo Debenedetti; Javier Rando; Daniel Paleka; Fineas Silaghi; Dragos Albastroiu; Niv Cohen; Yuval Lemberg; Reshmi Ghosh; Rui Wen; Ahmed Salem; Giovanni Cherubin;
Delve Into Neural Activations: Toward Understanding Dying Neurons
Ziping Jiang, Yunpeng Wang, Chang-Tsun Li, Plamen Angelov, Richard Jiang
EarthMatch: Iterative Coregistration for Fine-grained Localization of Astronaut Photography
Gabriele Berton, Gabriele Goletto, Gabriele Trivigno, Alex Stoken, Barbara Caputo, Carlo Masone
Edge Implementation of Unsupervised Self-evolving Vision Classifier
P. Angelov, A. Aghasanli
Fairness Meets Cross-Domain Learning: A Benchmark of Models and Metrics
Leonardo Iurada; Silvia Bucci; Timothy M. Hospedales; Tatiana Tommasi
FAST: Boosting Uncertainty-based Test Prioritization Methods for Neural Networks via Feature Selection (CWZS24)
Jialuo Chen, Jingyi Wang, Xiyue Zhang, Youcheng Sun, Marta Kwiatkowska, Jiming Chen, Peng Cheng
Finding Lottery Tickets in Vision Models via Data-driven Spectral Foresight Pruning
IEEE CVPR 2024· 2024
Leonardo Iurada, Marco Ciccone, Tatiana Tommasi
Investigating over-parameterized randomized graph networks
Giovanni Donghi, Luca Pasa, Luca Oneto, Claudio Gallicchio, Alessio Micheli, Davide Anguita, Alessandro Sperduti, Nicolò Navarin
Lecture Notes in Computer Science
TACAS 2024, 30th International Conference on Tools and Algorithms for the Construction and Analysis of Systems· 2024DOI
Xiyue Zhang, Benjie Wang, Marta Kwiatkowska
Living-off-The-Land Reverse-Shell Detection by Informed Data Augmentation
· 2024DOI
Trizna, D., Demetrio, L., Biggio, B., & Roli, F.
Make Me a BNN: A Simple Strategy for Estimating Bayesian Uncertainty from Pre-trained Models
Gianni Franchi, Olivier Laurent, Maxence Leguéry, Andrei Bursuc, Andrea Pilzer, Angela Yao
Mask2Anomaly: Mask Transformer for Universal Open-Set Segmentation
Shyam Nandan Rai, Fabio Cermelli, Barbara Caputo, Carlo Masone
Neuron Activation Pattern and Applications
Z. Jiang, P. Angelov, D. Kangin, …
Nonsmooth Implicit Differentiation: Deterministic and Stochastic Convergence Rates
Riccardo Grazzi; Saverio Salzo; Massimiliano Pontil
On Neuron Activation Pattern and Applications
· 2024DOI
Ziping Jiang, Plamen Angelov, Dmitry Kangin, Zhaonian Zhang, Richard Jiang
Self-supervised Representation Learning for Adversarial Attack Detection
Yi Li, Plamen Angelov, Neeraj Suri
STEP - Towards Structured Scene-Text Spotting
Sergi Garcia-Bordils, Dimosthenis Karatzas, Marçal Rusiñol
The BRAVO Semantic Segmentation Challenge Results in UNCV2024
Tuan-Hung Vu, Eduardo Valle, Andrei Bursuc, Tommie Kerssies, Daan de Geus, Gijs Dubbelman, Long Qian, Bingke Zhu, Yingying Chen, Ming Tang, Jinqiao Wang, Tomáš Vojíř, Jan Šochman, Jiří Matas, Michael Smith, Frank Ferrie, Shamik Basu, Christos Sakaridis, L
The European Union's AI Act: beyond motherhood and apple pie?
· 2024DOI
Nathalie A. Smuha, Karen Yeung
Towards algorithms and models that we can trust: A theoretical perspective
Luca Oneto, Sandro Ridella, Davide Anguita
Training-Free Open-Vocabulary Segmentation with Offline Diffusion-Augmented Prototype Generation
Luca Barsellotti, Roberto Amoroso, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara
Will You Participate? Exploring the Potential of Robotics Competitions on Human-centric Topics
Yuchong Zhang, Miguel Vasco, Mårten Björkman, Danica Kragic
σ-zero: Gradient-based Optimization of ℓ0-norm Adversarial Examples
· 2024DOI
Cinà, A.E., Villani, F., Pintor, M., Schönherr, L., Biggio, B., Pelillo, M.,
Abstract Interpretation of Fixpoint Iterators with Applications to Neural Networks
Mark Niklas Müller, Marc Fischer, Robin Staab, Martin Vechev
Advancing Personalized Federated Learning: Group Privacy, Fairness, and Beyond
Filippo Galli, Kangsoo Jung, Sayan Biswas, Catuscia Palamidessi, Tommaso Cucinotta
Adversarial Attacks Against Uncertainty Quantification
Emanuele Ledda, Daniele Angioni, Giorgio Piras, Giorgio Fumera, Battista Biggio, Fabio Roli;
Adversarial Causal Bayesian Optimization
S. Sussex, P. G. Sessa, A. Makarova, A. Krause
Automated Classification of Model Errors on ImageNet
Momchil Peychev, Mark Niklas Müller, Marc Fischer, Martin Vechev
Cybersecurity and AI: The PRALab Research Experience
3rd National Conference on Artificial Intelligence· 2023
Maura Pintor, Giulia Orrù, Davide Maiorca, Ambra Demontis, Luca Demetrio, Gian Luca Marcialis, Battista Biggio, Fabio Roli
Dispelling the Digital Enchantment: how can we move beyond its destructive influence and reclaim our right to an open future?
Karen Yeung
Distributionally Robust Model-based Reinforcement Learning with Large State Spaces
S.S. Ramesh, P. G. Sessa, Y. Hu, A. Krause, I. Bogunovic
DocILE Benchmark for Document Information Localization and Extraction
Document Analysis and Recognition - ICDAR 2023. ICDAR 2023. Lecture Notes in Computer Science· 2023DOI
Štěpán Šimsa, Milan Šulc, Michal Uřičář, Yash Patel, Ahmed Hamdi, Matěj Kocián, Matyáš Skalický, Jiří Matas, Antoine Doucet, Mickaël Coustaty, Dimosthenis Karatzas
Domain Randomization for Robust, Affordable and Effective Closed-loop Control of Soft Robots
IEEE Internationa Conference on Intelligent Robots and Systems (IROS) 2023· 2023
Gabriele Tiboni, Andrea Protopapa, Tatiana Tommasi, Giuseppe Averta
DPVIm: Differentially Private Variational Inference Improved
TMLR 9/2023· 2023
Joonas Jälkö, Lukas Prediger, Antti Honkela, Samuel Kaski
DRCFS: Doubly Robust Causal Feature Selection
Francesco Quinzan, Ashkan Soleymani, Patrick Jaillet, Cristian R. Rojas, Stefan Bauer
Exploring the role of Text in Visual Question Answering on Natural Scenes and Documents
· 2023
Ruben Perez Tito
Expressivity of ReLU-Networks under Convex Relaxations
Maximilian Baader, Mark Niklas Müller, Yuhao Mao, Martin Vechev
Fair Empirical Risk Minimization Revised
Franco, D. and Oneto, L. and Anguita, D.
Fuzzy Detectors Against Adversarial Attacks
IEEE Symposium Series on Computational Intelligence· 2023
Y. Li, P. Angelov, N. Suri
Hierarchical multimodal transformers for Multipage DocVQA
Rubèn Tito, Dimosthenis Karatzas, Ernest Valveny
Human Uncertainty in Concept-Based AI Systems
Katherine Maeve Collins ,Matthew Barker, Mateo Espinosa Zarlenga, Naveen Raman, Umang Bhatt, Mateja Jamnik, Ilia Sucholutsky ,Adrian Weller , Krishnamurthy Dvijotham
Improving Fairness via Intrinsic Plasticity in Echo State Networks
European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN)· 2023DOI
Ceni, A. and Bacciu, D. and De Caro, V. and Gallicchio, C. and Oneto, L.
Learning Counterfactually Invariant Predictors
Francesco Quinzan, Cecilia Casolo, Krikamol Muandet, Yucen Luo, Niki Kilbertus
Mitigating Robustness Bias: Theoretical Results and Empirical Evidences
European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN)· 2023DOI
Franco, D. and Oneto, L. and Anguita, D.
Multitask Learning with No Regret: From Improved Confidence Bounds to Active Learning
Advances in Neural Information Processing Systems 36 (NeurIPS 2023)· 2023
Pier Giuseppe Sessa, Pierre Laforgue, Nicolò Cesa-Bianchi, Andreas Krause
PØDA: Prompt-driven Zero-shot Domain Adaptation
Mohammad Fahes, Tuan-Hung Vu, Andrei Bursuc, Patrick Pérez, Raoul de Charette
Robust Meta-Representation Learning via Global Label Inference and Classification
· 2023DOI
Ruohan Wang, Isak Falk, Massimiliano Pontil, Carlo Ciliberto
STR-Cert: Robustness Certification for Deep Text Recognition on Deep Learning Pipelines and Vision Transformers
Daqian Shao, Lukas Fesser, Marta Kwiatkowska
Synthcap: Augmenting transformers with synthetic data for image captioning
Caffagni, D., Barraco, M., Cornia, M., Baraldi, L., Cucchiara, R
Text-DIAE: A Self-Supervised Degradation Invariant Autoencoder for Text Recognition and Document Enhancement
Mohamed Ali Souibgui, Sanket Biswas, Andres Mafla, Ali Furkan Biten, Alicia Fornés, Yousri Kessentini, Josep Lladós, Lluis Gomez, Dimosthenis Karatzas
Towards Robust Metrics for Concept Representation Evaluation
Mateo Espinosa Zarlenga, Pietro Barbiero, Zohreh Shams, Dmitry Kazhdan, Umang Bhatt, Adrian Weller, Mateja Jamnik
Unmasking Anomalies in Road-Scene Segmentation
Shyam Nandan Rai , Fabio Cermelli, Dario Fontanel, Carlo Masone, Barbara Caputo
3DOS: Towards 3D Open Set Learning - Benchmarking and Understanding Semantic Novelty Detection on Point Clouds
Advances in Neural Information Processing Systems 35 (NeurIPS 2022) Datasets and Benchmarks Track· 2022
Antonio Alliegro, Francesco Cappio Borlino, Tatiana Tommasi
Indicators of Attack Failure: Debugging and Improving Optimization of Adversarial Examples
Maura Pintor, Luca Demetrio, Angelo Sotgiu, Ambra Demontis, Nicholas Carlini, Battista Biggio, Fabio Roli
Secml-Malware: Pentesting Windows Malware Classifiers with Adversarial Exemples in Python
Luca Demetrio, Battista Biggio
Transient-Fault-Aware Design and Training to Enhance DNNs Reliability with Zero-Overhead
Niccolò Cavagnero; Fernando Dos Santos; Marco Ciccone; Giuseppe Averta; Tatiana Tommasi; Paolo Rech
"MargCTGAN: A ""Marginally"" Better CTGAN for the Low Sample Regime"
Tejumade Afonja, Dingfan Chen, Mario Fritz
1000 African Voices: Advancing inclusive multi-speaker multi-accent speech synthesis
Biomedical Research in Artificial Intelligence and Machine PerceptionDOI
Sewade Ogun, Abraham T. Owodunni, Tobi Olatunji, Eniola Alese, Babatunde Oladimeji, Tejumade Afonja, Kayode Olaleye, Naome A. Etori, Tosin Adewumi
Accelerating Transformer-Based Scene Text Detection and Recognition via Token Pruning
Document Analysis and Recognition - ICDAR 2023. ICDAR 2023. Lecture Notes in Computer ScienceDOI
S, Garcia-Bordils, D. Karatzas, M. Rusiñol
Actsafe: Active exploration with safety constraints for reinforcement learning
ICLR 2025
As, Yarden and Sukhija, Bhavya and Treven, Lenart and Sferrazza, Carmelo and Coros, Stelian and Krause, Andreas
Adaptive Hierarchical Certification for Segmentation using Randomized Smoothing
International Conference on Machine Learning (ICML)
Alaa Anani, Tobias Lorenz, Bernt Schiele, Mario Fritz
Adversarial Attack Detection via Fuzzy Predictions
IEEE Transactions on Fuzzy SystemsDOI
Y. Li, P. Angelov, N. Suri
Adversarial Robustness Certification for Bayesian Neural Networks
Lecture Notes in Computer ScienceDOI
Matthew Wicker, Andrea Patane, Luca Laurenti, Marta Kwiatkowska
AI Security and Safety: The PRALab Research Experience
Ital-IA 2023
Ambra Demontis, Maura Pintor, Luca Demetrio, Angelo Sotgiu, Daniele Angioni, Giorgio Piras, Srishti Gupta, Battista Biggio and Fabio Roli
An Empirical Study of Over-Parameterized Neural Models based on Graph Random Features
European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN)DOI
Navarin, N. and Pasa, L. and Oneto, L. and Sperduti, A.
Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?
ICLR 2024 Workshop on Secure and Trustworthy Large Language ModelsDOI
Egor Zverev, Sahar Abdelnabi, Mario Fritz, Christoph H. Lampert
CausalGraph2LLM: Evaluating LLMs for Causal Queries
NAACL'25
Ivaxi Sheth, Bahare Fatemi, Mario Fritz
Causality Is Key to Understand and Balance Multiple Goals in Trustworthy ML and Foundation Models
Ruta Binkyte, Ivaxi Sheth, Zhijing Jin, Mohammad Havaei, Bernhard Schölkopf, Mario Fritz
Certification of Distributional Individual Fairness
Neural Information Processing Systems (NeurIPS), 2023.DOI
M. Wicker, V. Piratla and A. Weller.
Certified Robust Models with Slack Control and Large Lipschitz Constants Proceedings Article
DAGM German Conference on Pattern Recognition (GCPR), 2023.DOI
Max Losch, David Stutz, Bernt Schiele, Mario Fritz
Certified Training: Small Boxes are All You Need
ICLR (Spotlight)DOI
Mark Niklas Müller, Franziska Eckert, Marc Fischer, Martin Vechev
Certifiers Make Neural Networks Vulnerable to Availability Attacks
16th ACM Workshop on Artificial Intelligence and Security (AISec 2023DOI
Tobias Lorenz, Marta Kwiatkowska, Mario Fritz
Client-specific Property Inference against Secure Aggregation in Federated Learning
Proceedings of the 22nd Workshop on Privacy in the Electronic Society (WPES), ACM, 2023DOI
Raouf Kerkouche, Gergely Ács, Mario Fritz
CoBo: Collaborative Learning via Bilevel Optimization
NeurIPS 2024
Diba Hashemi, Lie He, Martin Jaggi
CodeLMSec Benchmark: Systematically Evaluating and Finding Security Vulnerabilities in Black-Box Code Language Models
2nd IEEE Conference on Secure and Trustworthy Machine Learning (SATML), 2024DOI
Hossein Hajipour, Keno Hassler, Thorsten Holz, Lea Schönherr, Mario Fritz
Collaborative Learning via Prediction Consensus
Dongyang Fan, Celestine Mendler-Dünner, Martin Jaggi
Comparing Abstraction in Humans and Large Language Models Using Multimodal Serial Reproduction.
Conference of the Cognitive Science Society (CogSci) 2024DOI
S. Kumar, R. Marjieh, B. Zhang, D. Campbell, M. Hu, U. Bhatt, B. Lake and T. Griffiths.
Complex-Cycle-Consistent Diffusion Model for Monaural Speech Enhancement
AAAI Conference on Artificial IntelligenceDOI
Y. Li, Y. Sun, P. Angelov
Confidential-DPproof: Confidential Proof of Differentially Private Training
International 12th Conference on Learning Representations
Ali Shahin Shamsabadi, Gefei Tan, Tudor Ioan Cebere, Aurélien Bellet, Hamed Haddadi, Nicolas Papernot, Xiao Wang, Adrian Weller
Confidential-PROFITT: Confidential PROof of FaIr Training of Trees
The Eleventh International Conference on Learning Representations
Ali Shahin Shamsabadi, Sierra Calanda Wyllie, Nicholas Franzese, Natalie Dullerud, Sébastien Gambs, Nicolas Papernot, Xiao Wang, Adrian Weller
Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation
NeurIPS - Datasets and Benchmarks'24DOI
Sahar Abdelnabi, Amr Gomaa, Sarath Sivaprasad, Lea Schönherr, Mario Fritz
CoTFormer: More Tokens With Attention Make Up For Less Depth
Amirkeivan Mohtashami, Matteo Pagliardini, Martin Jaggi
Do Invariances in Deep Neural Networks Align with Human Perception?
Association for the Advancement of Artificial Intelligence Conference on Artificial Intelligence (AAAI), 2023.DOI
V. Nanda, A. Majumdar, C. Kolling, J. Dickerson, K. Gummadi, B. Love and A. Weller.
DocMIA: Document-Level Membership Inference Attacks against DocVQA Models
ICLR 2025
Khanh Nguyen, Raouf Kerkouche, Mario Fritz, Dimosthenis Karatzas
DocVXQA: Context-Aware Visual Explanations for Document Question Answering
International Conference on Machine Learning
Mohamed Ali Souibgui, Changkyu Choi, Andrey Barsky, Kangsoo Jung, Ernest Valveny, Dimosthenis Karatzas
DoGE: Domain Reweighting with Generalization Estimation
Simin Fan, Matteo Pagliardini, Martin Jaggi
EarthLoc: Astronaut Photography Localization by Indexing Earth from Space
CVPR 2024DOI
Gabriele Berton, Alex Stoken, Barbara Caputo, Carlo Masone
Efficient Certified Training and Robustness Verification of Neural ODEs
ICLR
Mustafa Zeqiri, Mark Niklas Müller, Marc Fischer, Martin Vechev
Efficient Model Editing with Task-Localized Sparse Fine-tuning
International Conference on Learning Representations
Leonardo Iurada, Marco Ciccone, Tatiana Tommasi
Efficient Robustness Verification of Neural Ordinary Differential Equations
The Symbiosis of Deep Learning and Differential Equations II
Mustafa Zeqiri, Mark Niklas Müller, Marc Fischer, Martin Vechev
epfLLM Megatron-LLM
AH Cano, M Pagliardini, A Köpf, K Matoba, A Mohtashami, OS Fan, A Marmet, D Bayazit, I Krawczuk, Z Chen, F Salvi, A Bosselut, M Jaggi
Establishing the Price of Privacy in Federated Data Trading
Kangsoo Jung, Sayan Biswas, Catuscia Palamidessi
Evaluating Language Models for Mathematics through Interactions
Katherine M. Collins, Albert Q. Jiang, Simon Frieder, Lionel Wong, Miri Zilka, Umang Bhatt, Thomas Lukasiewicz, Yuhuai Wu, Joshua B. Tenenbaum, William Hart, Timothy Gowers, Wenda Li, Adrian Weller, Mateja Jamnik
Evaluating the Evaluators: Trust in Adversarial Robustness Tests
Antonio Emanuele Cinà, Maura Pintor, Luca Demetrio, Ambra Demontis, Battista Biggio, Fabio Roli
Explainable Audio-Visual Representation Learning via Prototypical Contrastive Masked Autoencoder
Advances in neural information processing systems
Y. Li, P. Angelov
Fact-Saboteurs: A Taxonomy of Evidence Manipulation Attacks against Fact-Verification Systems Proceedings Article
USENIX Security Symposium (USENIX Security)}, 2023DOI
Sahar Abdelnabi, Mario Fritz
Fair graph representation learning: Empowering NIFTY via Biased Edge Dropout and Fair Attribute Preprocessing
NeurocomputingDOI
Danilo Franco, Vincenzo Stefano D’Amato, Luca Pasa, Nicolò Navarin, Luca Oneto
Fast Attention Over Long Sequences With Dynamic Sparse Flash Attention
Matteo Pagliardini ~Matteo_Pagliardini1 , Daniele Paliotta, Martin Jaggi, François Fleuret
Fast Feature Selection with Fairness Constraints
2nd Workshop on Formal Verification of Machine Learning (WFVML 2023)DOI
Francesco Quinzan, Rajiv Khanna, Moshik Hershcovitch, Sarel Cohen, Daniel Waddington, Tobias Friedrich and Michael W. Mahoney
Faster Causal Attention Over Large Sequences Through Sparse Flash Attention
Matteo Pagliardini, Daniele Paliotta, Martin Jaggi, François Fleuret
Federated Document Visual Question Answering: A Pilot Study
ICDAR 2024DOI
Khanh Nguyen, Dimosthenis Karatzas
FedLAP-DP: Federated Learning by Sharing Differentially Private Loss Approximations
Hui-Po Wang, Dingfan Chen, Raouf Kerkouche, Mario Fritz
FeedbackLogs: Recording and Incorporating Stakeholder Feedback into Machine Learning Pipelines
Matthew Barker, Emma Kallina, Dhananjay Ashok, Katherine M. Collins, Ashley Casovan, Adrian Weller, Ameet Talwalkar, Valerie Chen, Umang Bhatt
FineWeb2: A sparkling update with 1000s of languages
github open source release
Guilherme Penedo, Hynek Kydlíček, Vinko Sabolčec, Bettina Messmer, Negar Foroutan, Martin Jaggi, Leandro von Werra, Thomas Wolf
FLOSS: Free Lunch in Open-vocabulary Semantic Segmentation
International Conference on Computer Vision, ICCV 2025
Yasser Benigmim, Mohammad Fahes, Tuan-Hung Vu, Andrei Bursuc, Raoul de Charette
From Attachments to SEO: Click Here to Learn More about Clickbait PDFs!
ACSAC '23: Proceedings of the 39th Annual Computer Security Applications ConferenceDOI
Giada Stivala; Sahar Abdelnabi; Andrea Mengascini; Mariano Graziano; Mario Fritz; Giancarlo Pellegrino
FullCert: Deterministic End-to-End Certification for Training and Inference of Neural Networks (LKF24)
The German Conference on Pattern Recognition (GCPR)DOI
Tobias Lorenz, Marta Kwiatkowska, Mario Fritz
Generating Scenarios from High-Level Specifications for Object Rearrangement Tasks
2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)DOI
Sanne van Waveren, Christian Pek , Iolanda Leite, Jana Tumova, Danica Kragic
Geometric Multimodal Contrastive Representation Learning
Petra Poklukar, Miguel Vasco, Hang Yin, Francisco S. Melo, Ana Paiva, Danica Kragic
Get my drift? Catching LLM Task Drift with Activation Deltas
IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) , 2025.DOI
Sahar Abdelnabi; Aideen Fay; Giovanni Cherubin; Ahmed Salem; Mario Fritz; Andrew Paverd
Group Meritocratic Fairness in Linear Contextual Bandits
NeurIPS 2022DOI
Riccardo Grazzi, Arya Akhavan, John Isak Texas Falk, Leonardo Cella, Massimiliano Pontil
How to Probe: Simple Yet Effective Techniques for Improving Post-hoc Explanations
International Conference on Learning Representations
Siddhartha Gairola, Moritz Böhle, Francesco Locatello, and Bernt Schiele
Human-in-the-Loop Mixup
Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence
Katherine M. Collins, Umang Bhatt, Weiyang Liu, Vihari Piratla, Ilia Sucholutsky, Bradley Love, Adrian Weller
Hyperbolic Safety-Aware Vision-Language Models
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Tobia Poppi;Tejaswi Kasarla;Pascal Mettes;Lorenzo Baraldi;Rita Cucchiara
Hypothesizing Missing Causal Variables with LLMs
NeurIPS 2024 Workshop on Causality and Large Models (CaLM).DOI
Ivaxi Sheth; Sahar Abdelnabi; Mario Fritz
IEEE Transactions on Pattern Analysis and Machine Intelligence
Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2024.DOI
Moritz Böhle, Navdeeppal Singh, Mario Fritz, Bernt Schiele
Individual Privacy Accounting with Gaussian Differential Privacy
ICLR 2023
Antti Koskela, Marlon Tobaben, Antti Honkela
Interpretable-through-prototypes deepfake detection for diffusion models
2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)DOI
Agil Aghasanli; Dmitry Kangin; Plamen Angelov
Is Mamba Capable of In-Context Learning?
AutoML24DOI
Riccardo Grazzi, Julien Niklas Siems, Simon Schrodi, Thomas Brox, Frank Hutter
Iterative Teaching by Data Hallucination
International Conference on Artificial Intelligence and Statistics (AISTATS), 2023.DOI
Z. Qiu, W. Liu, T. Xiao, Z. Liu, U. Bhatt, Y. Luo, A. Weller and B. Schölkopf.
Landmark Attention: Random-Access Infinite Context Length for Transformers
Amirkeivan Mohtashami, Martin Jaggi
Language Models as Zero-shot Lossless Gradient Compressors: Towards General Neural Parameter Prior Models
38th Conference on Neural Information Processing Systems (NeurIPS 2024)DOI
Hui-Po Wang; Mario Fritz
Large Class Separation is Not What You Need for Relational Reasoning-Based OOD Detection
International Conference on Image Analysis and Processing (ICIAP) 2023DOI
Lorenzo Li Lu, Giulia D’Ascenzi, Francesco Cappio Borlino & Tatiana Tommasi
Large Language Models Must Be Taught What They Don’t know
Conference on Neural Information Processing Systems (NeurIPS 2024).DOI
Sanyam Kapoor, Nate Gruver, Manley Roberts, Katherine Collins, Arka Pal, Umang Bhatt, Adrian Weller, Samuel Dooley, Micah Goldblum, Andrew Gordon Wilson
Learning Decision Policies with Instrumental Variables through Double Machine Learning
Forty-first International Conference on Machine Learning
Daqian Shao, Ashkan Soleymani, Francesco Quinzan, Marta Kwiatkowska
Learning Personalized Decision Support Policies
Association for the Advancement of Artificial Intelligence Conference on Artificial IntelligenceDOI
Umang Bhatt, Valerie Chen, Katherine M. Collins, Parameswaran Kamalaruban, Emma Kallina, Adrian Weller, Ameet Talwalkar
Learning Safety Constraints for Large Language Models
ICML 2025
Xin Chen and Yarden As and Andreas Krause
Learning to Generate Training Datasets for Robust Semantic Segmentation
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2024DOI
Marwane Hariat, Olivier Laurent, Rémi Kazmierczak, Shihao Zhang, Andrei Bursuc, Angela Yao, Gianni Franchi
Learning to Receive Help: Intervention-Aware Concept Embedding Models
Neural Information Processing SystemsDOI
Mateo Espinosa Zarlenga, Katherine M. Collins, Krishnamurthy Dvijotham, Adrian Weller, Zohreh Shams, Mateja Jamnik
Less is More? An Ablation Study on AutoAttack for Adversarial Robustness Evaluation
Luca Melis, Luca Scionis, Fabio Brau, Maura Pintor, Battista Biggio
Let's ViCE! Mimicking Human Cognitive Behavior in Image Generation Evaluation
Federico Betti, Jacopo Staiano, Lorenzo Baraldi, Lorenzo Baraldi, Rita Cucchiara, Nicu Sebe
LLM Task Interference: An Initial Study on the Impact of Task-Switch in Conversational History
Proceedings of the 2024 Conference on Empirical Methods in Natural Language ProcessingDOI
Akash Gupta; Ivaxi Sheth; Vyas Raina; Mark Gales; Mario Fritz
LLM2Swarm: Robot Swarms that Responsively Reason, Plan, and Collaborate through LLMs
NeurIPS 2024 Workshop on Open-World AgentsDOI
Volker Strobel, Marco Dorigo, Mario Fritz
LLMs on interactive feature collections with implicit dynamic decision strategy
Proceedings of the 31st International Conference on Computational Linguistics
Juyeon Heo, Vihari Piratla, Kyunghyun Lee, Hyonkeun Joh, Adrian Weller
Lost in translation: the troubling logics underpinning the embrace of governmental machine-learning based prediction tools for ‘citizen scoring’
Global Governance by DataDOI
Karen Yeung
Machine learning within latent spaces formed by foundation models
2024 IEEE 12th International Conference on Intelligent Systems (IS)
B Tomczyk, P Angelov, D Kangin
Machine Unlearning for Document Classification
ICDAR 2024DOI
Lei Kang, Mohamed Ali Souibgui, Fei Yang, Lluis Gomez, Ernest Valveny, Dimosthenis Karatzas
MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximization
ICLR 2025
Bhavya Sukhija, Stelian Coros, Andreas Krause, Pieter Abbeel, Carmelo Sferrazza
Medical Multimodal Model Stealing Attacks via Adversarial Domain Alignment
Yaling Shen; Zhixiong Zhuang; Kun Yuang; Maria-Irina Nicolae; Nassir Navab; Nicolas Padoy; Mario Fritz
MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models
International Conference on Learning Representations (ICLR), 2024
Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T. Kwok, Zhenguo Li, Adrian Weller, Weiyang Liu
Mitigating Unintended Memorization with LoRA in Federated Learning for LLMs
arXivDOI
Thierry Bossy, Julien Vignoud, Tahseen Rabbani, Juan R Troncoso Pastoriza, Martin Jaggi
ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models
USENIX Security Symposium (USENIX Security), 2022DOI
Yugeng Liu, Rui Wen, Xinlei He, Ahmed Salem, Zhikun Zhang, Michael Backes, Emiliano De Cristofaro, Mario Fritz, Yang Zhang
Modulating Language Model Experiences through Frictions
Neural Information Processing Systems (NeurIPS 2024) Workshop on Behavioral Machine LearningDOI
Katherine M. Collins, Valerie Chen, Ilia Sucholutsky, Hannah Rose Kirk, Malak Sadek, Holli Sargeant, Ameet Talwalkar, Adrian Weller, Umang Bhatt
Multi-Page Document Visual Question Answering using Self-Attention Scoring Mechanism
ICDAR 2024DOI
Lei Kang, Rubèn Tito, Ernest Valveny, Dimosthenis Karatzas
Multi-task representation learning with stochastic linear bandits
AISTATS 2023
Leonardo Cella, Karim Lounici, Grégoire Pacreau, Massimiliano Pontil
Multiplication-Free Transformer Training via Piecewise Affine Operations
Atli Kosson, Martin Jaggi
Multitask Online Learning: Listen to the Neighborhood Buzz
Artificial Intelligence and Statistics 2024DOI
Juliette Achddou, Nicolò Cesa-Bianchi, Pierre Laforgue
NeurIPS 2023 Competition: Privacy Preserving Federated Learning Document VQA
NEURIPS 2024DOI
Marlon Tobaben, Mohamed Ali Souibgui, Rubèn Tito, Khanh Nguyen, Raouf Kerkouche, Kangsoo Jung, Joonas Jälkö, Lei Kang, Andrey Barsky, Vincent Poulain d'Andecy, Aurélie JOSEPH, Aashiq Muhamed, Kevin Kuo, Virginia Smith, Yusuke Yamasaki, Takumi Fukami, Kent
Noise-Aware Differentially Private Regression via Meta-Learning
NeurIPS 2024
Ossi Räisä, Stratis Markou, Matthew Ashman, Wessel P Bruinsma, Marlon Tobaben, Antti Honkela, Richard E. Turner
Noise-Aware Statistical Inference with Differentially Private Synthetic Data
PMLR
Ossi Räisä, Joonas Jälkö, Samuel Kaski, Antti Honkela
Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection Proceedings Article
16th ACM Workshop on Artificial Intelligence and Security (AISec), 2023DOI
Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, Mario Fritz
On Adversarial Training without Perturbing All Examples Proceedings Article
The Twelfth International Conference on Learning Representations (ICLR), 2024
Max Losch; Mohamed Omran; David Stutz; Mario Fritz; Bernt Schiele
▶Deliverables (6)
Data sets, microdata, etc
Documents, reports
Documents, reports
Documents, reports