Literature Database Entry

li2026byzantineresilient-preprint

Kai Li, Hanlin Cai, Zihao Liu, Yichen Li, Jose Alcaraz Calero, Wei Ni, Falko Dressler and Ozgur B. Akan, "Byzantine-Resilient Federated Large Language Model via Hypergraph Message Passing," arXiv, cs.LG, 2610.12711, October 2026.

Abstract

Federated Large Language Models (FedLLMs) are promising to deliver privacy-preserving, collaborative intelligence. The decentralized training process is vulnerable to Byzantine attacks that corrupt semantic consistency and factual resilience through malicious model updates. This paper presents a new resilience strategy that prevents semantically corrupted knowledge from propagating during federated aggregation. A defense problem is formulated to jointly maximize global accuracy and suppress manipulated model updates, enabling trustworthy knowledge aggregation without accessing private local data. To strengthen the resilience of FedLLMs, we develop a novel Hypergraph Message Passing Graph Autoencoder (HMP-GAE) framework, where low-rank model updates are represented as nodes in a hypergraph and high-order semantic relationships among multiple agents are exploited through iterative message passing. By leveraging historical checkpoints and cross-agent relational reasoning, HMP-GAE identifies and suppresses semantically inconsistent updates while preserving knowledge alignment among agents throughout federated training. HMP-GAE is implemented using Qwen-2.5 and Llama-3.2 as pretrained FedLLM backbones and evaluated on AG News and Yahoo! Answers datasets. Compared to existing defenses, HMP-GAE improves global accuracy by 1.21%, reduces final-model classification semantic entropy by 44.08%, and lowers perplexity by 13.34%. HMP-GAE outperforms the GAE-based defense by improving the accuracy 1.39%, reducing the semantic entropy by 54.5%, and lowering perplexity by 35.5%.

Quick access

Original Version DOI (at publishers web site)
BibTeX BibTeX

Contact

Kai Li
Hanlin Cai
Zihao Liu
Yichen Li
Jose Alcaraz Calero
Wei Ni
Falko Dressler
Ozgur B. Akan

BibTeX reference

@techreport{li2026byzantineresilient-preprint,
    author = {Li, Kai and Cai, Hanlin and Liu, Zihao and Li, Yichen and Calero, Jose Alcaraz and Ni, Wei and Dressler, Falko and Akan, Ozgur B.},
    doi = {10.48550/arXiv.2610.12711},
    title = {{Byzantine-Resilient Federated Large Language Model via Hypergraph Message Passing}},
    institution = {arXiv},
    month = {10},
    number = {2610.12711},
    type = {cs.LG},
    year = {2026},
   }
   
   

Copyright notice

Links to final or draft versions of papers are presented here to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted or distributed for commercial purposes without the explicit permission of the copyright holder.

The following applies to all papers listed above that have IEEE copyrights: Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.

The following applies to all papers listed above that are in submission to IEEE conference/workshop proceedings or journals: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.

The following applies to all papers listed above that have ACM copyrights: ACM COPYRIGHT NOTICE. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Publications Dept., ACM, Inc., fax +1 (212) 869-0481, or permissions@acm.org.

The following applies to all SpringerLink papers listed above that have Springer Science+Business Media copyrights: The original publication is available at www.springerlink.com.

This page was automatically generated using BibDB and bib2web.

Last modified: 2026-10-13