A federated learning study shows 96% accuracy can miss every rare attack
Initial-Street6388 · reddit · 2026-07-22
A federated learning study shows global accuracy can hide total failure on rare attacks
The author compared FedAvg, FedProx, FedNova, and a centralized baseline on CICIDS2017 for network intrusion detection, with the data split across four simulated silos by attack type.
A key minority silo, Web Attacks, had only about 3k samples out of 3 million total. The post reports that:
- FedAvg reached about 96% global accuracy while showing only 49% accuracy on the minority silo and 0.00 recall for the attack class.
- The centralized baseline varied wildly across 10 random seeds, swinging from about 57% to 99.5% on the same rare category.
- FedNova was the most stable, staying in the high 90s across silos and seeds without sacrificing global accuracy.
The takeaway: in federated intrusion detection, global accuracy alone is misleading, and aggregation choice matters a lot for rare attack detection.
More from Research
- Multi-resolution image stacks beat pyramidal magnitude in a new audio test — johnowhitaker · 2026-07-22
- AlphaFold3 MSA retraining probes whether it learns inverse covariance structure — anshulkundaje · 2026-07-22
- New NBER paper on how organizations use AI completes a three-paper series — daveholtz · 2026-07-22
- OAT uses 100 successful trajectories to debug failing AI agents without failure labels — TheTuringPost · 2026-07-22
- MoE, the mixture-of-experts architecture behind many top LLMs — vista8 · 2026-07-22
- NexForge synthesizes agent training data from requirements and lifts Qwen3.5 by 30 points — nex-agi · 2026-07-22