Time to Fake It: Continual Machine Learning Intensifies Unfairness in Deepfake Detection
Kajaree Das, Oliver Böse, Subarnaduti Paul, Gabriel Storch, Martin Mundt
cs.LG, cs.AI
2026-10-15
On 11 generators in release order, SOYO keeps ~93% accuracy, but age FOAE is worse than naive fine-tuning for every continual learner, and skin-tone FOAE hits 0.547.
Face generators turn over every few months. A detector frozen on last year's data misses the next synthesis method. Fine-tuning on the new data wipes accuracy on the old one. That is catastrophic forgetting: the update that learns the new task overwrites the old decision boundary.
Continual learning is the usual patch, and on deepfakes it has been scored almost only by accuracy. Li et al. order generators by how hard they look and treat each method as its own class. Fontana et al. order them by release year and still report whether the fake was caught. A healthy average can hide a rising miss rate, or a rising false alarm rate, on one age band or skin tone. Training uses real/fake labels only. Gender, age, and skin tone never enter the loss.
Work that claims continual learning improves fairness mostly tracks one metric, or feeds sensitive attributes as tasks that do not overlap in time. A new generator does not arrive as a demographically pure batch. The mix inside each dataset shifts on its own.
The Bremen group pulls 11 generators from AI-Face-FairnessBench and sorts them by public release, from FaceForensics++ in January 2019 through Stable Diffusion in September 2023. In between sit DFD, CelebDF, DFDC, VQGAN, three StyleGAN generations, Palette, and DCFace. Early tasks are face-swap pipelines. Later ones move from GANs to diffusion. Scale is lopsided: Palette's training split has 4,804 images, DCFace's has 424,452.
The backbone is an ImageNet-pretrained ViT-B/16. Each step is binary real versus fake. Demographics are read only at test time: binary gender; five age bands (child, youth, adult, middle-aged, senior); the 10-step Monk skin-tone scale; and gender crossed with light (ST1-3), medium (ST4-7), and dark (ST8-10).
Six methods sit between two baselines, in three families. Accumulated retrains on every image seen so far and is the upper bound on accuracy and on fairness stability. Naive fine-tunes on the current generator only and is the forgetting floor. EWC penalizes changes to parameters the Fisher information marks as important for past tasks, with strength λ = 5000. DER++ and GradMatch each keep a replay buffer of 3,000 images. DER++ replays stored logits and labels. GradMatch keeps samples whose gradients match a validation set. S-Prompt, PINA, and SOYO freeze the backbone, learn a short prompt per generator, and route each test image to the nearest domain. They store no old images.
Unfairness is four normalized gaps. Zero is parity across groups. Values nearer 1 are worse. Demographic parity (FDP) is the gap in how often a group is labeled fake, true label ignored. Equal opportunity (FEOP) compares only how often fakes are caught. Equalized odds (FEO) also counts real faces wrongly flagged. Overall accuracy equality (FOAE) is the widest accuracy gap across groups. The four can move in opposite directions, so all four are reported, as means over five seeds.
On accuracy, continual learning blocks most of the forgetting. Naive average accuracy on tasks seen so far falls to 46.6% by task 5. After task 10, the task-3 test set is down to 13.98%. Accumulated stays near 99% the whole way. SOYO finishes near 93%, still above 80% on every earlier task. DER++ and GradMatch finish near 88%, with some old tasks under 80%. EWC forgets about as fast as naive training. PINA and S-Prompts land lower and flatter.
Fairness is a separate ledger. The table fixes the first generator's test set. The first column is the gap right after that task. The other columns are the same test set after all 11 updates. Prompt methods do not share that starting point. The paper lines them up against it anyway.
| Method | Age FOAE | Skin-tone FOAE | Gender FDP |
| Task 1 only | 0.060 | 0.070 | 0.001 |
| Accumulated | 0.020 | 0.062 | 0.001 |
| Naive fine-tune | 0.104 | 0.414 | 0.176 |
| EWC | 0.109 | 0.457 | 0.143 |
| DER++ | 0.215 | 0.288 | 0.047 |
| GradMatch | 0.221 | 0.231 | 0.068 |
| SOYO | 0.137 | 0.547 | 0.009 |
| S-Prompts | 0.166 | 0.286 | 0.014 |
| PINA | 0.110 | 0.531 | 0.012 |
Every continual learner has a worse age FOAE than naive fine-tuning, and the margin is uneven. PINA at 0.110 sits just above naive at 0.104. DER++ at 0.215 and GradMatch at 0.221 are about double, against an initial 0.060. Age FDP moves the same way: DER++ 0.302 and GradMatch 0.295, from 0.111.
SOYO, the accuracy leader, also posts the worst skin-tone FOAE, 0.547. PINA is next at 0.531. Both sit above naive fine-tuning at 0.414, and far above the initial 0.070. EWC's skin-tone FDP of 0.608 matches naive at 0.597. Gender goes the other way. Prompt methods stay near the upper bound: SOYO's FDP is 0.009, naive is 0.176, accumulated is 0.001. On gender crossed with skin tone, naive FDP is 0.550 and EWC is 0.474. SOYO comes back to 0.108.
Forgetting and unfairness do not land on the same tasks. Under DER++, task 3 forgets only about 6% and still shows the largest rise in age FOAE. Tasks 4 and 6 forget 4% to 13% and still lose fairness. FDP sometimes improves, including age on tasks 3 to 5, while FEO, FEOP, and FOAE mostly get worse. FEO rises more than FEOP, so the extra harm is real faces from some groups being flagged as fake.
Headcount does not finish the story. In GradMatch the buffer share tracks cumulative sample share, so rare groups are rehearsed less. SOYO's routing does not follow the same rule. Gender is nearly balanced, yet female faces are sent to another task's prompt a bit more often. Children are as scarce as seniors and are routed about as well as adults. Routing accuracy for dark tones (ST8-10) falls below 50%, while light tones (ST1-3) stay at 50-60%, and the dark-tone slice is not the smaller one. Under EWC, light and dark tones are both less protected than medium tones. Female × dark, about 12% of the data by the end, is protected better than female × medium at about 34%. Replay follows counts. Routing and weight protection follow the geometry of the features.
A detector that has to track new generators has to keep training. This run splits "average accuracy held" from "each group is still covered". SOYO's final average near 93% is close to retraining on the full history, and its skin-tone accuracy gap is larger than naive fine-tuning. Replay keeps old generators and also writes the age skew into a 3,000-slot buffer.
FOAE and FDP need to be read together. A single ratio can hide a rise in false accusations. Any fix has to match the method family, and it cannot assume gender or skin-tone labels at train time. Those attributes are sensitive, and this training setup leaves them out on purpose. There is no new algorithm here. Continual learning does not drag fairness up along with accuracy.
The ask is that later methods put social outcomes into the stability-plasticity tradeoff: weights have to retain old generators and still absorb new ones. No procedure is given. The four fairness definitions conflict, so a claim that fairness improved has to name which definition and which group pays. Zero-shot detection of an unseen generator is unsolved for every method here, the all-data upper bound included.
The headline table is one test set, the first generator, after 11 updates. Prompt methods pick up fairness on later tasks (8 through 10) and lose more on tasks 2 and 6. Reading the result as "unfairness rises at every step, on every generator" overclaims that table.
The buffer is fixed at 3,000 against 424,452 DCFace training images, with no size sweep, so empty slots and the selection rule are tied together. The backbone is only ViT-B/16, and EWC's λ is a single setting of 5000. There is no head-to-head with continual learners that optimize a fairness term. The negative result covers the usual accuracy-oriented methods.
Real test faces for VQGAN, Palette, and DCFace are reused from earlier tasks, so the real-face distribution repeats. External training reals, including CelebA and MetFaces, have no demographic labels and drop out of the fairness eval, which lets the train mix and the test mix diverge. Dates are dataset release dates and can lag the generator paper. The Fisher and routing plots are observational. The mismatch with headcount points at representation learning. Nothing in the paper changes the loss and checks whether the gap moves.