GANFS scores 99.99% DDoS accuracy but only ties classic Chi-Square filter

Feature Selection via GANs (GANFS): Enhancing Machine Learning Models for DDoS Mitigation

Harsh Patel

cs.CR, cs.AI, cs.LG

2025-04-22

GANFS trains a GAN on attack traffic, then ranks features by how much each perturbation moves the discriminator; it reaches 99.99% on CIC-DDoS2019 but only ties Chi-Square.

What problem this solves

A DDoS attack floods a target with so much traffic that its service goes down. To tell attack traffic apart from benign, network-flow datasets break each flow into dozens of statistics (packet lengths, timing, TCP flag counts), and CIC-DDoS2019 carries more than 80 of them. Many are redundant or irrelevant, and they slow detection, invite overfitting, and quietly drag accuracy down.

Feature selection is the act of keeping only the dimensions that matter. The three classic families each have a weak spot. Filter methods (Chi-Square, ANOVA, mutual information) score features by statistical relevance; they are fast but blind to how features interact. Wrapper methods such as recursive feature elimination (RFE) retrain a classifier over many subsets; they are accurate but expensive. Embedded methods (Random Forest importance) sit in between. The paper wants an unsupervised route that still scales.

The honest framing: cheap statistical methods already handle this problem well, so any newcomer has to win somewhere else.

Method

GANFS turns the GAN's discriminator into a feature-quality oracle, and it does this without classification labels.

Training uses only attack traffic (label = 1). A two-layer MLP generator (64/128 neurons) turns noise into synthetic attack samples, and a two-layer MLP discriminator (128/64) learns to tell real from fake. It trains for 500 epochs with batch size 4096, the Adam optimizer, and binary cross-entropy. Once done, the generator is thrown away; only the discriminator survives.

The key step is perturbation-based sensitivity analysis. For each feature, the method perturbs it bidirectionally at magnitudes {0.5, 1.0, 2.0, 5.0, 10.0} and measures how much the discriminator's confidence on real attack samples moves. Features that move the confidence most are the ones the discriminator leans on hardest to call a sample a real attack, so they rank highest. All 81 features get a score, and the top-k feed a downstream classifier (logistic regression or Random Forest).

The logic holds up. To separate real from fake attacks, the discriminator must lock onto the features that best characterize attacks, so perturbing those features hits its confidence hardest. The score needs no labels and avoids the retraining loop of wrapper methods. That is the paper's real contribution.

The top-ranked features are URG Flag Count, Protocol, Inbound, Bwd Packet Length Max, and Down/Up Ratio, mostly TCP flag and protocol features, which matches what reflection and amplification DDoS should look like.

Results

The dataset is CIC-DDoS2019, sampled from about 28 million records down to roughly 4 million, with 81 features after preprocessing. GANFS is compared against mutual information, Chi-Square, ANOVA, RFE, and Random Forest importance, with logistic regression and Random Forest as downstream classifiers.

Metrics under the Random Forest classifier:

MethodAccuracyF1AUC-ROC
GANFS99.9995%99.9997%0.99999999
Mutual information99.9998%99.9999%0.99999
ANOVA99.9996%99.9998%0.99976
Chi-Square99.9995%99.9998%0.99976

With logistic regression, GANFS lands at 99.954% accuracy, 99.9769% F1, and 0.99976 AUC.

One thing to read carefully: GANFS ties the baselines; it does not beat them. On accuracy it is marginally below mutual information (99.9998%) and ANOVA (99.9996%). The paper writes 'comparable' and 'higher' in the same breath, but its own numbers do not support 'higher.' Every method sits above 99.99%, and the gaps live in the fifth and sixth significant figures. Because the dataset is overwhelmingly attack-heavy, that band is reachable by almost anything, and the gaps read as noise rather than an edge.

The result the paper really wants to sell is training efficiency, claiming GANFS is 'most efficient in training time.' But the actual seconds appear only in figures, not in any number the text lets you check.

Why it matters

For practitioners, the value is not the accuracy number, which is a tie. It is the recipe: an unsupervised feature ranking that needs only attack samples, no benign labels, and no repeated classifier retraining. That matters when you have plenty of attack captures but few clean labeled flows, or when the feature set keeps shifting and you want to avoid the wrapper retraining tax.

As a drop-in replacement for Chi-Square on CIC-DDoS2019, there is no accuracy case for switching today. Whether the unsupervised angle pays off on harder, lower-signal datasets is something this paper never tests.

This is a method-idea paper, not a state-of-the-art-results paper.

Limitations

What the authors concede: GAN training is compute-heavy and slow (an awkward clash with the efficiency pitch); feature rankings wobble slightly across runs, though the top features stay stable; the pipeline is offline, so changing the feature set forces downstream retraining and rules out real-time high-availability use; only binary classification (DDoS vs benign) is covered, though the dataset has 12 attack types; and the benchmark classifiers run on untuned defaults.

What stands out on a close read: the 'outperforms' framing is not backed by the paper's own numbers, with accuracy gaps in the sixth decimal and partly in the wrong direction; there are no repeated runs or error bars on roughly 4 million highly imbalanced samples, even though the paper admits the GAN is stochastic; the training-time numbers live only in figures, so the one result that could differentiate the method is asserted rather than shown; and the dataset is so nearly separable that any feature selector reaches 99.99%, which means this benchmark cannot tell methods apart.

The 'unsupervised' label is also generous. Training the GAN still requires labeled attack samples; what gets dropped is the benign label and the per-subset retraining, not supervision entirely.

Terms

Source

What people are saying

Related papers

All paper explainers