FRAUDSkill: Structured Frozen-Weight Skill Optimization for Audio Anti-Fraud Detection
Chengxian Hu, Zhiming Ma, Mingjun Pan, Yifan Wang, Shun Zhang, Qifan Wang, Zhilei Zhao, Yijin Zhou, Yuxi Zhao, Huiyuan Liu, Peidong Wang, Peng Chen
cs.SD, cs.CL
2026-09-16
FRAUDSkill freezes Qwen2-Audio-7B and tunes external skills, hitting 73.50% Macro-F1 on TeleAntiFraud, +31.96 over the shared baseline, invalid outputs 1.94%.
Audio-language models can listen to a call and write a rationale. A deployed anti-fraud stack still needs a closed label set and a fixed decision chain: identify the service scene, decide fraud versus normal, and emit a fraud type only after a positive fraud decision. Free-form generation often gets the meaning and misses the ontology. "Loan service" and "fake loan fraud" sound right, sit outside the official vocabulary, and void the whole chain.
Fine-tuning writes those rules into weights, so a change in scam patterns or labeling policy means another training run. Hand-written prompts move faster, but they rarely guarantee three legal, mutually consistent slots. The missing piece is adaptation that keeps the audio model frozen and still forces open-ended generation into a closed protocol.
FRAUDSkill externalizes task knowledge, label constraints, and routing policy as replaceable skill programs. The audio actor stays frozen. A program has three parts: a root instruction for role, output contract, and official labels; a skill library of fraud cues, type boundaries, and cross-route consistency rules; and a route policy that picks which skills fire at the current step.
Offline, frozen Qwen2-Audio-7B-Instruct rolls full trajectories on a development split. A text critic diagnoses wrong official labels, unmappable outputs, incorrectly skipped type routes, and systematic minority-class errors. An editor revises only the external program. Search uses beam 3, branch 3, and five rounds, then keeps programs by held-out validation. At deployment each retained program walks the scene-fraud-type chain. Deterministic projection maps aliases onto official labels. Route normalization forces type=NA on a normal decision and a valid type on a fraud decision. A class-balanced selector fitted on a disjoint calibration split aggregates the trajectories.
The evaluation unit is a unique audio recording. After dedup, the official TeleAntiFraud SFT split has 10,711 train and 2,677 test samples, 1,453 of them with fraud-type labels. Every frozen-weight method shares Qwen2-Audio-7B-Instruct.
| Method | Macro-F1 | Joint Acc. | Invalid |
| Shared baseline | 41.54 | 7.10 | 36.04 |
| SkillOpt | 37.67 | 5.94 | 29.00 |
| EvoSkill | 39.07 | 6.16 | 34.19 |
| FRAUDSkill-Text | 42.42 | 8.18 | 34.48 |
| FRAUDSkill | 73.50 | 58.87 | 1.94 |
| SFT (reference) | 66.06 | – | – |
| SFT+Memory (reference) | 75.51 | – | – |
SkillOpt and EvoSkill cut invalid outputs and still land below the baseline on Macro-F1. Text-only program search averages 42.42±1.39, a 0.88-point mean gain smaller than seed noise. The full system is structured inference: best text program 43.66, closed-set projection 54.47, route normalization 66.21, multi-path 70.31, reliability weighting 70.84, class-balanced selection 73.50. W-F1 is 79.40 and accuracy 78.72. The frozen stack beats plain SFT and sits slightly under SFT+Memory.
FRAUDSkill-Text scene error is 74.2%, and 91.1% of those errors are missing or off-ontology. Fraud-route error is 32.9%, 60.0% of it false-normal. Type-route error on annotated cases is 83.1%.
For a team that must revise fraud-type vocabularies on a schedule and cannot fine-tune a 7B audio model each time, this is the more maintainable path. Almost all of the 32-point jump comes from projection, route constraints, and the selector, not from prettier prompts. Generic skill evolution is not enough here: cleaner format can coexist with worse class-balanced discrimination.
It is still an incremental system. The protocol is wired to TeleAntiFraud's three-stage chain. A new ontology needs new projection and normalization rules. This is not a drop-in audio classifier.
Text-layer search is unstable: the three-seed Macro-F1 standard deviation is 1.39, larger than the mean gain over the baseline. The 73.50 figure depends on validation-selected programs and a fitted selector. Test labels are isolated, yet the discrete selector family can still overfit the calibration split. The SFT rows omit W-F1, invalid rate, and joint accuracy, so the comparison is incomplete. Audio-level dedup also makes these numbers incomparable to the original dataset paper's 7,021 interaction records. The actor is fixed to Qwen2-Audio-7B-Instruct; there is no backbone swap.