Structured RAG Dataset for EU AI Act Released
Automatic-Forever-63 · reddit · 2026-07-17
The author released EU AI Act OpenRAG, a downloadable dataset for RAG and legal NLP experiments.
Dataset Design
- Instead of splitting by fixed character windows, it chunks by the legal structure of the regulation:
- One chunk per paragraph
n - One chunk per recital
- One chunk per Article 3 definition
- One chunk per annex entry
- Metadata for chapters, sections, and provisions are saved separately.
- The database contains 933 chunks, with a 1024-dimensional BGE-M3 embedding for each.
- It also includes exact EUR-Lex links, Article 113 effective date metadata, and intentionally narrow derived labels.
Evaluation Results
The author compared it on the AI Act Evaluation Benchmark:
- Scenario article recall@20: 0.541 (structured chunking) vs 0.449 (baseline)
- QA article hit@10: 0.927 vs 0.898
- However, overall RAG classification performance was slightly lower. The author believes this suggests that generator behavior might be more critical than chunk granularity in such tasks.
Other Information
The author also made public:
- All experimental results
- Limitations
- Derivation methods
- Label auditing
- Licensing notes
They invite technical feedback on retrieval evaluation, structured chunking methods, and more appropriate baselines.
More from Safety
- AI Security Institute says every tested model tried to cheat in cyber evaluations — connoraxiotes · 2026-07-21
- Congressional brief warns AI could speed biology research while creating new biosecurity risks — sebkrier · 2026-07-21
- AI Companies Are Buying Tons of Old Books Because They're Free of AI Slop — 404 Media · 2026-07-21
- A simple standup question exposes who owns AI model approval in customer workflows — YvesMulkers · 2026-07-21
- Anthropic says frontier models showed harmful behavior in tool-rich simulations — gerardsans · 2026-07-21
- Cisco releases Antares small models to localize code vulnerabilities — aminkarbasi · 2026-07-21