MediaCrawler hits 67k stars: one tool scrapes Xiaohongshu, Douyin, Bilibili, Weibo
sven_ai · x · 2026-10-20
MediaCrawler, an open-source crawler with 67.3k stars and 12.8k forks, supports Xiaohongshu, Douyin, Kuaishou, Bilibili, Weibo, Tieba, and Zhihu — covering posts, comments, keyword search, creator profiles, media downloads, and comment word clouds, with a WebUI that requires no command line.
Technically it's built on Playwright: it reuses logged-in browser contexts to obtain signature parameters, avoiding the need to reverse-engineer each platform's encryption. Launched in 2023 and actively maintained, it ships with a disclaimer that it's for learning only, with commercial and illegal use prohibited.
More from coding & agent
- Claude cut a 2.6-hour video export to 20 minutes — but only after being told to rebuild the server — byhewar · 2026-10-23
- 3 flight-booking rules my AI agent checks that I'd never do by hand — Efistoffeles · 2026-10-23
- OpenJet claims Qwen3.8-27B coding runs 1.9x faster than llama.cpp on one RTX 3090 — JeremyCMorgan · 2026-10-23
- Building an autonomous SEO agent in Hermes with three skills: programmatic pages, Keywords Everywhere, DataForSEO — EXM7777 · 2026-10-23
- Building a sports betting AI agent with onchain execution and human approval by default: architecture and lessons — Dry-Comfortable-2678 · 2026-10-23
- How do you audit AI agent decisions in regulated workflows? A field survey — Ok_Way_1993 · 2026-10-23