Local inference in practice: mining repos, AI chat logs and media libraries with a tiny model

natesiggard · x · 2026-10-05

The author shares three practical ways he uses a small local model for offline inference:

Mining code repositories: walk through all his repos to identify where he has invested the most effort and what he actually cares about;
Post-mortem on AI sessions: archive all his Claude, GPT, and Cursor sessions, then have the model surface recurring patterns of intent and where the models repeatedly failed;
Media library tagging: extract people, places, and keywords from his media assets.

The takeaway: local inference's privacy and zero marginal cost make it a good fit for batch-processing private data you wouldn't send to the cloud.

More from coding & agent