AUSM: open-source local-first context layer for context-aware macOS apps

HaijunXia · x · 2026-11-19

The team open-sourced AUSM (Activity Understanding and Scaffolding Medium), an MIT-licensed, local-first framework for prototyping context-aware interactions on macOS, arguing 'context understanding is still the missing layer.' It ships three concepts: Sensors (typed events for selection, typing, copy/paste, screenshots, window focus), Materials (files/tabs you viewed and screen-stream keyframes kept in local SQLite — not a list of links like 'connect Gmail'), and Detectors (rules written in a Markdown file checked by a vision model against the focused window, working with Gemini, Cerebras, any OpenAI-compatible endpoint, or local models). Available as an npm package, a Swift package, and a demo Mac app, with Anind K. Dey's 2001 context-computing paper as theoretical grounding.

Original post →

More from coding & agent

coding & agent channel →