5 open-source repos to build your own local AI apps: Ollama, llama.cpp, Jan and more
Shruti_0810 · x · 2026-10-10
A thread recommends 5 open-source projects for running AI on your own machine instead of relying on the cloud:
- Ollama: simple local LLM setup for experimenting and building apps
- Open WebUI: self-hosted chat interface that connects to multiple AI backends
- llama.cpp: run LLMs efficiently across hardware, even without high-end GPUs
- LocalAI: self-hosted AI services with OpenAI-compatible APIs and multi-model support
- Jan: a desktop AI assistant for private, offline model use
More from Infra
- Lemire reruns 2026 WebSocket benchmarks: his old Bun-vs-Node.js result was wrong, Anthropic bought Bun and Cloudflare bought Deno — lemire · 2026-10-10
- Cornell/IBM paper: shared KV cache cuts looped-transformer memory 76-79% while improving quality — yoavartzi · 2026-10-10
- Cascade GPU Topology Can Be Slower: The PCIe Hop Trap in Multi-GPU P2P — TheZachMueller · 2026-10-10
- Custom CUDA Megakernel Hits 140 tok/s on Qwen3.8-27B With a Single RTX 3090 — Adorable_Weakness_39 · 2026-10-10
- HK GPU market check: 5090 rigs near HK$90k, 5090 discontinuation rumor gains credence — karminski3 · 2026-10-10
- Orbio raises $1.2M to build rails for agents to provision their own compute — econoar · 2026-10-10