DK · Senior Software Engineer
OfflineAI
Personal project
A local-first research workstation. Chat, documents, and generated files stay on the machine, with a React interface in front of FastAPI and Ollama.
I designed and built it. Ongoing. Public source. It runs locally. There is no hosted demo, on purpose.
View the repository
Overview
OfflineAI is a workstation for talking to local models, researching a topic, and keeping what you find. The README describes a React and TypeScript frontend, a FastAPI service, and Ollama for inference. Conversations stay in the browser. Research projects are plain files on disk.
The problem
Hosted assistants are convenient, and they also mean the conversation, the documents, and the follow-up questions live on someone else's computer. I wanted a tool I could use for real research without making that trade.
Architecture
The interface is a Vite app in React 18 and TypeScript. FastAPI serves that UI, proxies Ollama, and streams chat. The same service runs web search, page reading, document and code generation, audio transcription, and image generation. Conversations are stored in IndexedDB, with a localStorage fallback documented in the project. An active research project injects saved findings back into the chat. Those projects live as folders on disk, with a knowledge file, notes, sources, and generated output. Web search goes through DuckDuckGo and is off until you enable it. The server binds to localhost unless you opt in to LAN access, which then requires a token.
Decisions
- Inference stays on Ollama, on the machine, instead of a hosted model API.
- The UI and the Python service are separate, so token streams, research steps, and file work can run on the server without pretending the browser can do all of it.
- Research is stored as files. A chat log is a poor place to keep sources you may want again.
- Syntax-highlighting assets are vendored. The README calls out that the running app does not depend on a CDN for that.
- Generated tools are checked for dangerous patterns, and project file routes guard against path traversal. Request bodies are size-limited.
Trade-offs
- Local models need disk and a machine that can run them. The installer pulls multi-gigabyte weights. That is the cost of not sending the prompt away.
- Image generation, Whisper transcription, and multi-step research make the service broad. The API surface is large for a personal project, which is powerful and also more to maintain.
- LAN mode is useful on a home network and is a security boundary. It stays opt-in, with a token for anything that is not localhost.
Where it stands
The repository documents the installer, the API, the on-disk layout, and tests: pytest around the service, React unit tests, and a Playwright suite. I treat those as part of the project, not an afterthought. I have not published usage numbers, and I am not going to invent any.
Stack
- React
- TypeScript
- Vite
- FastAPI
- Python
- Ollama
- IndexedDB
- DuckDuckGo search
Back to lekker.dev