Cards / needs / Generative Artificial Intelligence (GenAI)
Generative Artificial Intelligence (GenAI)
Self-hosted alternatives to ChatGPT.
What people say
- chat with AI privately
- ask questions about my own documents
- run AI models without sending data away
- one API for several models
- run open models on my own GPU and use them from anywhere
- an OpenAI-compatible API for my own apps
Why people self-host it
People leave ChatGPT, Claude.ai for private documents, hosted LLM APIs for it. Why, strongest first: privacy, cost, control.
Does it suit a machine of your own?
Fits, with conditions local models need a strong machine and answer slower than hosted ones; small models run on CPU
Questions that decide it
- what hardware do you have?
- chat, documents, or an API for your own apps?
How to publish it
- Usually
- the chat on org; the model API on api-key
- When the data is private or sensitive
- org and api-key only
- When the data is regulated
- keeping documents and prompts off outside AI services is a strong reason to run models locally; still check the professional rules, have answers cite their source, and have a person check them before relying on them
Who sees what inside it
knowledge bases and groups decide which documents each person's questions can draw on; with need-to-know rules, such as a law firm's ethical walls, one shared knowledge base is wrong
The machine
much memory; a GPU or Apple Silicon for anything but small models
For builders
a private model and document chat for a prototype, so early users' data does not go to an outside AI service
Cards for it
- assistantChat that answers from your own documents, and the model that runs itnot tested yet
- litellmMetered model API and its admin, one LiteLLM processnot tested yet
Category from awesome-selfhosted.