Files
docs/guides/local-chat-apps.md
T
bengansky 1827f7c55e Add guide: using the co-op privately from a local chat app
Walk-through for going from zero (no IC account/membership) to using IC-provided models on your own device (desktop or phone) via local apps.
2026-11-19 20:06:58 +00:00

224 lines
14 KiB
Markdown

# Using the co-op privately on your phone or computer
## Index
- [Why use a local app](#why-use-a-local-app)
- [Which app should I use?](#which-app-should-i-use)
- [What you need](#what-you-need)
- [Part 1. Join the co-op (everyone)](#part-1-join-the-co-op-everyone)
- [Step 1. Become a member](#step-1-become-a-member)
- [Step 2. Create your co-op login](#step-2-create-your-co-op-login)
- [Step 3. Make your API key](#step-3-make-your-api-key)
- [Part 2A. Jan on a computer](#part-2a-jan-on-a-computer)
- [4. Install Jan](#jan-step-4-install-jan)
- [5. Connect Jan to the co-op](#jan-step-5-connect-jan-to-the-co-op)
- [6. Turn on tool access](#jan-step-6-turn-on-tool-access)
- [7. Say hello](#jan-step-7-say-hello)
- [Part 2B. Chatbox on a phone or computer](#part-2b-chatbox-on-a-phone-or-computer)
- [4. Install Chatbox](#chatbox-step-4-install-chatbox)
- [5. Connect Chatbox to the co-op](#chatbox-step-5-connect-chatbox-to-the-co-op)
- [6. Turn on images and tools](#chatbox-step-6-turn-on-images-and-tools)
- [7. Say hello](#chatbox-step-7-say-hello)
- [Which model should I use?](#which-model-should-i-use)
- [Keeping an eye on your allowance](#keeping-an-eye-on-your-allowance)
- [If something goes wrong](#if-something-goes-wrong)
## Why use a local app
*A step-by-step walkthrough for members who aren't developers/super technical. Whole process takes about 15 minutes. Drafted for [co-op/support#56](https://git.inference.coop/co-op/support/issues/56).*
The co-op's web chat keeps your conversation history on the co-op's server. If you'd rather keep that history on your own phone or computer, you can talk to the same co-op models from a chat app installed on your device. Any app that accepts a custom "OpenAI-compatible" connection will work. This guide uses two free examples: **Jan** for computers and **Chatbox** for phones and computers.
> **What changes, and what doesn't**
>
> - **Changes:** your chat history is saved on your device, not on the co-op's server.
> - **Doesn't change:** every message still travels through the co-op's gateway to the model provider (Tinfoil, GreenPT or Regolo), exactly as in the web chat. The co-op records usage (which model, how many tokens, the cost), not what you wrote.
> - **You give up:** the web chat's document search and image tool, and history that syncs between your devices. Each device keeps its own chats.
### Which app should I use?
| | Jan | Chatbox |
| --- | --- | --- |
| Runs on | Mac (Apple silicon), Windows, Linux | iPhone, iPad, Android, Mac, Windows, Linux |
| Connecting to the co-op | Finds the co-op's models for you | You may need to type model names in |
| Running open models on your own computer | Built in | Needs a separate app |
| Cost | Free, no paid plans | Free with your co-op key; also sells its own optional plans |
| Open source | Yes, Apache 2.0 | Community Edition is GPL-3.0 |
**Rule of thumb:** on a phone, use Chatbox. On a computer, either works; Jan is a little easier to connect.
### What you need
- A phone, tablet or computer from the table above
- An email address
- A card or account you can pay dues with on Open Collective
- About 15 minutes
## Part 1. Join the co-op (everyone)
### Step 1. Become a member
1. Go to [inference.coop](https://inference.coop) and click **Become a member**.
2. Open Collective opens with $5/month filled in. Dues are a sliding scale: pay $5, or more if you can. Every member gets the same $15 of usage each month, whatever they pay.
3. Finish the checkout.
> **Important:** use the same email address here that you'll use for your co-op login. That's how the co-op knows you're a member.
### Step 2. Create your co-op login
1. Within a few minutes you'll get a **welcome email**. Check spam if it doesn't arrive.
2. Click the link in it and create your username and password.
You can now log in to the web chat and the Member Dashboard.
### Step 3. Make your API key
An **API key** is a long password that lets an app use your membership. It starts with `sk-`.
1. Go to [dashboard.inference.coop](https://dashboard.inference.coop) and log in.
2. Find the API keys section and create a new key. Give it a name you'll recognise, like `Jan_on_my_laptop` or `Chatbox_on_my_phone`. A separate key for each device lets you switch one off without affecting the others.
3. **Copy the key now** and keep it somewhere safe, such as your password manager.
> **Treat the key like a password.** Anyone who has it can spend your monthly allowance. Don't paste it into email, chat or shared documents. If it leaks, delete it in the dashboard and make a new one.
Now continue with [Part 2A for Jan](#part-2a-jan-on-a-computer) or [Part 2B for Chatbox](#part-2b-chatbox-on-a-phone-or-computer).
## Part 2A. Jan on a computer
### Jan step 4. Install Jan
1. Go to [jan.ai](https://jan.ai) and download Jan for your computer.
2. **Mac:** open the downloaded `.dmg` file and drag Jan into Applications. **Windows:** run the `.exe` installer. **Linux:** use the Flatpak (`flatpak install flathub ai.jan.Jan`) or the `.deb`/AppImage.
3. Open Jan. On first launch it asks whether to share anonymous usage data. It's off unless you say yes; choose whichever you prefer.
Jan is free and open source (Apache 2.0). You don't need a Jan account.
### Jan step 5. Connect Jan to the co-op
1. In Jan, open **Settings** (the gear), then **Model Providers**.
2. Click **Add Provider**.
3. Fill in the **Add Custom Provider** box exactly like this:
| Field | What to enter |
| --- | --- |
| API format | **OpenAI-compatible** |
| Provider name | `Inference Co-op` |
| Base URL | `https://gateway.inference.coop/v1` |
| API key | your `sk-…` key from Step 3 |
4. Click **Create**.
Jan now asks the co-op which models you can use and lists them under the new provider. If the list is empty, click the **refresh** icon next to **Models**.
> **Check the Base URL.** It must end in `/v1`, with nothing after it. Don't add `/chat/completions`.
### Jan step 6. Turn on tool access
Jan can't tell what each co-op model is able to do, so it treats every co-op model as plain chat. Until you switch on **Tools**, a model can't use Jan's web search or other tools, even when it wants to. When that happens, the reply can get stuck repeating "Let me search the web…" over and over.
1. Go to **Settings → Model Providers → Inference Co-op**.
2. Click the **pencil** next to a model you plan to use.
3. Under **Capabilities**, switch on **Tools**. Switch on **Vision** too if the model can read images.
4. Repeat for each model you use. The [model table](#which-model-should-i-use) shows which models support what.
### Jan step 7. Say hello
1. Start a **New Chat**.
2. Click the model name above the message box and choose a model under **Inference Co-op**. Not sure? Start with `deepseek-flash-green`, the web chat's default. A **tools** label next to a model means step 6 worked.
3. **Optional: let it search the web.** Click the **globe icon** in the toolbar under the message box. It reads **Web Search (Active)** when on. Only models with Tools switched on can use it.
4. Type a message and press Enter.
If an answer comes back, you're done. Your conversation is saved on your computer, in Jan.
> **Where your searches go.** Jan's web search doesn't run through the co-op. By default it uses a search service called Exa, which needs no key. You can choose a different service in **Settings → Web Search**.
## Part 2B. Chatbox on a phone or computer
### Chatbox step 4. Install Chatbox
1. **iPhone or iPad:** search for *Chatbox AI* in the App Store. **Android:** search in Google Play, or download the APK from [chatboxai.app](https://chatboxai.app). **Mac, Windows, Linux:** download the installer from [chatboxai.app](https://chatboxai.app).
2. Open Chatbox.
3. If it offers to sign you up for **Chatbox AI**, skip it. That's Chatbox's own paid model service. You don't need an account or a plan to use your co-op key.
> **Two kinds of AI in one app.** Chatbox sells its own models (Lite, Pro and Pro+ plans). This guide uses only the co-op's models, paid for by your membership. When you pick a model, make sure it's one listed under your co-op provider.
### Chatbox step 5. Connect Chatbox to the co-op
1. Open **Settings**, then **Model Provider**. On a phone, Settings is in the side menu.
2. Scroll to the bottom and tap **Add**.
3. Fill it in exactly like this:
| Field | What to enter |
| --- | --- |
| Name | `Inference Co-op` |
| API mode | **OpenAI API Compatible** |
| API Host | `https://gateway.inference.coop/v1` |
| API Path | leave empty |
| API Key | your `sk-…` key from Step 3 |
4. Tap **Check** next to the API Key. It should report a successful connection.
5. Tap **Fetch** to load the co-op's models, then select the ones you want to use. If Fetch doesn't work, add a model by hand by typing its name exactly, e.g. `deepseek-flash-green`.
6. Save.
> **Check the API Host.** It ends in `/v1`, with nothing after it. Don't add `/chat/completions`.
### Chatbox step 6. Turn on images and tools
Chatbox treats a model as text-only until you tell it otherwise.
1. Go back to **Settings → Model Provider → Inference Co-op**.
2. Open a model's settings.
3. Turn on the capabilities it supports, such as **vision** for reading images and **tool use**. The [model table](#which-model-should-i-use) shows which models support what.
### Chatbox step 7. Say hello
1. Go back to the main screen and start a **new chat**.
2. Tap the model name and choose a model under **Inference Co-op**. Not sure? Start with `deepseek-flash-green`, the web chat's default.
3. Type a message and send it.
If an answer comes back, you're done. Your conversation is saved on this device, in Chatbox.
**Using more than one device?** Repeat these steps on each one, ideally with its own key. Chats don't move between devices.
## Which model should I use?
All co-op models draw from the same $15 monthly allowance. They differ in **privacy**, **energy** and **cost**. If you're adding models by hand, copy these names exactly.
| If you care most about… | Use this alias | What it is | Vision |
| --- | --- | --- | --- |
| A good all-rounder, renewable energy | `deepseek-flash-green` | The web chat's default model. Policy-based privacy. | ✓ |
| The same model with strong privacy | `deepseek-flash-tee` | Runs inside attested, encrypted hardware (confidential computing). Supports tools. | ✓ |
| Fast and inexpensive, renewable energy | `glm-flash-green` | A quick GLM model. Policy-based privacy. | |
| The strongest model, with strong privacy | `glm-tee` | GLM flagship in confidential computing. | |
These are the co-op's **stable aliases**. When a provider retires or replaces a model, the co-op points the alias at its successor, so your app keeps working without changes. The `-green` aliases run on renewable energy; the `-tee` aliases run the same model family in confidential-computing hardware. A blank Vision cell means the co-op doesn't document it. If you want a specific model instead, such as Regolo's zero-retention models, use its full name from the [models page](https://docs.inference.coop/guides/models/).
> **Which models can use tools (web search and other add-ons)?** Switch on Tools only for models that support it.
>
> - **Documented by the co-op:** `deepseek-flash-tee` (`tinfoil/deepseek-v4-1-flash`), `regolo/gemma4-31b` and `regolo/mistral-small-4-119b`.
> - **Worked in a member's test in Jan:** `greenpt/glm-5.3`, the strongest model in the green tier.
> - **Not documented yet:** `deepseek-flash-green`, `glm-flash-green` and `glm-tee`. Try them with Tools on; if you get errors or odd replies, switch Tools off for that model.
## Keeping an eye on your allowance
The [Member Dashboard](https://dashboard.inference.coop) shows how much of your $15 is left this month. Neither app shows it. The allowance resets on the day of the month you joined. If you run out, you can buy co-op credit packs, which never expire. You don't need Chatbox's own plans.
## If something goes wrong
| What you see | App | What it usually means | Fix |
| --- | --- | --- | --- |
| Error **401** or **403**, or Check fails | Both | The app isn't sending a valid key | Re-paste the key in the co-op provider's settings. Check there are no spaces at either end. |
| Error **404** on every message | Jan | The Base URL is wrong | Make it exactly `https://gateway.inference.coop/v1`. |
| Error **404** on every message | Chatbox | The API Host is wrong | API Host `https://gateway.inference.coop/v1` with API Path empty. If that still fails, try API Host `https://gateway.inference.coop` with API Path `/v1/chat/completions`. |
| No models listed | Jan | Jan couldn't fetch the list | Click the refresh icon next to Models, or click **+** and type a model name, e.g. `deepseek-flash-green`. |
| Fetch finds no models | Chatbox | Chatbox couldn't load the list | Add a model by hand, e.g. `deepseek-flash-green`. |
| "Model not found" | Both | A typo in the model name | Copy the name from the model table. |
| Messages fail straight away | Both | Wrong API format | Edit the provider and choose OpenAI-compatible. |
| The reply keeps repeating "Let me search…" or shows stray `</think>` tags | Both | The model is trying to use a tool it hasn't been given | Stop the reply, then switch on Tools for that model (Jan step 6 or Chatbox step 6) and try again. |
| Replies come from a model you didn't expect | Chatbox | You picked a Chatbox AI model | Choose a model listed under **Inference Co-op**. |
| A message says your balance is out | Both | You've used this month's $15 | Wait for your reset date, or buy a co-op credit pack. |
Still stuck? Ask in the co-op's Skillshare room on Matrix, open an issue on the [support tracker](https://git.inference.coop/co-op/support/issues), or email info@inference.coop.
*Written against Jan 0.8.8's documentation (October 2026), Chatbox's documentation and a third-party Chatbox setup guide (Vercel, September 2026). The Jan steps have been tested by a member; the Chatbox steps haven't yet. Menu names can change between app versions and between phone and desktop.*