Compare commits
64
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
87e0a085f4 | ||
|
|
5605f32527 | ||
|
|
3cd441999b | ||
|
|
d14395e4ec | ||
|
|
64c2cbcc1e | ||
|
|
ab96402d15 | ||
|
|
0a0a66ae28 | ||
|
|
33a9e3e2a4 | ||
|
|
1e91cfcd1e | ||
|
|
f23513a41f | ||
|
|
a3fabdd53c | ||
|
|
c9a6e23154 | ||
|
|
12da5c1268 | ||
|
|
1f3f0ce9b8 | ||
|
|
fde5d143c5 | ||
|
|
0e708a99c6 | ||
|
|
467d19357d | ||
|
|
6a0d092e53 | ||
|
|
bac352f50e | ||
|
|
34a23a34f3 | ||
|
|
fa979a5ed9 | ||
|
|
5bb99b8592 | ||
|
|
e7c86b16ba | ||
|
|
d3c3c8f8f7 | ||
|
|
884340427f | ||
|
|
9699ff08cc | ||
|
|
0d4a62a2da | ||
|
|
f3a757c9a5 | ||
|
|
196a3387ad | ||
|
|
7ea5e1a4a1 | ||
|
|
1e57fd8f00 | ||
|
|
af2b6eacd0 | ||
|
|
83942f7bd0 | ||
|
|
fd2cf4089a | ||
|
|
67a69dc8b6 | ||
|
|
d047ced4c6 | ||
|
|
d0ec489af7 | ||
|
|
ccebba45af | ||
|
|
b9953ed086 | ||
|
|
4f995be9f1 | ||
|
|
fed58a3437 | ||
|
|
0f657b7cf4 | ||
|
|
a3defe1bed | ||
|
|
e175f07cb5 | ||
|
|
a5e33b73dd | ||
|
|
21b31b3a0a | ||
|
|
9d520130f4 | ||
|
|
7b0b13a54a | ||
|
|
bed53d6cbd | ||
|
|
121afa101a | ||
|
|
d865501fbf | ||
|
|
adabc8bccd | ||
|
|
3c346e420f | ||
|
|
7313f821e3 | ||
|
|
d8bd033f05 | ||
|
|
c022a0b52e | ||
|
|
85f5723357 | ||
|
|
a647284170 | ||
|
|
888ac8c299 | ||
|
|
dcdde204b0 | ||
|
|
0a79c80483 | ||
|
|
d2e90c6dc3 | ||
|
|
7e2fa8c0a3 | ||
|
|
85a6045389 |
No files matched your search
@@ -11,8 +11,10 @@ funded through our [Open Collective](https://opencollective.com/inference-cooper
|
|||||||
## Start here
|
## Start here
|
||||||
|
|
||||||
- **New here?** → [Getting started](getting-started.md) — join, use the chat, and get involved in governance.
|
- **New here?** → [Getting started](getting-started.md) — join, use the chat, and get involved in governance.
|
||||||
|
- **Why does this exist?** → [Background](background.md) — a one-page summary of the mission and history.
|
||||||
- **Which models can I use?** → [Models](models.md) — the full catalog of providers and models.
|
- **Which models can I use?** → [Models](models.md) — the full catalog of providers and models.
|
||||||
- **Want API access?** → [API access](api-access.md) — use our models programmatically.
|
- **Want API access?** → [API access](api-access.md) — use our models programmatically.
|
||||||
|
- **Want community chat?** → [Getting started · Matrix](getting-started.md) — our federated Matrix server, including how to connect from any client and how to use an existing Matrix account.
|
||||||
- **Curious how it works?** → [Infrastructure](infrastructure.md) — a map of the stack and the codebase.
|
- **Curious how it works?** → [Infrastructure](infrastructure.md) — a map of the stack and the codebase.
|
||||||
- **Need help or have an idea?** → [Support & feature requests](https://git.inference.coop/co-op/support) — open an issue.
|
- **Need help or have an idea?** → [Support & feature requests](https://git.inference.coop/co-op/support) — open an issue.
|
||||||
|
|
||||||
@@ -20,6 +22,7 @@ funded through our [Open Collective](https://opencollective.com/inference-cooper
|
|||||||
|
|
||||||
- **Website:** https://inference.coop
|
- **Website:** https://inference.coop
|
||||||
- **AI Chat:** https://chat.inference.coop
|
- **AI Chat:** https://chat.inference.coop
|
||||||
|
- **Matrix chat:** https://matrix.inference.coop
|
||||||
- **Member Dashboard:** https://dashboard.inference.coop
|
- **Member Dashboard:** https://dashboard.inference.coop
|
||||||
- **Governance forum:** https://forum.inference.coop
|
- **Governance forum:** https://forum.inference.coop
|
||||||
- **Source code:** https://git.inference.coop
|
- **Source code:** https://git.inference.coop
|
||||||
|
|||||||
+3
-4
@@ -57,7 +57,7 @@ endpoint configuration:
|
|||||||
- **API type / protocol:** "OpenAI compatible" (or `openai-completions` where a
|
- **API type / protocol:** "OpenAI compatible" (or `openai-completions` where a
|
||||||
protocol string must be named) — not "Ollama", "Anthropic", or "Responses"
|
protocol string must be named) — not "Ollama", "Anthropic", or "Responses"
|
||||||
- **API key:** your member key (the `sk-…` string)
|
- **API key:** your member key (the `sk-…` string)
|
||||||
- **Model:** one from the [model list](models.md), e.g. `tinfoil/deepseek-v4-1-flash` or `greenpt/green-r`
|
- **Model:** one from the [model list](models.md), e.g. `tinfoil/deepseek-v4-1-flash` or `greenpt/glm-5.3-flash`
|
||||||
|
|
||||||
A typical provider entry looks like:
|
A typical provider entry looks like:
|
||||||
|
|
||||||
@@ -71,9 +71,8 @@ A typical provider entry looks like:
|
|||||||
"models": [
|
"models": [
|
||||||
{ "id": "tinfoil/deepseek-v4-1-flash" },
|
{ "id": "tinfoil/deepseek-v4-1-flash" },
|
||||||
{ "id": "tinfoil/gpt-oss-120b" },
|
{ "id": "tinfoil/gpt-oss-120b" },
|
||||||
{ "id": "tinfoil/glm-5-3-flash" },
|
{ "id": "tinfoil/glm-5.3" },
|
||||||
{ "id": "greenpt/green-r" },
|
{ "id": "greenpt/glm-5.3-flash" }
|
||||||
{ "id": "greenpt/green-l" }
|
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,42 @@
|
|||||||
|
# Background
|
||||||
|
|
||||||
|
The Inference Cooperative exists to give people a real alternative for AI: a
|
||||||
|
service owned and directed by the people who use it, rather than rented from
|
||||||
|
an extractive industry. Members pay dues on a sliding scale and receive
|
||||||
|
private AI chat and API access through infrastructure the members govern
|
||||||
|
together.
|
||||||
|
|
||||||
|
We are a small cooperative by design, fiscally sponsored by the nonprofit
|
||||||
|
[Metagov](https://metagov.org) and managing member dues on
|
||||||
|
[Open Collective](https://opencollective.com/inference-cooperative).
|
||||||
|
For how to join and use it, see [Getting started](getting-started.md);
|
||||||
|
for our governance model, see the [Charter](charter.md).
|
||||||
|
|
||||||
|
## How we got here
|
||||||
|
|
||||||
|
The cooperative launched in September 2026, growing out of research on democratic
|
||||||
|
digital infrastructure through Metagov's [Collective Governance for AI:
|
||||||
|
Points of Intervention](https://metagov.org/cg-ai/).
|
||||||
|
|
||||||
|
It began with a single privacy-first provider and a deliberately small
|
||||||
|
toolset: a chat, an API gateway, and the basics for membership and payments.
|
||||||
|
From there it grew the way co-ops do: in response to what members asked for.
|
||||||
|
A member dashboard made usage and spending transparent. A federated
|
||||||
|
[Matrix](https://matrix.inference.coop) server, a
|
||||||
|
[forum](https://forum.inference.coop), and a newsletter gave members places to
|
||||||
|
talk to each other and to the project. The
|
||||||
|
code that runs the member-facing services is available in public
|
||||||
|
repositories.
|
||||||
|
|
||||||
|
In October 2026, members adopted the co-op's first
|
||||||
|
[charter](charter.md) — a democratic constitution setting out values, roles,
|
||||||
|
and decision-making, with a commitment to redesign governance as the
|
||||||
|
community grows past 250 members.
|
||||||
|
|
||||||
|
## Where we're going
|
||||||
|
|
||||||
|
The near-term road: more model choices with strong privacy and environmental
|
||||||
|
records, infrastructure hosted with values-aligned providers, and the
|
||||||
|
working-group structure the charter describes. The longer arc: proving that
|
||||||
|
people can govern their AI the way they govern a credit union or a land trust
|
||||||
|
— and sharing what we learn with the co-ops that come after us.
|
||||||
+33
-16
@@ -1,35 +1,52 @@
|
|||||||
# Inference Cooperative Charter
|
# Inference Cooperative Charter
|
||||||
|
|
||||||
A cooperative for AI resources, governed by members.
|
## Preamble
|
||||||
|
|
||||||
|
The Inference Cooperative exists to enable the people who use its AI infrastructure to co-govern it. We came together to build and run an alternative to extractive, profit-driven AI—a service owned and directed by its members, committed to pursuing ethical practices, and governed cooperatively. This charter is our core governing document. It sets out how we make decisions together.
|
||||||
|
|
||||||
## Values
|
## Values
|
||||||
|
|
||||||
The Inference Cooperative seeks to advance accountable AI grounded in democracy, privacy, and sustainability.
|
In addition to the international [cooperative principles](https://www.ica.coop/en/cooperatives/cooperative-identity/), we share these basic commitments:
|
||||||
|
|
||||||
## Code of Conduct
|
- **Participation.** Our service is accountable to the people who use it, and they both can and should have an active role in shaping it.
|
||||||
|
- **Experimentation.** In order to learn together, we will maintain a spirit of exploration and experimentation, and be respectful of all members' levels of knowledge and expertise.
|
||||||
|
- **Sustainability.** We commit, wherever possible, to minimizing harm to natural ecologies and to promoting environmental justice.
|
||||||
|
- **Trustworthiness.** We seek to provide reliable service to one another and protect each other's privacy, in contrast to an industry widely characterized by surveillance.
|
||||||
|
- **Transparency.** The co-op should make its operations and decisions as available as possible to members, and to share our learnings with the wider world.
|
||||||
|
|
||||||
Members must adhere to the [Contributor Covenant](https://www.contributor-covenant.org/).
|
## Roles
|
||||||
|
|
||||||
## Membership
|
- **Members** are the heart of the co-op. All account holders currently [contributing monthly dues](https://opencollective.com/inference-cooperative) are considered members and may participate in decision-making and working groups. Members may be sanctioned or removed for violating co-op rules by the GM or a designated working group.
|
||||||
|
- A **general manager** oversees day-to-day operations and implements member decisions, through their own actions or additional roles that they delegate. The GM is accountable to members and may be replaced by them through a policy decision.
|
||||||
|
- **Working groups** make and carry out operational decisions within their own purview. They should be formally established, modified, and disbanded through a policy decision, and they are obligated, at minimum, to publish to members their membership, meeting schedule, and major activities.
|
||||||
|
- A **spokescouncil** coordinates across working groups, and all working groups are responsible for ensuring that at least one member consistently participates. The spokescouncil meets monthly, and meetings are open to all members.
|
||||||
|
|
||||||
Members are dues-paying users of the service. Join at [opencollective.com/inference-cooperative](https://opencollective.com/inference-cooperative).
|
## Decisions
|
||||||
|
|
||||||
### Deliberation
|
The co-op operates through the practice of modified [policy governance](https://en.wikipedia.org/wiki/Policy_Governance). Member decisions should seek to instill responsibility and trust in role-holders, while guiding them only through goal-setting and necessary constraints. Role-holders should act on the basis of their expertise and the co-op's values, as well as through consultation with members.
|
||||||
|
|
||||||
Members can participate in deliberation about co-op business on Loomio at [forum.inference.coop](https://forum.inference.coop). Decisions are not currently binding, but in the future they will be.
|
**Policy.** Any member may propose a policy in the co-op's [forum](https://forum.inference.coop/). A proposal requires at least one vote from another member and stays open for 7 days. It passes by consent as long as it receives support from at least 60% of voting members. If a member raises a block—a reasoned objection that the proposal would seriously harm the cooperative—it requires 75% support. The proposal's author is always encouraged to revise the proposal based on feedback from other members and restart the process.
|
||||||
|
|
||||||
### Removal
|
**Charter.** Changes to this charter require a proposal open for **14 days**, along with approval by 75% of voting members, regardless of blocks. Policy decisions must not contradict the charter.
|
||||||
|
|
||||||
Members may be removed by the General Manager for violations of the Code of Conduct or failure to contribute dues.
|
Proposals may be adjusted with modest changes during the open discussion period by the proposer, as long as changes are announced in the discussion.
|
||||||
|
|
||||||
## General Manager
|
This governance design is oriented around a relatively small community. Upon reaching 250 members, we commit to revising this charter with a larger community in mind.
|
||||||
|
|
||||||
The General Manager, initially Nathan Schneider, temporarily has full governance responsibilities over the cooperative.
|
## Disputes
|
||||||
|
|
||||||
## Evolution
|
We regard disputes as an opportunity to learn and grow. Members should try to work things out directly with each other first. When necessary, disputes either among members or between a member and the co-op should be brought to the disputes working group—or, if one does not exist, the GM. Disputes that cannot be adequately addressed within the co-op may be escalated to the parent legal entity, which may in turn forward them to the relevant legal jurisdiction.
|
||||||
|
|
||||||
The current governance model will evolve into a true member-governed cooperative upon or before reaching 50 members. The General Manager is responsible for ensuring the transition occurs within two weeks of that milestone. If it does not, the General Manager must resign and transfer the role to someone who does commit to that deadline, or the co-op is dissolved.
|
## Agreements
|
||||||
|
|
||||||
---
|
This charter is the core governing document. Members are also bound by the following agreements:
|
||||||
|
|
||||||
*Created by the [Inference Cooperative](https://inference.coop/).*
|
* [Code of Conduct](https://git.inference.coop/co-op/docs/src/branch/main/code-of-conduct.md)
|
||||||
|
* [Terms of Service](https://git.inference.coop/co-op/docs/src/branch/main/terms-of-service.md)
|
||||||
|
* [Privacy Policy](https://git.inference.coop/co-op/docs/src/branch/main/privacy-policy.md)
|
||||||
|
|
||||||
|
## Legal entity
|
||||||
|
|
||||||
|
The Inference Cooperative operates under the fiscal sponsorship of [Metagov](https://metagov.org/), and our governance is subject to [Metagov's fiscal sponsorship policy](https://metagov.pubpub.org/pub/7bj5ia5g/release/1). We commit to honor that agreement and to operate in a spirit that avoids causing harm to our fiscal host. Metagov may rescind a decision it deems at odds with its policies.
|
||||||
|
|
||||||
|
If that commitment is ever at odds with the cooperative's evolution, the cooperative may move to a different fiscal host. Changing fiscal hosts should occur through a revision of this Charter.
|
||||||
@@ -0,0 +1,91 @@
|
|||||||
|
|
||||||
|
# Contributor Covenant 3.0 Code of Conduct
|
||||||
|
|
||||||
|
## Our Pledge
|
||||||
|
|
||||||
|
We pledge to make our community welcoming, safe, and equitable for all.
|
||||||
|
|
||||||
|
We are committed to fostering an environment that respects and promotes the dignity, rights, and contributions of all individuals, regardless of characteristics including race, ethnicity, caste, color, age, physical characteristics, neurodiversity, disability, sex or gender, gender identity or expression, sexual orientation, language, philosophy or religion, national or social origin, socio-economic position, level of education, or other status. The same privileges of participation are extended to everyone who participates in good faith and in accordance with this Covenant.
|
||||||
|
|
||||||
|
## Encouraged Behaviors
|
||||||
|
|
||||||
|
While acknowledging differences in social norms, we all strive to meet our community's expectations for positive behavior. We also understand that our words and actions may be interpreted differently than we intend based on culture, background, or native language.
|
||||||
|
|
||||||
|
With these considerations in mind, we agree to behave mindfully toward each other and act in ways that center our shared values, including:
|
||||||
|
|
||||||
|
1. Respecting the **purpose of our community**, our activities, and our ways of gathering.
|
||||||
|
2. Engaging **kindly and honestly** with others.
|
||||||
|
3. Respecting **different viewpoints** and experiences.
|
||||||
|
4. **Taking responsibility** for our actions and contributions.
|
||||||
|
5. Gracefully giving and accepting **constructive feedback**.
|
||||||
|
6. Committing to **repairing harm** when it occurs.
|
||||||
|
7. Behaving in other ways that promote and sustain the **well-being of our community**.
|
||||||
|
|
||||||
|
|
||||||
|
## Restricted Behaviors
|
||||||
|
|
||||||
|
We agree to restrict the following behaviors in our community. Instances, threats, and promotion of these behaviors are violations of this Code of Conduct.
|
||||||
|
|
||||||
|
1. **Harassment.** Violating explicitly expressed boundaries or engaging in unnecessary personal attention after any clear request to stop.
|
||||||
|
2. **Character attacks.** Making insulting, demeaning, or pejorative comments directed at a community member or group of people.
|
||||||
|
3. **Stereotyping or discrimination.** Characterizing anyone’s personality or behavior on the basis of immutable identities or traits.
|
||||||
|
4. **Sexualization.** Behaving in a way that would generally be considered inappropriately intimate in the context or purpose of the community.
|
||||||
|
5. **Violating confidentiality**. Sharing or acting on someone's personal or private information without their permission.
|
||||||
|
6. **Endangerment.** Causing, encouraging, or threatening violence or other harm toward any person or group.
|
||||||
|
7. Behaving in other ways that **threaten the well-being** of our community.
|
||||||
|
|
||||||
|
### Other Restrictions
|
||||||
|
|
||||||
|
1. **Misleading identity.** Impersonating someone else for any reason, or pretending to be someone else to evade enforcement actions.
|
||||||
|
2. **Failing to credit sources.** Not properly crediting the sources of content you contribute.
|
||||||
|
3. **Promotional materials**. Sharing marketing or other commercial content in a way that is outside the norms of the community.
|
||||||
|
4. **Irresponsible communication.** Failing to responsibly present content which includes, links or describes any other restricted behaviors.
|
||||||
|
|
||||||
|
|
||||||
|
## Reporting an Issue
|
||||||
|
|
||||||
|
Tensions can occur between community members even when they are trying their best to collaborate. Not every conflict represents a code of conduct violation, and this Code of Conduct reinforces encouraged behaviors and norms that can help avoid conflicts and minimize harm.
|
||||||
|
|
||||||
|
When an incident does occur, it is important to report it promptly. To report a possible violation, **contact the administrative team at info@inference.coop.**
|
||||||
|
|
||||||
|
Community Moderators take reports of violations seriously and will make every effort to respond in a timely manner. They will investigate all reports of code of conduct violations, reviewing messages, logs, and recordings, or interviewing witnesses and other participants. Community Moderators will keep investigation and enforcement actions as transparent as possible while prioritizing safety and confidentiality. In order to honor these values, enforcement actions are carried out in private with the involved parties, but communicating to the whole community may be part of a mutually agreed upon resolution.
|
||||||
|
|
||||||
|
|
||||||
|
## Addressing and Repairing Harm
|
||||||
|
|
||||||
|
**The Community Moderators consist of the disputes working group or, if it is not yet formed, the General Manager.**
|
||||||
|
|
||||||
|
If an investigation by the Community Moderators finds that this Code of Conduct has been violated, the following enforcement ladder may be used to determine how best to repair harm, based on the incident's impact on the individuals involved and the community as a whole. Depending on the severity of a violation, lower rungs on the ladder may be skipped.
|
||||||
|
|
||||||
|
1) Warning
|
||||||
|
1) Event: A violation involving a single incident or series of incidents.
|
||||||
|
2) Consequence: A private, written warning from the Community Moderators.
|
||||||
|
3) Repair: Examples of repair include a private written apology, acknowledgement of responsibility, and seeking clarification on expectations.
|
||||||
|
2) Temporarily Limited Activities
|
||||||
|
1) Event: A repeated incidence of a violation that previously resulted in a warning, or the first incidence of a more serious violation.
|
||||||
|
2) Consequence: A private, written warning with a time-limited cooldown period designed to underscore the seriousness of the situation and give the community members involved time to process the incident. The cooldown period may be limited to particular communication channels or interactions with particular community members.
|
||||||
|
3) Repair: Examples of repair may include making an apology, using the cooldown period to reflect on actions and impact, and being thoughtful about re-entering community spaces after the period is over.
|
||||||
|
3) Temporary Suspension
|
||||||
|
1) Event: A pattern of repeated violation which the Community Moderators have tried to address with warnings, or a single serious violation.
|
||||||
|
2) Consequence: A private written warning with conditions for return from suspension. In general, temporary suspensions give the person being suspended time to reflect upon their behavior and possible corrective actions.
|
||||||
|
3) Repair: Examples of repair include respecting the spirit of the suspension, meeting the specified conditions for return, and being thoughtful about how to reintegrate with the community when the suspension is lifted.
|
||||||
|
4) Permanent Ban
|
||||||
|
1) Event: A pattern of repeated code of conduct violations that other steps on the ladder have failed to resolve, or a violation so serious that the Community Moderators determine there is no way to keep the community safe with this person as a member.
|
||||||
|
2) Consequence: Access to all community spaces, tools, and communication channels is removed. In general, permanent bans should be rarely used, should have strong reasoning behind them, and should only be resorted to if working through other remedies has failed to change the behavior.
|
||||||
|
3) Repair: There is no possible repair in cases of this severity.
|
||||||
|
|
||||||
|
This enforcement ladder is intended as a guideline. It does not limit the ability of Community Managers to use their discretion and judgment, in keeping with the best interests of our community.
|
||||||
|
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public or other spaces. Examples of representing our community include using an official email address, posting via an official social media account, or acting as an appointed representative at an online or offline event.
|
||||||
|
|
||||||
|
|
||||||
|
## Attribution
|
||||||
|
|
||||||
|
This Code of Conduct is adapted from the Contributor Covenant, version 3.0, permanently available at [https://www.contributor-covenant.org/version/3/0/](https://www.contributor-covenant.org/version/3/0/).
|
||||||
|
|
||||||
|
Contributor Covenant is stewarded by the Organization for Ethical Source and licensed under CC BY-SA 4.0. To view a copy of this license, visit [https://creativecommons.org/licenses/by-sa/4.0/](https://creativecommons.org/licenses/by-sa/4.0/)
|
||||||
|
|
||||||
|
For answers to common questions about Contributor Covenant, see the FAQ at [https://www.contributor-covenant.org/faq](https://www.contributor-covenant.org/faq). Translations are provided at [https://www.contributor-covenant.org/translations](https://www.contributor-covenant.org/translations). Additional enforcement and community guideline resources can be found at [https://www.contributor-covenant.org/resources](https://www.contributor-covenant.org/resources). The enforcement ladder was inspired by the work of [Mozilla’s code of conduct team](https://github.com/mozilla/inclusion).
|
||||||
+92
-14
@@ -9,11 +9,12 @@ using the chat, and getting involved in governance.
|
|||||||
|
|
||||||
## 1. Become a member
|
## 1. Become a member
|
||||||
|
|
||||||
Membership is a **sliding-scale contribution of $10–20/month** through Open
|
Membership is a **sliding-scale contribution starting at $5/month** through
|
||||||
Collective. What you pay is up to you.
|
Open Collective (the suggested level is $15 — pay what fits your
|
||||||
|
circumstances).
|
||||||
|
|
||||||
1. Go to [opencollective.com/inference-cooperative](https://opencollective.com/inference-cooperative).
|
1. Go to [opencollective.com/inference-cooperative](https://opencollective.com/inference-cooperative).
|
||||||
2. Choose a contribution amount in the $10–20 range and subscribe.
|
2. Choose a contribution amount ($5 minimum) and subscribe.
|
||||||
3. After you contribute, you'll land on a "check your inbox" page.
|
3. After you contribute, you'll land on a "check your inbox" page.
|
||||||
|
|
||||||
**Within a few minutes, you'll get a welcome email** with a link to set up your
|
**Within a few minutes, you'll get a welcome email** with a link to set up your
|
||||||
@@ -38,29 +39,105 @@ account. Click it and create your login.
|
|||||||
The chat lives at **[chat.inference.coop](https://chat.inference.coop)**. Log in
|
The chat lives at **[chat.inference.coop](https://chat.inference.coop)**. Log in
|
||||||
with the account you created in step 1.
|
with the account you created in step 1.
|
||||||
|
|
||||||
- **Choose your model.** You can pick not just *which* model but *where* it runs:
|
- **Choose your model.** You can pick not just *which* model but *where* it
|
||||||
**private** (Tinfoil, runs in encrypted enclaves) or **green** (GreenPT, 100%
|
runs: **private** (Tinfoil, runs in encrypted enclaves) or **green**
|
||||||
renewable energy). See the [model list](models.md) for the full list.
|
(GreenPT and Regolo, 100% renewable energy — Regolo adds a written
|
||||||
- **Web search** is enabled by default (self-hosted, no external dependency).
|
zero-data-retention policy). See the [model list](models.md) for the full
|
||||||
- **File uploads** work out of the box — ask the chat about a document you attach.
|
list.
|
||||||
- Your usage draws from your monthly allowance; you can see it on the
|
- **Web search** — click the **globe** next to the message box to ground
|
||||||
|
answers in current web results (self-hosted search, no external dependency).
|
||||||
|
- **File uploads** — attach a document with the **paperclip** and ask the
|
||||||
|
chat about it.
|
||||||
|
- **Tools** — the **Tools menu** next to the message box lists everything the
|
||||||
|
chat can do: generate images, search the co-op documentation, and more. If
|
||||||
|
there's a tool you'd like available, ask in #general.
|
||||||
|
- **Voice** works in both directions: speak to the chat with the microphone
|
||||||
|
icon, and have replies read aloud.
|
||||||
|
|
||||||
|
Your usage draws from your monthly allowance; you can see it on the
|
||||||
[Member Dashboard](https://dashboard.inference.coop).
|
[Member Dashboard](https://dashboard.inference.coop).
|
||||||
|
|
||||||
If the chat ever says your balance is out, that's the allowance, not an error —
|
If the chat ever says your balance is out, that's the allowance, not an error —
|
||||||
see "Your allowance" below.
|
see "Your allowance" below.
|
||||||
|
|
||||||
|
> **Moved from the old chat?** The previous Open WebUI instance remains
|
||||||
|
> readable at [chat-legacy.inference.coop](https://chat-legacy.inference.coop)
|
||||||
|
> — your full history is here in the new chat, but files you attached in the
|
||||||
|
> old one are still accessible only there.
|
||||||
|
|
||||||
|
### Add the chat to your phone
|
||||||
|
|
||||||
|
The chat works like an app on your phone — no app store needed. Open
|
||||||
|
[chat.inference.coop](https://chat.inference.coop) in your phone's browser and
|
||||||
|
log in, then:
|
||||||
|
|
||||||
|
- **iPhone (Safari):** tap the **Share** button (the square with an arrow),
|
||||||
|
then **Add to Home Screen**.
|
||||||
|
- **Android (Chrome):** tap the **⋮** menu, then **Add to Home Screen**
|
||||||
|
(or **Install app**).
|
||||||
|
|
||||||
|
You'll get an icon on your home screen that opens the chat full-screen, like
|
||||||
|
any other app. Your login stays active — just tap and chat.
|
||||||
|
|
||||||
## 3. Get involved in governance
|
## 3. Get involved in governance
|
||||||
|
|
||||||
You don't just use the co-op; you help run it. Decisions about models, pricing,
|
You don't just use the co-op; you help run it. Decisions about models, pricing,
|
||||||
privacy, and direction happen on our **Loomio forum**:
|
privacy, and direction happen on our **Loomio forum**:
|
||||||
|
|
||||||
- **[forum.inference.coop](https://forum.inference.coop)** — where members discuss and decide.
|
- **[forum.inference.coop](https://forum.inference.coop)** — where members discuss and decide.
|
||||||
|
- **[matrix.inference.coop](https://matrix.inference.coop)** — our Matrix chat server. Log in
|
||||||
|
with your co-op account (Cloudron SSO); your Matrix ID is `@username:inference.coop`.
|
||||||
|
Every member is invited to the **Inference Cooperative** space automatically — check your
|
||||||
|
invites on first login.
|
||||||
|
|
||||||
|
**Using it from any Matrix client** — the server speaks the standard Matrix protocol,
|
||||||
|
so you can connect from any client:
|
||||||
|
|
||||||
|
- **Element** (web, desktop, iOS, Android): on the sign-in screen, click *Edit* next to
|
||||||
|
"Homeserver" and enter `https://matrix.inference.coop`, then sign in with your co-op
|
||||||
|
account. (On [app.element.io](https://app.element.io) or the desktop/mobile apps alike.)
|
||||||
|
**Note:** the new **Element X** apps (iOS/Android/macOS) are *not* currently supported —
|
||||||
|
they only work with the newer Matrix Authentication Service, which this server doesn't
|
||||||
|
run yet. Use the classic Element apps (Element Web / Element Desktop / "Element" on the
|
||||||
|
app stores) or another client below.
|
||||||
|
- **FluffyChat, Cinny, SchildiChat** or any other client: look for the "custom homeserver"
|
||||||
|
/ "other server" option in the client's login screen and enter the same address.
|
||||||
|
- Your co-op account and your existing Matrix account on another server (e.g.
|
||||||
|
matrix.org or a personal homeserver) are separate identities. **Pick your
|
||||||
|
identity (optional):** by default, invitations go to your co-op identity
|
||||||
|
(`@username:inference.coop`). To use an account you already have instead:
|
||||||
|
|
||||||
|
1. Open your [dashboard](https://dashboard.inference.coop) and find the
|
||||||
|
**Matrix chat** card. It shows the Matrix ID currently registered for
|
||||||
|
you — your co-op identity, or `pending` if your co-op Matrix account
|
||||||
|
hasn't been created yet (it is created the first time you log in to
|
||||||
|
Matrix).
|
||||||
|
2. Click **Use external account**, enter your existing Matrix ID in the
|
||||||
|
format `@username:server.org`, and click **Save**.
|
||||||
|
3. Accept the space invitation that arrives in that account — usually
|
||||||
|
instantly. If the account can't be reached right away, the invitation
|
||||||
|
is retried automatically each night.
|
||||||
|
|
||||||
|
That's all it takes: invitations (and the space) follow whichever identity
|
||||||
|
you've chosen. Switching back is one click — **Use
|
||||||
|
@username:inference.coop** on the same card. One identity is active at a
|
||||||
|
time, and both work equally well in every co-op room.
|
||||||
|
|
||||||
|
**Members-only, readable history.** The space is invite-only and unencrypted
|
||||||
|
so every member can read the full history (including messages from before
|
||||||
|
they joined). Invites are restricted to admins — so while any member is
|
||||||
|
welcome to bring a collaborator or guest, an admin sends the invite rather
|
||||||
|
than members handing the space to outsiders directly.
|
||||||
|
|
||||||
|
- **#general** is the main room — say hello, ask questions, start
|
||||||
|
conversations. Beyond it, the space holds a handful of topical rooms
|
||||||
|
(infrastructure, design, an automated co-op updates feed) — explore the
|
||||||
|
space and join whatever is interesting to you. New rooms appear there as
|
||||||
|
the co-op grows.
|
||||||
|
|
||||||
- See the [Charter](charter.md) for how governance is structured.
|
- See the [Charter](charter.md) for how governance is structured.
|
||||||
- See [Open questions](open-questions.md) for the decisions currently on the table.
|
- See [Open questions](open-questions.md) for the decisions currently on the table.
|
||||||
|
|
||||||
During the pilot phase, Nathan Schneider serves as General Manager and holds
|
|
||||||
final discretion — but members deliberate now, and full member governance is the
|
|
||||||
goal as the co-op grows.
|
|
||||||
|
|
||||||
### Help build it
|
### Help build it
|
||||||
|
|
||||||
@@ -71,8 +148,9 @@ and the "Contributing" section for how to propose changes through
|
|||||||
|
|
||||||
## Your allowance and credits
|
## Your allowance and credits
|
||||||
|
|
||||||
Membership includes **$15 of usage credit each month**, shared between chat and
|
Membership includes **$15 of usage credit each month** (same for every member,
|
||||||
API. It's a starting value, not a hard rule:
|
regardless of what you pay), shared between chat and API. It's a starting
|
||||||
|
value, not a hard rule:
|
||||||
|
|
||||||
- The allowance resets each month on your personal billing date (the day your
|
- The allowance resets each month on your personal billing date (the day your
|
||||||
membership payment recurs) — not on a fixed calendar day.
|
membership payment recurs) — not on a fixed calendar day.
|
||||||
|
|||||||
+56
-10
@@ -5,17 +5,22 @@ and contributors who want to work on it.
|
|||||||
|
|
||||||
## The big picture
|
## The big picture
|
||||||
|
|
||||||
Members reach the co-op through two front doors: the **chat** (Open WebUI) and
|
Members reach the co-op through two front doors: the **chat** (LibreChat) and
|
||||||
the **API** (the LiteLLM gateway). Both route through a single gateway to cloud
|
the **API** (the LiteLLM gateway). Both route through a single gateway to
|
||||||
LLM providers running inside **TEE-protected enclaves** (Tinfoil), so inference
|
multiple LLM providers — so members choose what they value for each task:
|
||||||
is private end-to-end. A thin layer of custom middleware — the member portal —
|
architecturally private inference in **TEE-protected enclaves** (Tinfoil),
|
||||||
ties Open Collective membership to Cloudron accounts and LiteLLM keys.
|
100% renewable energy (GreenPT, Regolo). A thin layer of custom middleware —
|
||||||
|
the member portal — ties Open
|
||||||
|
Collective membership to Cloudron accounts and LiteLLM keys.
|
||||||
|
|
||||||
```
|
```
|
||||||
Members ──> Open WebUI (chat) ─┐
|
┌─> Tinfoil (TEE) 🔒 private
|
||||||
├──> LiteLLM gateway ──> Tinfoil (TEE) ──> models
|
Members ──> LibreChat (chat) ──┐ │
|
||||||
Members ──> your own tools ────┘
|
├──> LiteLLM ──┼─> GreenPT (renewable) 🌱 green
|
||||||
(API)
|
Members ──> your own tools ────┘ gateway │
|
||||||
|
(API) (keys, └─> Regolo (renewable) 🌱 green
|
||||||
|
metering,
|
||||||
|
routing)
|
||||||
```
|
```
|
||||||
|
|
||||||
The whole stack runs on [Cloudron](https://cloudron.io), which handles hosting,
|
The whole stack runs on [Cloudron](https://cloudron.io), which handles hosting,
|
||||||
@@ -25,13 +30,15 @@ single sign-on, and per-app isolation.
|
|||||||
|
|
||||||
| Component | Purpose | URL | Code |
|
| Component | Purpose | URL | Code |
|
||||||
|-----------|---------|-----|------|
|
|-----------|---------|-----|------|
|
||||||
| **Open WebUI** | Member-facing chat (web search, file uploads) | chat.inference.coop | — |
|
| **LibreChat** | Member-facing chat (models, web search, image generation, voice) | chat.inference.coop | [co-op/librechat-custom](https://git.inference.coop/co-op/librechat-custom) |
|
||||||
|
| **Open WebUI** | Legacy chat instance — read-only reference (old history + file attachments) | chat-legacy.inference.coop | — |
|
||||||
| **LiteLLM** | AI gateway — keys, metering, model routing | gateway.inference.coop | [code/litellm](https://git.inference.coop/code/litellm) |
|
| **LiteLLM** | AI gateway — keys, metering, model routing | gateway.inference.coop | [code/litellm](https://git.inference.coop/code/litellm) |
|
||||||
| **Member Portal** | Membership middleware — onboarding, key provisioning, Loomio sync | portal.inference.coop | [code/member-portal](https://git.inference.coop/code/member-portal) |
|
| **Member Portal** | Membership middleware — onboarding, key provisioning, Loomio sync | portal.inference.coop | [code/member-portal](https://git.inference.coop/code/member-portal) |
|
||||||
| **Member Dashboard** | Member-facing usage view + API key management | dashboard.inference.coop | [code/member-dashboard](https://git.inference.coop/code/member-dashboard) |
|
| **Member Dashboard** | Member-facing usage view + API key management | dashboard.inference.coop | [code/member-dashboard](https://git.inference.coop/code/member-dashboard) |
|
||||||
| **Admin Panel** | Admin-only member overview, spend, balance management | panel.inference.coop | [code/admin-panel](https://git.inference.coop/code/admin-panel) |
|
| **Admin Panel** | Admin-only member overview, spend, balance management | panel.inference.coop | [code/admin-panel](https://git.inference.coop/code/admin-panel) |
|
||||||
| **SearXNG** | Self-hosted web search (feeds the chat's search) | search.inference.coop | — |
|
| **SearXNG** | Self-hosted web search (feeds the chat's search) | search.inference.coop | — |
|
||||||
| **Loomio** | Member governance | forum.inference.coop | — |
|
| **Loomio** | Member governance | forum.inference.coop | — |
|
||||||
|
| **Matrix (Synapse)** | Member chat server (federated; SSO login via Cloudron) | matrix.inference.coop | — |
|
||||||
| **Gitea** | Git hosting (code + docs) | git.inference.coop | — |
|
| **Gitea** | Git hosting (code + docs) | git.inference.coop | — |
|
||||||
| **Surfer** | Static hosting (landing page, thanks page) | inference.coop | [co-op/website](https://git.inference.coop/co-op/website) |
|
| **Surfer** | Static hosting (landing page, thanks page) | inference.coop | [co-op/website](https://git.inference.coop/co-op/website) |
|
||||||
| **Listmonk** | Member newsletter (auto-synced from membership) | newsletter.inference.coop | — |
|
| **Listmonk** | Member newsletter (auto-synced from membership) | newsletter.inference.coop | — |
|
||||||
@@ -41,6 +48,45 @@ single sign-on, and per-app isolation.
|
|||||||
Open Collective (billing and fiscal sponsorship) is the one external service in
|
Open Collective (billing and fiscal sponsorship) is the one external service in
|
||||||
the flow — see [opencollective.com/inference-cooperative](https://opencollective.com/inference-cooperative).
|
the flow — see [opencollective.com/inference-cooperative](https://opencollective.com/inference-cooperative).
|
||||||
|
|
||||||
|
## Matrix
|
||||||
|
|
||||||
|
Members get a federated Matrix account on our self-hosted Synapse
|
||||||
|
(matrix.inference.coop, user IDs `@username:inference.coop`) using their
|
||||||
|
Cloudron SSO login. The portal invites every active member — and Nathan's
|
||||||
|
federated account — to the private **Inference Cooperative** Space (with a
|
||||||
|
**General** room) on every membership sync, so new members land in the space
|
||||||
|
automatically. Invites honor rate limits; external/federated members can be
|
||||||
|
added via the portal's `MATRIX_EXTRA_MXIDS` env.
|
||||||
|
|
||||||
|
Federation is enabled (delegated via `.well-known/matrix/server` on the bare
|
||||||
|
domain, port 443), so members can also join from any homeserver they already
|
||||||
|
use. The `@inference-bot:inference.coop` admin account (used by the portal
|
||||||
|
for invites) has no password — token lives in the portal env.
|
||||||
|
|
||||||
|
## How the apps run themselves
|
||||||
|
|
||||||
|
The portal and gateway are self-maintaining: their recurring housekeeping is
|
||||||
|
built into the apps as scheduled tasks, not run from any member's or operator's
|
||||||
|
personal machine.
|
||||||
|
|
||||||
|
- **Member portal** — two scheduled jobs (Cloudron scheduler):
|
||||||
|
- *Membership sweep* (nightly, 3am) — reconciles members against the live
|
||||||
|
Open Collective list: provisions new members who signed up while a webhook
|
||||||
|
was missed, deactivates lapsed members, and syncs Loomio + Matrix
|
||||||
|
membership. A missed webhook self-heals within 24 hours.
|
||||||
|
- *Docs knowledge refresh* (nightly, 3:30am) — pulls this documentation from
|
||||||
|
the public Git repo and updates the shared knowledge base behind the chat's
|
||||||
|
models, so answers grounded in the docs stay current. Changes to the docs
|
||||||
|
appear in the chat within a day, no manual step.
|
||||||
|
- **Gateway** — an *audio keep-alive* (every 10 minutes) sends a tiny text-to-
|
||||||
|
speech ping to the private enclave so voice mode responds quickly even after
|
||||||
|
long quiet periods, without compromising the end-to-end encryption model.
|
||||||
|
|
||||||
|
Scheduled tasks are declared in each app's `CloudronManifest.json` (see the
|
||||||
|
`code/` repos) and survive redeploys. If a task fails, it shows up in the
|
||||||
|
app's logs on the admin panel; the portal also exposes a manual trigger for
|
||||||
|
the membership sweep.
|
||||||
|
|
||||||
## How membership is wired
|
## How membership is wired
|
||||||
|
|
||||||
Membership flows through the member portal, which is the single source of truth
|
Membership flows through the member portal, which is the single source of truth
|
||||||
|
|||||||
@@ -0,0 +1,27 @@
|
|||||||
|
# Chat task models (titles, tags, suggestions)
|
||||||
|
|
||||||
|
Both chat front doors generate small helper outputs — conversation titles,
|
||||||
|
tags, retrieval queries — with a model separate from the one answering the
|
||||||
|
member. These tasks are pinned to a small, inexpensive model so they never
|
||||||
|
bill against the member's chosen (larger) model.
|
||||||
|
|
||||||
|
## Open WebUI (chat.inference.coop)
|
||||||
|
|
||||||
|
- Title generation and tag generation are **enabled**; follow-up reply
|
||||||
|
suggestions and input autocomplete are **disabled** (October 2026).
|
||||||
|
- The task model (`task.model.default` in the Postgres `config` table) is
|
||||||
|
pinned to `greenpt/glm-5.3-flash` — a small, inexpensive model — so titles
|
||||||
|
and tags always run on it regardless of which model the member is chatting
|
||||||
|
with. Before October 2026 it was empty, which meant helper tasks ran on the
|
||||||
|
member's selected model.
|
||||||
|
- Config lives in the app's Postgres database (`config` table, key/value rows
|
||||||
|
with JSON values) and is read at startup: change a row, then restart the
|
||||||
|
app (`cloudron restart --app <id>`; note `restart` takes no `--timeout`).
|
||||||
|
|
||||||
|
## LibreChat (librechat-test.inference.coop)
|
||||||
|
|
||||||
|
- Title generation is configured in `/app/data/librechat.yaml`:
|
||||||
|
`titleConvo: true`, `titleModel: greenpt/glm-5.3-flash`,
|
||||||
|
`titleMethod: completion` — the same small GreenPT model.
|
||||||
|
- LibreChat v0.8.8 has no reply-suggestion feature; there is nothing to
|
||||||
|
disable on that front.
|
||||||
@@ -0,0 +1,156 @@
|
|||||||
|
# LibreChat sandbox maintenance
|
||||||
|
|
||||||
|
The LibreChat evaluation instance at **librechat-test.inference.coop**
|
||||||
|
(Cloudron app `4023ccbf-d57a-4500-97df-14de9cf3516e`, appstore package
|
||||||
|
`ai.librechat.cloudronapp@1.4.1`, which bundles **LibreChat v0.8.8**).
|
||||||
|
|
||||||
|
The single source of truth for the deployment is the repo
|
||||||
|
**[co-op/librechat-custom](https://git.inference.coop/co-op/librechat-custom)**
|
||||||
|
— its README is the full runbook (config deploy, image build/deploy, rollback,
|
||||||
|
env vars, branding overlay, known state). This doc is the operator-facing
|
||||||
|
summary.
|
||||||
|
|
||||||
|
## Architecture: what lives where
|
||||||
|
|
||||||
|
- **App config** — `/app/data/librechat.yaml` (persistent volume, mirrored
|
||||||
|
byte-identically in the repo). Holds the "Inference Co-op" custom endpoint
|
||||||
|
(gateway), the 8-model chat picker, speech, web search, and the two MCP
|
||||||
|
servers. Volume edits deploy via `cloudron exec` (base64), never via the
|
||||||
|
image — the image never touches `/app/data`.
|
||||||
|
- **Env file** — `/app/data/env` (persistent volume). **`start.sh` sources it
|
||||||
|
AFTER the process env, so it wins over `cloudron env`** — that is where
|
||||||
|
`APP_TITLE="Inference Co-op"`, `ALLOW_EMAIL_LOGIN=false` (password login
|
||||||
|
removed; Cloudron SSO is the only login path), and
|
||||||
|
`SCHEDULES_SINGLE_PROCESS=true` must live.
|
||||||
|
- **Other env vars** — via `cloudron env`: `TTS_API_KEY`/`STT_API_KEY`
|
||||||
|
(dedicated LiteLLM virtual key, alias `librechat-speech`, $5/30d, audio
|
||||||
|
models only — never the gateway master key), `IMAGE_GEN_OAI_BASEURL` +
|
||||||
|
`IMAGE_GEN_OAI_MODEL` (agent image tool → gateway), `SEARXNG_INSTANCE_URL`.
|
||||||
|
- **MCP servers** — `/app/data/mcp/server.py` (docs search, `search_docs`,
|
||||||
|
starts at boot) and `/app/data/mcp/image_server.py` (image generation,
|
||||||
|
`generate_image`, `startup: false`); both mirrored in the repo under
|
||||||
|
`mcp/`, stdlib-only Python.
|
||||||
|
- **Branding overlay** (logo, favicons, PWA icons, `<title>`, `custom.css`,
|
||||||
|
`custom2.js`) — baked into a pinned overlay image `FROM` the official
|
||||||
|
appstore image. Deployed tag (last verified 2026-10-19):
|
||||||
|
`registry.inference.coop/co-op/librechat-custom:20261019-fc9b11e`.
|
||||||
|
- **Branding CSS/JS** — `custom.css` overrides LibreChat's Tailwind theme
|
||||||
|
variables on `html.light`/`html.dark` using the HOMEPAGE's canonical
|
||||||
|
palette (`--cream #faf8f5`, `--wash #eef4ea`, `--green-deep #5b8c5a`,
|
||||||
|
`--green-mid #6ba86b`, `--ink #2d3327`, `--muted #6b7a62`) and brands
|
||||||
|
the loading flash (`#loading-container`); the login logo is the green
|
||||||
|
pine-dots vector (white-plate variant was invisible on cream — the
|
||||||
|
'flash on load' glitch), cache-busted via `?v=3` in the client bundle.
|
||||||
|
and hides the register/sign-up prompt via
|
||||||
|
`p:has(> a[href*="register"])`; `custom2.js` is a MutationObserver that
|
||||||
|
hides the prompt when React renders it (the auth form renders after
|
||||||
|
`load`). The `interface.theme` block in the yaml remains
|
||||||
|
forward-compatible only (inert on v0.8.8).
|
||||||
|
|
||||||
|
## Image generation (two member-facing paths)
|
||||||
|
|
||||||
|
1. **MCP tool (default, no key ever)**: the `inference-image` MCP server
|
||||||
|
exposes `generate_image` in the composer's **+** palette — no agent setup,
|
||||||
|
no key prompt. Per-member billing via `customUserVars.IC_API_KEY` (seeded
|
||||||
|
from the portal) + `env: IC_API_KEY: "{{IC_API_KEY}}"`; a system-key
|
||||||
|
fallback (`system-image-fallback`, $5/30d) covers the rare empty-value
|
||||||
|
case. The `customUserVars` gate blocks a not-yet-seeded member until the
|
||||||
|
next sync (≤10 min) — accepted as a short delay over billing everything
|
||||||
|
to the system account.
|
||||||
|
2. **Agent tool**: LibreChat's built-in "OpenAI Image Tools"
|
||||||
|
(`IMAGE_GEN_OAI_BASEURL`/`IMAGE_GEN_OAI_MODEL` → gateway; members paste
|
||||||
|
their own key — v0.8.8 limitation).
|
||||||
|
|
||||||
|
## Update policy
|
||||||
|
|
||||||
|
- The app is **image-sourced** (the `--force` update moved it off the
|
||||||
|
appstore source), so Cloudron will not auto-update it. This is deliberate —
|
||||||
|
same lockdown model as LiteLLM.
|
||||||
|
- **Image deploys**: build `--no-cache`, write the new tag into
|
||||||
|
`CloudronManifest.json.dockerImage` (the only editable field), commit, then
|
||||||
|
`cloudron update --app <id> --force`. **`--image` alone is IGNORED** when
|
||||||
|
the manifest sits in the deploy directory. **The remote builder's cache can
|
||||||
|
serve stale COPY layers even when file content changed** (six consecutive
|
||||||
|
"Build successful" runs did on 2026-10-19) — so after every image deploy,
|
||||||
|
verify: the Cloudron API's `manifest.dockerImage`, container-file md5 vs
|
||||||
|
repo md5, the served HTML, and the rendered DOM. Full checklist in the
|
||||||
|
repo README.
|
||||||
|
- **When a new appstore package ships**: pull its manifest copy into the repo
|
||||||
|
(`GET /api/v1/apps/<id>` → `.manifest`), repoint `dockerImage` to the
|
||||||
|
rebuilt overlay, bump the Dockerfile `FROM` to the new `manifest.dockerImage`
|
||||||
|
(pinned), rebuild, verify in the sandbox, then update.
|
||||||
|
- **Rollback**: repoint `CloudronManifest.json.dockerImage` to the previous
|
||||||
|
tag and rerun `cloudron update --force` (or the original appstore image).
|
||||||
|
Cloudron also snapshots the app before every image update.
|
||||||
|
|
||||||
|
## Hard-won pitfalls (do not re-learn these)
|
||||||
|
|
||||||
|
- `cloudron update` deploys **`manifest.dockerImage`** from the manifest in
|
||||||
|
the deploy directory — the `--image` flag alone is ignored (and the update
|
||||||
|
output still prints the `--image` value, so it *looks* right). The repo
|
||||||
|
manifest is a byte-identical copy of the OFFICIAL appstore manifest except
|
||||||
|
`dockerImage`; editing anything else in it (or hand-writing one that omits
|
||||||
|
the addons) strips `postgresql`/`mongodb`/`oidc`/`sendmail`/`localstorage`
|
||||||
|
from the app and crash-loops it (`mkdir: Read-only file system`). That is
|
||||||
|
what broke the sandbox on 2026-10-18.
|
||||||
|
- The remote builder's layer cache can pin COPY steps to old file content
|
||||||
|
even across "successful" rebuilds — use `--no-cache` after asset edits and
|
||||||
|
confirm by comparing container-file md5 against the repo file.
|
||||||
|
- Deploy `librechat.yaml` only via base64 round-trip. A `sed` insert once
|
||||||
|
broke the YAML indentation and LibreChat **silently fell back to defaults**
|
||||||
|
(all custom branding/models/MCP gone) with a single log line.
|
||||||
|
- Reading large files from the container over `cloudron exec` with a plain
|
||||||
|
`cat` resets the connection (`ECONNRESET`) — pipe through
|
||||||
|
`gzip | base64 | fold` and decode locally.
|
||||||
|
- `cloudron env set` alone does NOT disable the password form —
|
||||||
|
`ALLOW_EMAIL_LOGIN` must be flipped in `/app/data/env` (precedence above).
|
||||||
|
- Image generation in LibreChat is the **MCP tool** (composer **+** palette);
|
||||||
|
the agent "OpenAI Image Tools" path also exists. Image models in the chat
|
||||||
|
picker were the source of the old 500 errors and are deliberately excluded.
|
||||||
|
- No `branding:` block exists in LibreChat's config schema, and the login
|
||||||
|
page has no `.login-page`/`.primary` classes to target — theming happens by
|
||||||
|
overriding the Tailwind CSS variables on `html.light`/`html.dark`
|
||||||
|
(`--surface-primary`, `--accent-primary`, …) from the overlay `custom.css`.
|
||||||
|
|
||||||
|
|
||||||
|
## Per-member key auto-load (no API-key prompts)
|
||||||
|
|
||||||
|
Members never enter an API key. The portal provisions each member's LiteLLM
|
||||||
|
key at onboarding; the sandbox seeds it into LibreChat's user-key store:
|
||||||
|
|
||||||
|
- `GET /admin/librechat/keys` on the portal returns active members' chat keys
|
||||||
|
(dedicated `X-Sync-Token` auth — `LIBRECHAT_SYNC_TOKEN` on both apps).
|
||||||
|
- `/app/data/sync-keys.js` (in the LibreChat container, mirrored in the repo
|
||||||
|
as `sync-keys.js`) fetches that feed and seeds the `keys` collection
|
||||||
|
(chat endpoint, `apiKey: "user_provided"`) and `pluginauths`
|
||||||
|
(`mcp:inference-image` → `IC_API_KEY`, injected per-user via
|
||||||
|
`customUserVars` so image generation bills to the member's own team).
|
||||||
|
- Members without a sandbox account are skipped until they first log in; run
|
||||||
|
the sync after onboarding waves and on a schedule (the command is in the
|
||||||
|
repo README). Idempotent.
|
||||||
|
- Encryption matches LibreChat's own (`CREDS_KEY`/`CREDS_IV`, read from
|
||||||
|
`/proc/1/environ` — exec sessions don't inherit the app env).
|
||||||
|
|
||||||
|
> Terminal output redacts values after `apiKey:` as `***` even for harmless
|
||||||
|
> strings (`user_provided`, `${TTS_API_KEY}`). Probe lengths/counts instead
|
||||||
|
> of trusting redacted output.
|
||||||
|
|
||||||
|
## Verification (after any change)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cloudron logs --app 4023ccbf-d57a-4500-97df-14de9cf3516e --lines 60 | \
|
||||||
|
grep -iE 'custom config|error|invalid|MCP|schedules'
|
||||||
|
curl -s https://librechat-test.inference.coop/api/config # appTitle, emailLoginEnabled
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: `Custom config file loaded`, `[MCP][inference-docs]` initialized
|
||||||
|
with tool `search_docs`, `inference-image` registered (its `Tools: undefined`
|
||||||
|
boot line is expected — per-user startup), `[schedules] engine started`,
|
||||||
|
`appTitle: Inference Co-op`, `emailLoginEnabled: false`. The login page shows
|
||||||
|
the co-op logo, the co-op favicon, a single "Login with Inference
|
||||||
|
Cooperative" (Cloudron SSO) path with no email/password form, **no
|
||||||
|
"Don't have an account?" line**, and the cream/green co-op color scheme.
|
||||||
|
|
||||||
|
After an IMAGE deploy, additionally run the four-step verification in the
|
||||||
|
repo README (API `manifest.dockerImage`, container md5 vs repo md5, served
|
||||||
|
HTML tags, rendered DOM colors).
|
||||||
@@ -16,20 +16,18 @@ provider today is best on both.
|
|||||||
|---|---|---|---|---|
|
|---|---|---|---|---|
|
||||||
| **Tinfoil** | *private* | **Strongest — architectural** | Not a green claim | Runs in hardware enclaves (TEEs) with end-to-end encryption (EHBP), verifiable via remote attestation. Neither we nor Tinfoil can read your prompts or responses at inference time. Energy mix is not disclosed. |
|
| **Tinfoil** | *private* | **Strongest — architectural** | Not a green claim | Runs in hardware enclaves (TEEs) with end-to-end encryption (EHBP), verifiable via remote attestation. Neither we nor Tinfoil can read your prompts or responses at inference time. Energy mix is not disclosed. |
|
||||||
| **GreenPT** | *green* | **Policy-based** | **Strongest — 100% renewable** | Standard (non-enclave) hosting in the EU; privacy rests on GreenPT's commitments and EU data rules, not on cryptography. Compute runs on 100% renewable energy. |
|
| **GreenPT** | *green* | **Policy-based** | **Strongest — 100% renewable** | Standard (non-enclave) hosting in the EU; privacy rests on GreenPT's commitments and EU data rules, not on cryptography. Compute runs on 100% renewable energy. |
|
||||||
| **PublicAI** | *public* | Policy-based | Undisclosed | A gateway for **publicly developed, sovereign models** — open-weight models built by national/public programs rather than private labs. Privacy is policy-based (no enclave); energy mix is not disclosed. |
|
| **Regolo** | *green* | Policy-based — **zero data retention** | **100% renewable** | EU-hosted (Italy, Seeweb) with a stated zero-data-retention policy: prompts are not stored or reused. Compute runs on 100% renewable energy. |
|
||||||
|
|
||||||
**Reading the gradations:**
|
**Reading the gradations:**
|
||||||
|
|
||||||
- **Privacy:** *architectural* (Tinfoil) beats *policy-based* (GreenPT, PublicAI). An
|
- **Privacy:** *architectural* (Tinfoil) beats *policy-based* (GreenPT,
|
||||||
enclave protects you even if the provider is compromised; a policy protects
|
Regolo). An enclave protects you even if the provider is compromised; a
|
||||||
you only as far as the provider keeps its word. Both are better than a
|
policy protects you only as far as the provider keeps its word. Both are
|
||||||
provider with neither.
|
better than a provider with neither — and Regolo adds a written
|
||||||
- **Green energy:** *100% renewable* (GreenPT) beats *undisclosed* (Tinfoil,
|
zero-data-retention commitment on top of its policy.
|
||||||
PublicAI). GreenPT reports its energy mix; the others do not, so we can't
|
- **Green energy:** *100% renewable* (GreenPT, Regolo) beats *undisclosed*
|
||||||
claim greenness for them.
|
(Tinfoil). GreenPT and Regolo both run on 100% renewable energy; Tinfoil's
|
||||||
- **Sovereignty:** *publicly developed* (PublicAI) is its own value — the models
|
mix is not disclosed.
|
||||||
come from public/national AI programs, built as public goods rather than
|
|
||||||
proprietary products. It's a third axis, independent of privacy and energy.
|
|
||||||
|
|
||||||
Trade-offs are real: the most private option (Tinfoil) isn't the greenest, and
|
Trade-offs are real: the most private option (Tinfoil) isn't the greenest, and
|
||||||
the greenest option (GreenPT) isn't the most private. The point of the co-op is
|
the greenest option (GreenPT) isn't the most private. The point of the co-op is
|
||||||
@@ -37,51 +35,67 @@ that **you choose** which axis matters more for a given task.
|
|||||||
|
|
||||||
**How to pick, quick guide:**
|
**How to pick, quick guide:**
|
||||||
|
|
||||||
- **Everyday chat and drafting** → a fast model: `tinfoil/glm-5-3-flash`,
|
- **Default** → `greenpt/deepseek-v4.1-flash`: DeepSeek's efficient multimodal
|
||||||
`greenpt/glm-5.3-flash`, or `greenpt/green-l`. These handle the bulk of what
|
reasoning model on renewable energy, and the model the chat starts on. For the
|
||||||
people use chat for, at a fraction of flagship cost.
|
TEE-protected (private) version of the same model, use `tinfoil/deepseek-v4-1-flash`.
|
||||||
- **Hard reasoning, coding, agentic work** → a flagship: `tinfoil/glm-5-3` or
|
- **Everyday chat and drafting** → a fast model: `greenpt/glm-5.3-flash`.
|
||||||
`greenpt/glm-5.3` (same underlying model, different provider trade-off),
|
These handle the bulk of what people use chat for, at a fraction of flagship
|
||||||
or `tinfoil/deepseek-v4-1-flash` for long documents and tool-driven tasks.
|
cost.
|
||||||
- **Images in, text out (vision)** → `tinfoil/glm-5-3-flash`,
|
- **Hard reasoning, coding, agentic work** → a flagship: `greenpt/glm-5.3`
|
||||||
`tinfoil/deepseek-v4-1-flash`, or the Apertus models — see the ✅ in the
|
(renewable) or `tinfoil/glm-5.3` (TEE-protected) — the GLM flagship on the
|
||||||
table below.
|
provider of your choice — or `tinfoil/deepseek-v4-1-flash` for long documents
|
||||||
|
and tool-driven tasks.
|
||||||
|
- **Images in, text out (vision)** → `tinfoil/deepseek-v4-1-flash` or the
|
||||||
|
Apertus models — see the ✅ in the table below.
|
||||||
- **Cheap and cheerful** → `tinfoil/gpt-oss-120b` or `greenpt/glm-5.3-flash`.
|
- **Cheap and cheerful** → `tinfoil/gpt-oss-120b` or `greenpt/glm-5.3-flash`.
|
||||||
- **Model origins matter to you** → the Apertus models (publicly developed;
|
- **Model origins matter to you** → the Apertus models (publicly developed;
|
||||||
see below).
|
see below).
|
||||||
|
|
||||||
## Current models
|
## Current models
|
||||||
|
|
||||||
Pricing is per 1 million tokens, input / output (what the co-op pays the
|
Pricing is per 1 million tokens, input / output / **cached input** (what the
|
||||||
provider — your own cost is your monthly allowance, not per-token billing).
|
co-op pays the provider — your own cost is your monthly allowance, not
|
||||||
**Vision** means the model accepts images as input (screenshots, photos,
|
per-token billing). The cached-input rate applies when a model reuses a
|
||||||
documents to reason about) — all models output text. None of our current
|
shared prompt prefix (multi-turn conversations, agent loops); most providers
|
||||||
models generate images.
|
charge a steep discount for it. Models with `—` don't offer a distinct
|
||||||
|
cache-read price. **Vision** means the model accepts images as input
|
||||||
|
(screenshots, photos, documents to reason about). **Image generation** is
|
||||||
|
available via the chat's image feature (the 🎨 button) using
|
||||||
|
`regolo/qwen-image`.
|
||||||
|
|
||||||
| Model | Provider | Privacy | Green | Vision | Context | Pricing (in / out) | Notes |
|
The emoji in the model name marks its **value tier** (the same icons used in the chat picker):
|
||||||
|
|
||||||
|
- 🔒 **Private** — Tinfoil: architecturally private (TEE enclaves)
|
||||||
|
- 🌱 **Green** — GreenPT and Regolo: renewable-powered EU hosts (Regolo adds
|
||||||
|
a zero-data-retention commitment). The 🌱 emoji marks renewable-powered
|
||||||
|
providers, not a specific company.
|
||||||
|
|
||||||
|
| Model | Provider | Privacy | Green | Vision | Context | Pricing (in / out / cached) | Notes |
|
||||||
|---|---|---|---|---|---|---|---|
|
|---|---|---|---|---|---|---|---|
|
||||||
| `tinfoil/glm-5-3-flash` | Tinfoil | architectural | undisclosed | ✅ | 1M | $0.40 / $1.25 | Fast multimodal MoE; great everyday default with image input. |
|
| `🌱 greenpt/deepseek-v4.1-flash` | GreenPT | policy | ✅ 100% renewable | ✅ | 1M | $0.22 / $1.10 / $0.011 | **Default.** DeepSeek's multimodal reasoning model, renewable energy; TEE twin below. |
|
||||||
| `tinfoil/glm-5-3` | Tinfoil | architectural | undisclosed | — | 1M | $1.80 / $5.75 | GLM flagship; strongest agentic model, private tier. |
|
| `🔒 tinfoil/deepseek-v4-1-flash` | Tinfoil | architectural | undisclosed | ✅ | 1M | $0.65 / $1.45 / $0.13 | TEE-protected version of the default; image input, tool calling, long docs/agents. |
|
||||||
| `tinfoil/deepseek-v4-1-flash` | Tinfoil | architectural | undisclosed | ✅ | 1M | $0.65 / $1.45 | Efficient MoE, image input, tool calling; great for long docs and agents. |
|
| `🔒 tinfoil/glm-5.3` | Tinfoil | architectural | undisclosed | — | 1M | $1.80 / $5.75 / $0.45 | GLM flagship, TEE-protected; strongest private-tier model for agentic coding and long-horizon tasks. |
|
||||||
| `tinfoil/gpt-oss-120b` | Tinfoil | architectural | undisclosed | — | 131K | $0.15 / $0.60 | Lightweight open-weight reasoning; cheapest private tier. |
|
| `🔒 tinfoil/gpt-oss-120b` | Tinfoil | architectural | undisclosed | — | 131K | $0.15 / $0.60 / — | Lightweight open-weight reasoning; cheapest private tier. |
|
||||||
| `tinfoil/kimi-k3` | Tinfoil | architectural | undisclosed | ✅ (image + video) | 262K | $4.00 / $20.00 | Moonshot's flagship multimodal MoE; premium tier. |
|
| `🌱 greenpt/glm-5.3-flash` | GreenPT | policy | ✅ 100% renewable | — | 1M | $0.11 / $0.44 / $0.022 | Fast GLM model, renewable energy. |
|
||||||
| `greenpt/green-r` | GreenPT | policy | 100% renewable | — | 128K | $0.35 / $0.95 | Reasoning-focused, renewable energy. |
|
| `🌱 greenpt/glm-5.3` | GreenPT | policy | ✅ 100% renewable | — | 1M | $1.10 / $4.40 / $0.275 | GLM flagship — strongest agentic model in the green tier. |
|
||||||
| `greenpt/green-l` | GreenPT | policy | 100% renewable | — | 128K | $0.25 / $0.80 | Lightweight, renewable energy. |
|
| `🌱 regolo/apertus-70b` | Regolo | policy + ZDR | ✅ 100% renewable | ✅ | 65K | $0.46 / $2.44 / — | Fully open sovereign model (Apache-2.0), large/capable, image understanding. |
|
||||||
| `greenpt/glm-5.3-flash` | GreenPT | policy | 100% renewable | — | 1M | $0.11 / $0.44 | Fast GLM model, renewable energy. |
|
| `🌱 regolo/qwen3.5-122b` | Regolo | policy + ZDR | ✅ 100% renewable | ✅ | 262K | $1.16 / $4.87 / — | Large vision+reasoning MoE (Qwen 3.5, 122B-A10B); 262K context. |
|
||||||
| `greenpt/glm-5.3` | GreenPT | policy | 100% renewable | — | 1M | $1.10 / $4.40 | GLM flagship — strongest agentic model in the green tier. |
|
|
||||||
| `publicai/apertus-v1.5-8b` | PublicAI | policy | undisclosed | ✅ | 262K | $0.10 / $0.20 | Fully open sovereign model, small/fast, image understanding. |
|
|
||||||
| `publicai/apertus-v1.5-70b` | PublicAI | policy | undisclosed | ✅ | 262K | $0.82 / $2.92 | Fully open sovereign model, large/capable, image understanding. |
|
|
||||||
|
|
||||||
Vision is verified working through the co-op gateway (base64 image input via
|
Vision is verified working through the co-op gateway (base64 image input via
|
||||||
`/v1/chat/completions`, standard OpenAI format). In the chat UI, just attach
|
`/v1/chat/completions`, standard OpenAI format). In the chat UI, just attach
|
||||||
an image to your message while using a ✅ model.
|
an image to your message while using a ✅ model.
|
||||||
|
|
||||||
**Same model, two providers:** `glm-5-3` (Tinfoil) and `glm-5.3` (GreenPT)
|
**Reading the Privacy column:** *architectural* means the protection is built
|
||||||
are the same Z.ai model offered by two providers — Tinfoil's is
|
into the infrastructure (TEE enclaves) — even the provider can't read your
|
||||||
architecturally private at $1.80/$5.75; GreenPT's is 100% renewable at
|
prompts. *Policy* means the provider commits, by contract and policy, not to
|
||||||
$1.10/$4.40. That price/privacy trade-off in a single comparison is the
|
store or reuse your data. *ZDR* (zero data retention) is a stronger policy:
|
||||||
co-op's whole idea in miniature: you pick which axis matters for the task
|
the provider keeps no copy of prompts or outputs at all after the response is
|
||||||
at hand.
|
delivered.
|
||||||
|
|
||||||
|
**Same model, two providers:** `glm-5.3` (Z.ai's GLM flagship) is offered both
|
||||||
|
on GreenPT (renewable, $1.10/$4.40) and on Tinfoil (TEE-protected, $1.80/$5.75)
|
||||||
|
— pick the axis that matters to you. The fast `glm-5.3-flash` variant is
|
||||||
|
GreenPT-only (Tinfoil deprecated their flash variant in October 2026).
|
||||||
|
|
||||||
### Audio models
|
### Audio models
|
||||||
|
|
||||||
@@ -92,8 +106,8 @@ model picker to avoid confusion.
|
|||||||
|
|
||||||
| Model | Provider | Use | Pricing |
|
| Model | Provider | Use | Pricing |
|
||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| `tinfoil/whisper-large-v3-turbo` | Tinfoil | Speech-to-text | $0.05 / 1M input tokens |
|
| `🔒 tinfoil/whisper-large-v3-turbo` | Tinfoil | Speech-to-text | $0.05 / 1M input tokens |
|
||||||
| `tinfoil/voxtral-tts` | Tinfoil | Text-to-speech | listed $0 (verify on invoice) |
|
| `🔒 tinfoil/voxtral-tts` | Tinfoil | Text-to-speech | listed $0 (verify on invoice) |
|
||||||
|
|
||||||
Both run inside Tinfoil's enclaves, so voice data gets the same architectural
|
Both run inside Tinfoil's enclaves, so voice data gets the same architectural
|
||||||
privacy as the private chat models. TTS voices available: `neutral_female`,
|
privacy as the private chat models. TTS voices available: `neutral_female`,
|
||||||
@@ -118,12 +132,28 @@ curl https://gateway.inference.coop/v1/audio/speech \
|
|||||||
curl https://gateway.inference.coop/v1/chat/completions \
|
curl https://gateway.inference.coop/v1/chat/completions \
|
||||||
-H "Authorization: Bearer ***" \
|
-H "Authorization: Bearer ***" \
|
||||||
-H "Content-Type: application/json" \
|
-H "Content-Type: application/json" \
|
||||||
-d '{"model": "tinfoil/glm-5-3-flash", "messages": [{"role": "user", "content": [
|
-d '{"model": "tinfoil/deepseek-v4-1-flash", "messages": [{"role": "user", "content": [
|
||||||
{"type": "text", "text": "What is in this image?"},
|
{"type": "text", "text": "What is in this image?"},
|
||||||
{"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}
|
{"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}
|
||||||
]}]}'
|
]}]}'
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Image generation
|
||||||
|
|
||||||
|
`🌱 regolo/qwen-image` (Regolo, EU-hosted, zero data retention) generates
|
||||||
|
images from a text prompt. In the chat, press the **Integrations** button
|
||||||
|
below the message box, choose **Image**, and describe what you want — the
|
||||||
|
generated image appears in the conversation. Via the API:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl https://gateway.inference.coop/v1/images/generations \
|
||||||
|
-H "Authorization: Bearer ***" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"model": "regolo/qwen-image", "prompt": "a watercolor painting of a mountain lake", "n": 1, "size": "512x512"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
The response returns base64-encoded JPEG(s). Billed per pixel generated.
|
||||||
|
|
||||||
### About the Apertus models
|
### About the Apertus models
|
||||||
|
|
||||||
Apertus is the **Swiss AI Initiative's** open foundation model, built by a
|
Apertus is the **Swiss AI Initiative's** open foundation model, built by a
|
||||||
@@ -147,6 +177,9 @@ window, improved tool use, and instruction-following. It's the *publicly
|
|||||||
developed* counterpart to the private (Tinfoil) and renewable (GreenPT)
|
developed* counterpart to the private (Tinfoil) and renewable (GreenPT)
|
||||||
offerings — sovereignty as a third axis of member choice.
|
offerings — sovereignty as a third axis of member choice.
|
||||||
|
|
||||||
|
For more information on open-source AI and the Apertus models, see the
|
||||||
|
[European Open Source AI Index](https://osai-index.eu/database/).
|
||||||
|
|
||||||
*Source: [apertus-ai.org](https://www.apertus-ai.org/) and the
|
*Source: [apertus-ai.org](https://www.apertus-ai.org/) and the
|
||||||
[Swiss AI Hugging Face](https://huggingface.co/swiss-ai) model cards.*
|
[Swiss AI Hugging Face](https://huggingface.co/swiss-ai) model cards.*
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -48,7 +48,7 @@ Currently chat history is stored on co-op servers. We could potentially switch t
|
|||||||
|
|
||||||
### How should pricing account for unequal usage?
|
### How should pricing account for unequal usage?
|
||||||
|
|
||||||
Members currently pay a flat $15/month for a $15/month credit budget. But usage is unequal — some members use far more tokens than others.
|
Members currently pay dues on a sliding scale (from $5/month) with a flat $15/month credit budget. But usage is unequal — some members use far more tokens than others.
|
||||||
|
|
||||||
- **Flat (current):** simple, predictable, but heavy users subsidize light users (or vice versa).
|
- **Flat (current):** simple, predictable, but heavy users subsidize light users (or vice versa).
|
||||||
- **Usage-based:** pay for what you use, but unpredictable and may discourage experimentation.
|
- **Usage-based:** pay for what you use, but unpredictable and may discourage experimentation.
|
||||||
|
|||||||
@@ -39,6 +39,47 @@ copy, the portal's Cloudron calls start returning `401` — which breaks
|
|||||||
provisioning (new members can't be created) and the member dashboard (its
|
provisioning (new members can't be created) and the member dashboard (its
|
||||||
usage/keys data is resolved through the portal's Cloudron calls).
|
usage/keys data is resolved through the portal's Cloudron calls).
|
||||||
|
|
||||||
|
## Chat image: spaCy model for document parsing (EPUB/DOCX)
|
||||||
|
|
||||||
|
Open WebUI's document parser (`unstructured`) lazily downloads the spaCy model
|
||||||
|
`en_core_web_sm` on first EPUB/DOCX parse — but the app directory is read-only
|
||||||
|
under Cloudron, so that install always fails ("Errno 30 Read-only file
|
||||||
|
system"). Fixed 2026-10-14 (issue #20) by pre-installing the model into
|
||||||
|
persistent storage:
|
||||||
|
|
||||||
|
- `/app/data/spacy_model/` holds the unpacked wheel
|
||||||
|
(SHA-verified `en_core_web_sm-3.8.0-py3-none-any.whl` from the pinned URL in
|
||||||
|
`unstructured/nlp/tokenize.py`).
|
||||||
|
- App env has `PYTHONPATH=/app/data/spacy_model` so `import en_core_web_sm`
|
||||||
|
resolves and the lazy download never triggers.
|
||||||
|
|
||||||
|
**Restore caveat:** a backup restore from before 2026-10-14 will NOT contain
|
||||||
|
the model dir — re-create it (download the pinned wheel, unzip into
|
||||||
|
`/app/data/spacy_model/`, verify `spacy.load("en_core_web_sm")` works in the
|
||||||
|
container). The `PYTHONPATH` env survives restores (it's in app config).
|
||||||
|
|
||||||
|
## Scheduled tasks (scheduler addon)
|
||||||
|
|
||||||
|
The portal and gateway run their own housekeeping via the Cloudron scheduler
|
||||||
|
addon, declared in each app's `CloudronManifest.json` under `addons.scheduler`:
|
||||||
|
|
||||||
|
- **portal**: `membership_sweep` (nightly 3am, `/app/code/sweep.sh` →
|
||||||
|
`POST /admin/sweep` with the `WEBHOOK_TOKEN` header) and `rag_refresh`
|
||||||
|
(nightly 3:30am, `/app/code/rag_refresh.py` — pulls docs from the Gitea raw
|
||||||
|
API and refreshes the chat knowledge base; needs `OWUI_API_KEY` +
|
||||||
|
`DOCS_KNOWLEDGE_ID` in the app env).
|
||||||
|
- **gateway**: `audio_keepalive` (every 10 minutes, `/app/code/audio_keepalive.py`
|
||||||
|
— pings the Tinfoil audio models locally to avoid the ~24s TEE cold-start;
|
||||||
|
reads the master key from `/app/data/.master_key`, which `start.sh` writes —
|
||||||
|
the env var `LITELLM_MASTER_KEY` is only exported into the litellm process).
|
||||||
|
|
||||||
|
**After any manifest change, verify the scheduler survived the deploy:** the
|
||||||
|
build log must show `setupAddons: setting up addon scheduler with options
|
||||||
|
{...}` listing your task(s), and each script must succeed when run manually
|
||||||
|
inside the container (`cloudron exec --app <id> -- <command>`). The scheduler
|
||||||
|
has historically gone missing from the manifest during edits — a deploy without
|
||||||
|
the setupAddons line means nightly reconciliation is NOT running.
|
||||||
|
|
||||||
## Custom apps have no env-var UI
|
## Custom apps have no env-var UI
|
||||||
|
|
||||||
The member portal is a *custom* app (packaged by us), not an App Store app.
|
The member portal is a *custom* app (packaged by us), not an App Store app.
|
||||||
|
|||||||
+14
-6
@@ -1,11 +1,11 @@
|
|||||||
# Privacy Policy
|
# Privacy Policy
|
||||||
|
|
||||||
**Last updated:** September 10, 2026
|
**Last updated:** October 6, 2026
|
||||||
|
|
||||||
## In plain language
|
## In plain language
|
||||||
|
|
||||||
- We collect only what we need: your email, name, and basic usage data.
|
- We collect only what we need: your email, name, and basic usage data.
|
||||||
- **Your AI conversations are encrypted** — even we can't read them at the inference step.
|
- We offer a **changing variety of models with different privacy and value affordances** — some run in encrypted enclaves we cannot read, others differ. **You are responsible for assessing each model's properties and picking the one that fits your needs.**
|
||||||
- We **do** store your chat history on our server (so you can revisit it). We promise not to read it, but this is a promise, not a technical guarantee.
|
- We **do** store your chat history on our server (so you can revisit it). We promise not to read it, but this is a promise, not a technical guarantee.
|
||||||
- We never train on your data, and we never sell it.
|
- We never train on your data, and we never sell it.
|
||||||
- You can export or delete your data at any time.
|
- You can export or delete your data at any time.
|
||||||
@@ -26,13 +26,20 @@ The Inference Cooperative is a member-governed project. We are **not** an ad-sup
|
|||||||
- **Content:** your prompts and the AI's responses (stored in your chat history).
|
- **Content:** your prompts and the AI's responses (stored in your chat history).
|
||||||
- **Billing data:** handled by Open Collective — we do not see full payment details.
|
- **Billing data:** handled by Open Collective — we do not see full payment details.
|
||||||
|
|
||||||
## 3. What we *can't* see (architectural privacy)
|
## 3. Privacy depends on the model you choose
|
||||||
|
|
||||||
Inference runs inside **encrypted enclaves** (TEEs). Your model inputs and outputs are encrypted end-to-end, so **even our own infrastructure cannot read them at the inference step.** This is verifiable through remote attestation — it's a property of the system, not a promise.
|
We serve a **changing catalog of models from multiple providers, and they do not all offer the same privacy**. Models differ in their affordances — some run inference inside **encrypted enclaves** (TEEs) where inputs and outputs are not readable in plaintext at the inference step; others are selected for renewable energy, openness, or cost rather than architectural privacy. The set of available models, and their properties, will change as members decide.
|
||||||
|
|
||||||
|
**You are responsible for assessing each model's privacy and value properties and choosing the one that fits your needs.** As a rule of thumb:
|
||||||
|
|
||||||
|
- Some models run inference in encrypted enclaves, verifiable through remote attestation — a property of the system, not a promise.
|
||||||
|
- Other models are processed in the clear under their provider's own terms and data-processing agreement.
|
||||||
|
|
||||||
|
We never read or sell your data regardless of model — but the *architectural* guarantee applies only to models that offer it. Check the current model catalog and each provider's terms before relying on a particular model for sensitive material.
|
||||||
|
|
||||||
## 4. What we *can* see (an honest caveat)
|
## 4. What we *can* see (an honest caveat)
|
||||||
|
|
||||||
Your **chat history is stored on our server** so you can revisit past conversations. Unlike the inference step, stored chat history is **not** encrypted in a way that prevents us from technically reading it.
|
Your **chat history is stored on our server** so you can revisit past conversations. Stored chat history is **not** encrypted in a way that prevents us from technically reading it, regardless of which model generated it.
|
||||||
|
|
||||||
We commit not to read it. But this is a **policy commitment**, not an architectural guarantee. We're transparent about this distinction because it matters.
|
We commit not to read it. But this is a **policy commitment**, not an architectural guarantee. We're transparent about this distinction because it matters.
|
||||||
|
|
||||||
@@ -52,7 +59,8 @@ Web search runs through our self-hosted search instance. Search queries are forw
|
|||||||
|
|
||||||
## 7. Third parties
|
## 7. Third parties
|
||||||
|
|
||||||
- **Tinfoil** — provides encrypted inference (cannot read your content).
|
The models we serve are provided by third parties that change over time, each with its own terms and privacy posture. We never sell your data, but when you use a model, that model's provider processes your inputs under its own terms and data-processing agreement — some in encrypted enclaves, some in the clear. Review each provider's terms before relying on it for sensitive material. Our current providers are listed in the model catalog.
|
||||||
|
|
||||||
- **Open Collective** — handles billing and membership (their own privacy policy applies).
|
- **Open Collective** — handles billing and membership (their own privacy policy applies).
|
||||||
- **Cloudron** — our self-hosted platform; no third-party cloud provider.
|
- **Cloudron** — our self-hosted platform; no third-party cloud provider.
|
||||||
- **Search engines** — receive your search queries (see §5).
|
- **Search engines** — receive your search queries (see §5).
|
||||||
|
|||||||
+27
-16
@@ -1,15 +1,15 @@
|
|||||||
# Terms of Service
|
# Terms of Service
|
||||||
|
|
||||||
**Last updated:** September 10, 2026
|
**Last updated:** October 6, 2026
|
||||||
|
|
||||||
## In plain language
|
## In plain language
|
||||||
|
|
||||||
- You are a **member**, not a customer. You help govern this project.
|
- You are a **member**, not a customer. You help govern this project.
|
||||||
- We provide **private AI chat** through open models, protected by encryption.
|
- We provide AI chat and API access through open models, with **a variety of privacy and value options** (some models run in encrypted enclaves; others differ — see §3).
|
||||||
- Your prompts and the AI's outputs are yours to keep.
|
- Your prompts and the AI's outputs are yours to keep.
|
||||||
- We don't train on your data, and we don't sell it.
|
- We don't train on your data, and we don't sell it.
|
||||||
- AI can be wrong. Use your judgment.
|
- AI can be wrong. Use your judgment.
|
||||||
- If you misuse the service, we may remove you (per our Charter).
|
- If you misuse the service, we may remove you (per our Charter) — and you are responsible for what you do with it (§4).
|
||||||
- Our liability is limited to what you've paid in dues.
|
- Our liability is limited to what you've paid in dues.
|
||||||
|
|
||||||
The full terms are below. If anything is unclear, write to info@inference.coop.
|
The full terms are below. If anything is unclear, write to info@inference.coop.
|
||||||
@@ -18,13 +18,14 @@ The full terms are below. If anything is unclear, write to info@inference.coop.
|
|||||||
|
|
||||||
## 1. Who we are
|
## 1. Who we are
|
||||||
|
|
||||||
The **Inference Cooperative** ("we", "us") is a member-governed project providing private AI inference. We are fiscally sponsored by **Metagov**, a nonprofit that provides our financial and legal services.
|
The **Inference Cooperative** ("we", "us") is a member-governed project providing AI inference. We are fiscally sponsored by **Metagov**, a nonprofit that provides our financial and legal structure.
|
||||||
|
|
||||||
When you join, you become a **member** — someone with a voice in how the project is run — not a customer of a vendor.
|
When you join, you become a **member** — someone with a voice in how the project is run — not a customer of a vendor.
|
||||||
|
|
||||||
## 2. Membership
|
## 2. Membership
|
||||||
|
|
||||||
- Membership is through **Open Collective**, where you pay dues on a sliding scale.
|
- Membership is through **Open Collective**, where you pay dues on a sliding
|
||||||
|
scale (currently $5/month minimum).
|
||||||
- You must be **18 or older** to join.
|
- You must be **18 or older** to join.
|
||||||
- Membership is governed by our [Charter](charter.md), including our Code of Conduct.
|
- Membership is governed by our [Charter](charter.md), including our Code of Conduct.
|
||||||
- We may suspend or remove membership for Code of Conduct violations or non-payment, as described in the Charter.
|
- We may suspend or remove membership for Code of Conduct violations or non-payment, as described in the Charter.
|
||||||
@@ -33,25 +34,34 @@ When you join, you become a **member** — someone with a voice in how the proje
|
|||||||
|
|
||||||
We provide:
|
We provide:
|
||||||
|
|
||||||
- **Private AI chat** through open-source models, served through encrypted (TEE-protected) infrastructure.
|
- **AI chat and a developer API** through a changing catalog of open models from multiple providers. You choose per task not just *which model*, but *which affordances it offers* — models differ in privacy architecture, energy source, openness, capability, and cost. Some run inference inside encrypted enclaves; others do not. Some run on renewable energy; others do not. The catalog changes as members decide, and the specific models and providers available will evolve.
|
||||||
- **Web search** and **file uploads** within the chat interface.
|
- **Web search** and **file uploads** within the chat interface.
|
||||||
- **Governance** — a seat in decisions about models, pricing, and direction.
|
- **Governance** — a seat in decisions about models, pricing, and direction.
|
||||||
|
|
||||||
|
Models are **not interchangeable in their affordances**. Each model has its own privacy, sustainability, and openness properties, and the set of available models changes over time. You are **responsible for assessing** which model fits your needs for a given task — in particular its privacy properties — before relying on it. See our [Privacy Policy](privacy-policy.md) for what this means in practice.
|
||||||
|
|
||||||
The service is provided **"as is"** and may change as the project evolves. We do not guarantee uptime or specific performance at this stage.
|
The service is provided **"as is"** and may change as the project evolves. We do not guarantee uptime or specific performance at this stage.
|
||||||
|
|
||||||
## 4. Acceptable use
|
## 4. Acceptable use
|
||||||
|
|
||||||
You agree not to:
|
You agree not to use the service to:
|
||||||
|
|
||||||
- Use the service for anything illegal.
|
- Violate any law or regulation.
|
||||||
- Harass, abuse, or harm others.
|
- Generate, distribute, or facilitate **illegal, harmful, abusive, or discriminatory content** — including content that sexualizes minors, defrauds, incites violence, or harasses others.
|
||||||
- Attempt to circumvent usage limits or the privacy architecture.
|
- Create **deepfakes, impersonations, or deceptive media** of real people without their consent.
|
||||||
- Share your account credentials or resell access.
|
- Process **regulated data** — including protected health information (e.g. HIPAA), payment-card data, or other categories that require a separate legal agreement — unless we have agreed to that in writing.
|
||||||
- Use the service to generate content that violates our [Code of Conduct](charter.md).
|
- Submit **special-category personal data** under GDPR Article 9 (health, ethnicity, political views, religion, biometrics, etc.) without a written agreement.
|
||||||
|
- Use the service in or for the benefit of **embargoed or sanctioned jurisdictions or persons**, or otherwise in violation of **export-control or sanctions law**.
|
||||||
|
- Use AI outputs as the **sole basis for consequential decisions** — legal, medical, financial, employment, housing, or safety-critical — without appropriate human review and professional oversight.
|
||||||
|
- Use the service for **high-risk or fail-safe** applications (e.g. critical infrastructure, life-support, weapons systems).
|
||||||
|
- Attempt to circumvent usage limits, the privacy architecture, or rate limits.
|
||||||
|
- Share your account credentials, or resell or redistribute access to the service.
|
||||||
|
|
||||||
|
You are **responsible for what you submit and what you do with AI outputs**, and you must have the rights needed to submit your content.
|
||||||
|
|
||||||
## 5. Usage limits
|
## 5. Usage limits
|
||||||
|
|
||||||
- Membership includes a **monthly credit budget** (currently $15 of usage).
|
- Membership includes a **monthly credit budget** (currently $15 of usage, the same for every member regardless of what you pay).
|
||||||
- Members may also purchase **credit packs**: non-expiring credits drawn only
|
- Members may also purchase **credit packs**: non-expiring credits drawn only
|
||||||
after the monthly budget is exhausted. Credits do not expire and carry over
|
after the monthly budget is exhausted. Credits do not expire and carry over
|
||||||
from month to month.
|
from month to month.
|
||||||
@@ -67,7 +77,7 @@ You agree not to:
|
|||||||
|
|
||||||
## 7. Privacy
|
## 7. Privacy
|
||||||
|
|
||||||
Your privacy is a core value, not an afterthought. Inference runs in encrypted enclaves so that even our own infrastructure cannot read your model inputs and outputs at the inference step. Details are in the [Privacy Policy](privacy-policy.md).
|
Privacy is a core value, but it is **not uniform across the service** — and the set of available models changes over time. Models differ in their privacy affordances: some run inference in encrypted enclaves so inputs and outputs are not readable in plaintext at the inference step, while others are selected for renewable energy or openness rather than architectural privacy. **You are responsible for assessing each model's privacy properties and choosing one that matches your needs.** See the [Privacy Policy](privacy-policy.md) for the current picture.
|
||||||
|
|
||||||
## 8. Termination
|
## 8. Termination
|
||||||
|
|
||||||
@@ -80,11 +90,12 @@ Your privacy is a core value, not an afterthought. Inference runs in encrypted e
|
|||||||
- Decisions are made through **Loomio** and governed by the [Charter](charter.md).
|
- Decisions are made through **Loomio** and governed by the [Charter](charter.md).
|
||||||
- During the pilot phase, the General Manager holds final discretion; this transitions to full member governance as the project grows.
|
- During the pilot phase, the General Manager holds final discretion; this transitions to full member governance as the project grows.
|
||||||
|
|
||||||
## 10. Disclaimers and liability
|
## 10. Disclaimers, liability, and indemnity
|
||||||
|
|
||||||
- **AI outputs may be inaccurate or inappropriate.** We provide no warranty of correctness. Use your judgment.
|
- **AI outputs may be inaccurate, biased, or inappropriate.** We provide no warranty of correctness. Use your judgment, and do not rely on outputs as a sole source of truth or as a substitute for professional advice.
|
||||||
- To the maximum extent permitted by law, our liability is limited to the amount of dues you have paid.
|
- To the maximum extent permitted by law, our liability is limited to the amount of dues you have paid.
|
||||||
- Members are not personally liable for the project's obligations.
|
- Members are not personally liable for the project's obligations.
|
||||||
|
- **You agree to indemnify and hold harmless** the Inference Cooperative, Metagov, and their officers, directors, employees, and agents from any claims, losses, liabilities, damages, costs, and expenses (including reasonable legal fees) arising from **your use of the service, your content, your violation of law or third-party rights, or your violation of these Terms**. In plain terms: if what you do with the service causes a legal problem, it is your responsibility, not the co-op's or Metagov's.
|
||||||
|
|
||||||
## 11. Changes to these terms
|
## 11. Changes to these terms
|
||||||
|
|
||||||
|
|||||||
Reference in new issue
Block a user