Merge pull request #14: docs(search): describe the search and fetch pipeline
This commit was merged in pull request #14.
This commit is contained in:
commit
1f23ea1d78
3 files changed
+55
-8
No files matched your search
@@ -83,13 +83,33 @@ Our server software keeps standard operational logs (dates, error messages, and
|
|||||||
|
|
||||||
## 5. Search
|
## 5. Search
|
||||||
|
|
||||||
Web search runs through our self-hosted search instance. For chat searches we use DuckDuckGo and Brave; those services receive your query from our server, and may see it. This is inherent to how web search works. Search queries and results are part of the conversation record, so they follow that conversation's retention: deleted when you delete a saved chat, or about an hour after a Temporary Chat is created. Operational details are in [Infrastructure](../operations/infrastructure.md#chat-retention-and-web-search).
|
Our self-hosted **SearXNG** instance at
|
||||||
|
[search.inference.coop](https://search.inference.coop) handles web search. For
|
||||||
|
chat searches it forwards queries to **DuckDuckGo** and **Brave**; those
|
||||||
|
services receive the query from our server and may see it. Search queries and
|
||||||
|
results are part of the conversation record, so they follow that conversation's
|
||||||
|
retention: deleted when you delete a saved chat, or about an hour after a
|
||||||
|
Temporary Chat is created. Operational details are in
|
||||||
|
[Infrastructure](../operations/infrastructure.md#chat-retention-and-web-search).
|
||||||
|
|
||||||
|
To read the pages behind the results, LibreChat sends the top result URLs to
|
||||||
|
**Keenable**, a third-party page-fetch service. We use Keenable's keyless public
|
||||||
|
endpoint, so its default data policy applies. Keenable logs and stores the
|
||||||
|
fetched URL, the page content, the returned results, and our server's IP address
|
||||||
|
and user agent. Keenable's policy sets no fixed retention period. Keenable
|
||||||
|
processes data in the United States and in the countries where its service
|
||||||
|
providers operate. Keenable may use content and usage data to improve and train
|
||||||
|
its products. Your chat messages and account identity stay with us.
|
||||||
|
|
||||||
|
Web search is optional. Leave the globe off for sensitive topics. If you need
|
||||||
|
search without a third-party page fetch, contact us about alternatives.
|
||||||
|
|
||||||
## 6. What we do NOT do
|
## 6. What we do NOT do
|
||||||
|
|
||||||
- We do **not** train models on your data.
|
- We do **not** train models on your data.
|
||||||
- We do **not** sell, rent, or share your data.
|
- We do **not** sell or rent your data.
|
||||||
- We do **not** use your data for advertising or profiling.
|
- We do **not** share data for advertising or profiling. The service providers
|
||||||
|
in §5 and §7 process data to run the service.
|
||||||
|
|
||||||
## 7. Third parties
|
## 7. Third parties
|
||||||
|
|
||||||
@@ -97,7 +117,9 @@ The models we serve are provided by third parties that change over time, each wi
|
|||||||
|
|
||||||
- **Open Collective** — handles billing and membership (their own privacy policy applies).
|
- **Open Collective** — handles billing and membership (their own privacy policy applies).
|
||||||
- **Cloudron** — our self-hosted platform; no third-party cloud provider.
|
- **Cloudron** — our self-hosted platform; no third-party cloud provider.
|
||||||
- **Search engines** — receive your search queries (see §5).
|
- **Search engines.** DuckDuckGo and Brave receive your search queries (see §5).
|
||||||
|
- **Keenable.** This page-fetch service receives result URLs and page content
|
||||||
|
(see §5).
|
||||||
|
|
||||||
## 8. Retention and deletion
|
## 8. Retention and deletion
|
||||||
|
|
||||||
@@ -110,7 +132,8 @@ The models we serve are provided by third parties that change over time, each wi
|
|||||||
|
|
||||||
- Self-hosted on Cloudron (sandboxed apps, SSL, backups).
|
- Self-hosted on Cloudron (sandboxed apps, SSL, backups).
|
||||||
- Per-member API keys; access gated on live membership.
|
- Per-member API keys; access gated on live membership.
|
||||||
- No external dependencies for core infrastructure.
|
- Core infrastructure has no external dependencies. Web search uses the
|
||||||
|
third-party services described in §5.
|
||||||
|
|
||||||
## 10. Your rights
|
## 10. Your rights
|
||||||
|
|
||||||
|
|||||||
@@ -56,8 +56,10 @@ with the account you created in step 1.
|
|||||||
(GreenPT and Regolo, 100% renewable energy — Regolo adds a written
|
(GreenPT and Regolo, 100% renewable energy — Regolo adds a written
|
||||||
zero-data-retention policy). See the [model list](models.md) for the full
|
zero-data-retention policy). See the [model list](models.md) for the full
|
||||||
list.
|
list.
|
||||||
- **Web search** — click the **globe** next to the message box to ground
|
- **Web search.** Click the **globe** next to the message box to base answers
|
||||||
answers in current web results (self-hosted search, no external dependency).
|
on current web results. Searches use our self-hosted SearXNG instance and
|
||||||
|
Keenable, a third-party page-fetch service. See [what happens during a
|
||||||
|
search](#what-happens-when-you-use-web-search).
|
||||||
- **File uploads** — attach a document with the **paperclip** and ask the
|
- **File uploads** — attach a document with the **paperclip** and ask the
|
||||||
chat about it.
|
chat about it.
|
||||||
- **Tools** — the **Tools menu** next to the message box lists everything the
|
- **Tools** — the **Tools menu** next to the message box lists everything the
|
||||||
@@ -84,6 +86,22 @@ see "Your allowance" below.
|
|||||||
> — your full history is here in the new chat, but files you attached in the
|
> — your full history is here in the new chat, but files you attached in the
|
||||||
> old one are still accessible only there.
|
> old one are still accessible only there.
|
||||||
|
|
||||||
|
### What happens when you use web search
|
||||||
|
|
||||||
|
Web search runs in two stages.
|
||||||
|
|
||||||
|
1. Our self-hosted **SearXNG** instance at
|
||||||
|
[search.inference.coop](https://search.inference.coop) sends your query to
|
||||||
|
**DuckDuckGo** and **Brave**.
|
||||||
|
2. LibreChat sends the top result URLs to **Keenable**, a third-party
|
||||||
|
page-fetch service. Keenable reads each page and extracts its text. The model
|
||||||
|
receives those passages, and the answer cites the sources.
|
||||||
|
|
||||||
|
SearXNG runs on co-op infrastructure. Keenable is an external service. The
|
||||||
|
[privacy policy](../governance/privacy-policy.md#5-search) describes how
|
||||||
|
Keenable handles fetched URLs and page content. Web search is optional. Leave
|
||||||
|
the globe off if you do not want pages fetched.
|
||||||
|
|
||||||
### Add the chat to your phone
|
### Add the chat to your phone
|
||||||
|
|
||||||
The chat works like an app on your phone — no app store needed. Open
|
The chat works like an app on your phone — no app store needed. Open
|
||||||
|
|||||||
@@ -73,8 +73,10 @@ The co-op's software behind all of these is open source — see [The
|
|||||||
code](#the-code) below for where each piece lives.
|
code](#the-code) below for where each piece lives.
|
||||||
|
|
||||||
Open Collective (billing and fiscal sponsorship) is the one external service in
|
Open Collective (billing and fiscal sponsorship) is the one external service in
|
||||||
the flow — see
|
the membership flow. See
|
||||||
[opencollective.com/inference-cooperative](https://opencollective.com/inference-cooperative).
|
[opencollective.com/inference-cooperative](https://opencollective.com/inference-cooperative).
|
||||||
|
Web search uses external services too; see
|
||||||
|
[Chat retention and web search](#chat-retention-and-web-search).
|
||||||
|
|
||||||
## Mail
|
## Mail
|
||||||
|
|
||||||
@@ -199,6 +201,10 @@ contain query text when an upstream engine request fails; those log lines carry
|
|||||||
no member identity or IP, and are retained only briefly. The reverse proxy does
|
no member identity or IP, and are retained only briefly. The reverse proxy does
|
||||||
not log search queries.
|
not log search queries.
|
||||||
|
|
||||||
|
LibreChat reads the pages behind the results through **Keenable**, a third-party
|
||||||
|
page-fetch service. Keenable's logging and retention are described in the
|
||||||
|
[privacy policy](../governance/privacy-policy.md#5-search).
|
||||||
|
|
||||||
## Security posture (plain language)
|
## Security posture (plain language)
|
||||||
|
|
||||||
- **Member isolation.** Each API key is scoped to its owner's team and budget; a
|
- **Member isolation.** Each API key is scoped to its owner's team and budget; a
|
||||||
|
|||||||
Reference in new issue
Block a user