solo.istandClaude Fable 5.1
9faa403aee
The tool names the whole experiment, not just this script, so it gets a name with some personality. Repo, directory and command are `coopertron`; "Coopertron 5000" is the display name in the README and the report title. The activity the form captures is still a braindump. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
136 lines
6.7 KiB
Markdown
136 lines
6.7 KiB
Markdown
# Coopertron 5000
|
||
|
||
A benevolent robot that reads the co-op's anonymous pre-meeting braindump and
|
||
tells the members what they said. Built for Inference Cooperative meetings.
|
||
Members answer a short form before a meeting (ideas, wants, needs, goals,
|
||
dissent). This script turns the export into a review document the group reads
|
||
together: where energy is concentrated, what is unusual, where people disagree.
|
||
|
||
It runs on the co-op's **own gateway**, on a **Tinfoil TEE model** by default,
|
||
so anonymous responses are processed inside an enclave rather than handed to an
|
||
outside vendor. Python standard library only.
|
||
|
||
## How it works
|
||
|
||
```
|
||
CryptPad Form ──export CSV──▶ synthesize.py ──▶ out/review.md + out/synthesis.json
|
||
│
|
||
├─ tag model extracts atomic points per response,
|
||
│ each with a verbatim quote that Python verifies
|
||
├─ cluster model groups points into themes, flags novel
|
||
│ points and dissent
|
||
└─ render Python writes the document from the structured
|
||
data: counts and quotes are exact, the model
|
||
never writes the final text
|
||
```
|
||
|
||
The model only ever returns structured data. Theme sizes are counted from item
|
||
ids in Python, quotes are checked to be real substrings of the source, and the
|
||
appendix reprints every response verbatim so anyone can audit the synthesis.
|
||
|
||
## Running
|
||
|
||
Requires Python 3.10+ and the [1Password CLI](https://developer.1password.com/docs/cli/)
|
||
for the key (or set `INFERENCE_COOP_API_KEY` yourself).
|
||
|
||
```bash
|
||
# 1. Dry run: parse the export, estimate tokens and cost, make no calls
|
||
python3 synthesize.py responses.csv --dry-run
|
||
|
||
# 2. Real run
|
||
op run --env-file .env.op -- python3 synthesize.py responses.csv --out out/
|
||
```
|
||
|
||
Useful flags:
|
||
|
||
| Flag | Purpose |
|
||
|---|---|
|
||
| `--model` | default `tinfoil/glm-5.3`. Non-Tinfoil models are refused unless `--allow-non-tee` is passed |
|
||
| `--limit N` | first N responses only (smoke tests) |
|
||
| `--batch-size N` | responses per tagging call (default 15; keeps each call well inside the gateway's 180 s timeout) |
|
||
| `--drop-col H` / `--choice-col H` | ignore a column, or force a column to be tallied as a checkbox question |
|
||
| `--use-tools` | request structured output via a forced function call instead of a fenced JSON block |
|
||
| `--reasoning-effort` | `reasoning_effort` sent with every call; default `low` (see below). `default` sends nothing |
|
||
| `--no-thinking` | send `chat_template_kwargs.enable_thinking=false` (use with `tinfoil/deepseek-v4-1-flash`) |
|
||
| `--max-tokens N` | completion budget per tagging call (default 8000; the cluster call gets 1.5×) |
|
||
| `--save-raw DIR` | keep raw request/response JSON (fixtures, debugging) |
|
||
| `--no-cache` | bypass `.cache/` (every call is cached by content hash, so prompt tuning reruns are free) |
|
||
|
||
Exit codes: `0` ok, `1` nothing extracted, `2` refused (non-TEE model without the flag).
|
||
|
||
### Reasoning models on the gateway (probed 2026-10-19)
|
||
|
||
The Tinfoil chat models are reasoning models. Left alone on a real extraction
|
||
batch, `tinfoil/glm-5.3` spent its entire 8000-token completion budget on hidden
|
||
reasoning and returned **no content** (`finish_reason: length`,
|
||
`reasoning_tokens: 8000`). The script now stops immediately in that case
|
||
instead of retrying and billing the same tokens again. Small probes of the
|
||
available knobs:
|
||
|
||
| Model | Setting | Result |
|
||
|---|---|---|
|
||
| `tinfoil/glm-5.3` | default | thinks, then answers; reasoning scales with task size and can exhaust the budget |
|
||
| `tinfoil/glm-5.3` | `reasoning_effort: low` | **a few reasoning tokens, clean JSON** (the default here) |
|
||
| `tinfoil/glm-5.3` | `reasoning_effort: none` or `chat_template_kwargs.enable_thinking=false` | reasoning tokens drop to 0 but the thinking leaks into the visible content; avoid |
|
||
| `tinfoil/deepseek-v4-1-flash` | `--no-thinking` | clean JSON, no reasoning, cheaper; good fallback |
|
||
| `tinfoil/gpt-oss-120b` | default or `reasoning_effort: low` | clean JSON; does not report reasoning tokens |
|
||
|
||
The gateway forwards these parameters unchanged, so top-level
|
||
`reasoning_effort` and `chat_template_kwargs` both reach the model.
|
||
|
||
## Input format
|
||
|
||
A CSV whose header row is the questions and each later row is one response,
|
||
which is what CryptPad Form's **Responses → EXPORT → CSV** produces. Columns
|
||
named like `Time`, `Timestamp`, `ID` are ignored; fully empty columns are
|
||
ignored. A column whose cells are short comma- or semicolon-separated tokens is
|
||
treated as a checkbox question and tallied in Python. Check the `Q:` lines the
|
||
script prints to confirm it read the header the way you expect.
|
||
|
||
`tests/fixtures/sample.csv` is a synthetic 12-response export in that shape.
|
||
|
||
## Output
|
||
|
||
- `out/review.md`: headline numbers, priorities tally, themes ranked by how
|
||
many distinct people raised them (with for/against counts and verbatim
|
||
quotes), standout single-person ideas, notable dissent, open questions, and
|
||
an appendix with every response.
|
||
- `out/synthesis.json`: the structured intermediate (items, clusters, token
|
||
usage) for auditing or re-rendering.
|
||
|
||
`tests/fixtures/sample.review.md` shows the shape, rendered offline from a fake
|
||
model so the themes there are placeholders.
|
||
|
||
## Privacy and cost
|
||
|
||
- Responses are sent to `gateway.inference.coop`. With the default model they
|
||
are processed inside a Tinfoil TEE enclave; the gateway sees metering
|
||
metadata (tokens, model, key) and, per the co-op's docs, does not store prompt
|
||
content. Other providers on the gateway are policy-based privacy, which is
|
||
why the script refuses them without an explicit flag.
|
||
- The API key is read from the environment only. Use `op run` so it never
|
||
lands in a file or shell history. The script never prints request headers.
|
||
- Cost: a 50-response, 8-question survey is on the order of 60k input tokens
|
||
and a few k output through GLM-5.3, well under $1 of a member allowance.
|
||
`--dry-run` prints the estimate; the real run prints the actual usage.
|
||
|
||
## Tests
|
||
|
||
```bash
|
||
python3 -m unittest -v
|
||
```
|
||
|
||
Offline. A fake gateway returns canned JSON so the tests cover CSV parsing,
|
||
quote verification, JSON extraction and retry, caching, and rendering.
|
||
|
||
## Status
|
||
|
||
Built for the first-week-of-November 2026 members' meeting. The form itself
|
||
lives on CryptPad for now; a small co-op-hosted app (anonymous form + a
|
||
synthesize button, same pipeline) is a possible next step if the output is
|
||
useful. Discussion and ideas: open an issue here or on
|
||
[co-op/support](https://git.inference.coop/co-op/support/issues).
|
||
|
||
Why the name: it is a small experiment with a big-robot name, on purpose. The
|
||
repo and command are `coopertron`; the 5000 lives in prose only.
|