Files
coopertron/README.md
T
solo.istandClaude Fable 5.1 05f4e88867 synthesize: anonymous braindump CSV → themed review doc via the co-op gateway
Stdlib-only pipeline for the pre-meeting braindump: CryptPad Form CSV in,
review.md + synthesis.json out. Two model stages (tag atomic points with
verbatim quotes, cluster into themes/novel/dissent) wrapped in deterministic
Python that verifies every quote against the source and computes counts from
item ids, so the model never writes the final document. Defaults to
tinfoil/glm-5.3 with reasoning_effort=low: without it the model spent the
whole 8000-token budget on hidden reasoning and returned nothing (probed
2026-10-19; findings in README). Refuses non-TEE models without a flag.
Content-hash cache so prompt tuning reruns are free. Offline unittest suite
with a fake gateway; smoke-tested live on 5 synthetic responses for ~$0.03.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-19 02:56:50 -04:00

6.5 KiB
Raw Blame History

braindump

Anonymous braindump form + LLM synthesis for Inference Cooperative meetings. Members answer a short form before a meeting (ideas, wants, needs, goals, dissent). This script turns the export into a review document the group reads together: where energy is concentrated, what is unusual, where people disagree.

It runs on the co-op's own gateway, on a Tinfoil TEE model by default, so anonymous responses are processed inside an enclave rather than handed to an outside vendor. Python standard library only.

How it works

CryptPad Form ──export CSV──▶ synthesize.py ──▶ out/review.md + out/synthesis.json
                               │
                               ├─ tag      model extracts atomic points per response,
                               │           each with a verbatim quote that Python verifies
                               ├─ cluster  model groups points into themes, flags novel
                               │           points and dissent
                               └─ render   Python writes the document from the structured
                                           data: counts and quotes are exact, the model
                                           never writes the final text

The model only ever returns structured data. Theme sizes are counted from item ids in Python, quotes are checked to be real substrings of the source, and the appendix reprints every response verbatim so anyone can audit the synthesis.

Running

Requires Python 3.10+ and the 1Password CLI for the key (or set INFERENCE_COOP_API_KEY yourself).

# 1. Dry run: parse the export, estimate tokens and cost, make no calls
python3 synthesize.py responses.csv --dry-run

# 2. Real run
op run --env-file .env.op -- python3 synthesize.py responses.csv --out out/

Useful flags:

Flag Purpose
--model default tinfoil/glm-5.3. Non-Tinfoil models are refused unless --allow-non-tee is passed
--limit N first N responses only (smoke tests)
--batch-size N responses per tagging call (default 15; keeps each call well inside the gateway's 180 s timeout)
--drop-col H / --choice-col H ignore a column, or force a column to be tallied as a checkbox question
--use-tools request structured output via a forced function call instead of a fenced JSON block
--reasoning-effort reasoning_effort sent with every call; default low (see below). default sends nothing
--no-thinking send chat_template_kwargs.enable_thinking=false (use with tinfoil/deepseek-v4-1-flash)
--max-tokens N completion budget per tagging call (default 8000; the cluster call gets 1.5×)
--save-raw DIR keep raw request/response JSON (fixtures, debugging)
--no-cache bypass .cache/ (every call is cached by content hash, so prompt tuning reruns are free)

Exit codes: 0 ok, 1 nothing extracted, 2 refused (non-TEE model without the flag).

Reasoning models on the gateway (probed 2026-10-19)

The Tinfoil chat models are reasoning models. Left alone on a real extraction batch, tinfoil/glm-5.3 spent its entire 8000-token completion budget on hidden reasoning and returned no content (finish_reason: length, reasoning_tokens: 8000). The script now stops immediately in that case instead of retrying and billing the same tokens again. Small probes of the available knobs:

Model Setting Result
tinfoil/glm-5.3 default thinks, then answers; reasoning scales with task size and can exhaust the budget
tinfoil/glm-5.3 reasoning_effort: low a few reasoning tokens, clean JSON (the default here)
tinfoil/glm-5.3 reasoning_effort: none or chat_template_kwargs.enable_thinking=false reasoning tokens drop to 0 but the thinking leaks into the visible content; avoid
tinfoil/deepseek-v4-1-flash --no-thinking clean JSON, no reasoning, cheaper; good fallback
tinfoil/gpt-oss-120b default or reasoning_effort: low clean JSON; does not report reasoning tokens

The gateway forwards these parameters unchanged, so top-level reasoning_effort and chat_template_kwargs both reach the model.

Input format

A CSV whose header row is the questions and each later row is one response, which is what CryptPad Form's Responses → EXPORT → CSV produces. Columns named like Time, Timestamp, ID are ignored; fully empty columns are ignored. A column whose cells are short comma- or semicolon-separated tokens is treated as a checkbox question and tallied in Python. Check the Q: lines the script prints to confirm it read the header the way you expect.

tests/fixtures/sample.csv is a synthetic 12-response export in that shape.

Output

  • out/review.md: headline numbers, priorities tally, themes ranked by how many distinct people raised them (with for/against counts and verbatim quotes), standout single-person ideas, notable dissent, open questions, and an appendix with every response.
  • out/synthesis.json: the structured intermediate (items, clusters, token usage) for auditing or re-rendering.

tests/fixtures/sample.review.md shows the shape, rendered offline from a fake model so the themes there are placeholders.

Privacy and cost

  • Responses are sent to gateway.inference.coop. With the default model they are processed inside a Tinfoil TEE enclave; the gateway sees metering metadata (tokens, model, key) and, per the co-op's docs, does not store prompt content. Other providers on the gateway are policy-based privacy, which is why the script refuses them without an explicit flag.
  • The API key is read from the environment only. Use op run so it never lands in a file or shell history. The script never prints request headers.
  • Cost: a 50-response, 8-question survey is on the order of 60k input tokens and a few k output through GLM-5.3, well under $1 of a member allowance. --dry-run prints the estimate; the real run prints the actual usage.

Tests

python3 -m unittest -v

Offline. A fake gateway returns canned JSON so the tests cover CSV parsing, quote verification, JSON extraction and retry, caching, and rendering.

Status

Built for the first-week-of-November 2026 members' meeting. The form itself lives on CryptPad for now; a small co-op-hosted app (anonymous form + a synthesize button, same pipeline) is a possible next step if the output is useful. Discussion and ideas: open an issue here or on co-op/support.