Add design doc for Create feature and Creator role
This commit is contained in:
Dan committed
1 parent
1e1408ad7f
commit
f79ce000e4
1 file changed
+261
@@ -0,0 +1,261 @@
|
||||
# DrawIt — Design: "Create" (AI-generated custom lessons) + the Creator role
|
||||
|
||||
_Designed June 2026. Purpose: let a privileged user type a subject (e.g. "Zebra"), have a self-hosted
|
||||
**ComfyUI** server generate a coloring-book line-art image, and turn that image into a real DrawIt
|
||||
lesson with the usual design elements. This is a design/research document — nothing is built yet. It
|
||||
assumes the current curriculum/runner architecture documented in `RESEARCH-beginner-level.md` and
|
||||
`RESEARCH-early-beginner-expansion.md`._
|
||||
|
||||
## 1. Goal & summary
|
||||
|
||||
A new **"Create"** option (visible only to a new **Creator** role) appears on the Early Beginner and
|
||||
Beginner levels. The Creator types a subject; the app sends a fixed prompt to a configurable ComfyUI
|
||||
endpoint; ComfyUI returns black-and-white line art; DrawIt processes that image into a playable lesson
|
||||
(trace + color phases, badge, progress, gallery — the same machinery every other lesson uses).
|
||||
|
||||
The fixed prompt (subject interpolated):
|
||||
|
||||
> A coloring book page of a **[subject]**, clean black-and-white line art, bold crisp outlines, simple
|
||||
> composition, large open areas to color, white background, no shading, no grayscale, no color, no
|
||||
> gradients, no heavy background detail, printable page.
|
||||
|
||||
Two hard problems drive the design: **(a)** an AI returns *one raster image*, but our lessons are
|
||||
multi-phase, SVG-step constructions; and **(b)** this is a children's app accepting free-text that
|
||||
feeds an image generator, so **safety/moderation is mandatory, not optional.**
|
||||
|
||||
## 2. The Creator role
|
||||
|
||||
Today `Role = "admin" | "learner"` (`src/lib/types.ts`); the DB `users.role` is already a free-text
|
||||
column defaulting to `learner`, and the first registered account is forced to `admin`. Adding a role is
|
||||
small and backward-compatible:
|
||||
|
||||
- **Type:** `Role = "admin" | "learner" | "creator"`. A Creator has **all Learner permissions plus the
|
||||
Create ability**. Admins implicitly have it too.
|
||||
- **No schema change** — `role` is already TEXT. Existing rows stay `learner`/`admin`.
|
||||
- **Assignment:** only an admin grants it, via the existing admin users API
|
||||
(`src/app/api/admin/users/[id]/route.ts`) — add `"creator"` to the allowed set (today it coerces to
|
||||
`admin`/`learner`). Signup is unchanged (first = admin, everyone else = learner).
|
||||
- **Server guard:** add `requireCreator()` in `src/lib/session.ts` (mirrors `requireAdmin()`):
|
||||
returns the user iff `role === "creator" || role === "admin"`. A tiny `canCreate(user)` helper feeds
|
||||
the UI. **Every Create endpoint must call this server-side** — never trust the client.
|
||||
|
||||
## 3. End-to-end user flow
|
||||
|
||||
1. Creator opens Early Beginner or Beginner → taps **✨ Create your own**.
|
||||
2. Types a subject ("Zebra"). Client does light validation; server does the real moderation (§5).
|
||||
3. Server creates a `created_lessons` row (`status: 'pending'`), interpolates the subject into the
|
||||
fixed prompt, and queues the ComfyUI workflow (`status: 'generating'`).
|
||||
4. Client polls status (or subscribes). ComfyUI renders (seconds to ~1 min on a GPU box).
|
||||
5. Server fetches the image, post-processes it (§6), derives the lesson assets, sets `status: 'ready'`
|
||||
(or `'needs_review'` if moderation requires admin approval).
|
||||
6. The lesson appears under a **"My Creations"** group on that level and plays through the normal
|
||||
runners (trace → color), earning a badge and saving to the gallery like any lesson.
|
||||
|
||||
## 4. ComfyUI integration
|
||||
|
||||
**Why ComfyUI / self-hosted:** it keeps DrawIt's privacy promise — the subject text and image never
|
||||
leave a server the operator runs. No third-party AI cloud. The endpoint is **configurable** so each
|
||||
self-hoster points at their own GPU box (or none, disabling Create).
|
||||
|
||||
**Config (env, mirroring the existing SMTP pattern in `.env.example`):**
|
||||
|
||||
```
|
||||
COMFYUI_URL= # e.g. http://127.0.0.1:8188 — blank disables the Create feature
|
||||
COMFYUI_API_KEY= # optional, if the box is behind an auth proxy
|
||||
COMFYUI_WORKFLOW=./comfyui/coloring-book.workflow_api.json # API-format workflow template
|
||||
COMFYUI_TIMEOUT_MS=120000
|
||||
COMFYUI_PROMPT_NODE=6 # id of the positive CLIP-text node to inject the prompt into
|
||||
```
|
||||
|
||||
**API calls** (ComfyUI's HTTP API):
|
||||
|
||||
1. `POST {COMFYUI_URL}/prompt` with `{ prompt: <workflow_api_json>, client_id }`. We load the saved
|
||||
**API-format workflow** (exported from ComfyUI's "Save (API format)"), and set the positive prompt
|
||||
node's `inputs.text` to our interpolated prompt (and randomize the seed node). Returns `prompt_id`.
|
||||
2. Poll `GET {COMFYUI_URL}/history/{prompt_id}` until the entry exists; read
|
||||
`outputs[node].images[]` → `{ filename, subfolder, type }`. (Optionally subscribe to
|
||||
`GET /ws?clientId=` for live progress to show a friendly "drawing your zebra…" bar.)
|
||||
3. `GET {COMFYUI_URL}/view?filename=…&subfolder=…&type=output` → the PNG bytes.
|
||||
|
||||
**Async model:** generation is slow, so treat it as a job. `created_lessons.status` is the state
|
||||
machine: `pending → generating → (needs_review) → ready | failed | blocked`. The POST route kicks off
|
||||
generation server-side and returns immediately; the client polls `GET /api/create/:id`. On a single
|
||||
self-hosted node an in-process async task is fine; document that horizontal scaling would want a real
|
||||
queue. Always set a timeout and a retry-once policy; surface `failed` with a kid-friendly message.
|
||||
|
||||
## 5. Safety & moderation (mandatory — this is a kids' app)
|
||||
|
||||
Free text → image generation for children is the riskiest part. Layered defenses, **input + output**:
|
||||
|
||||
**Input (the subject string):**
|
||||
- **Sanitize:** trim; cap length (~30 chars); collapse whitespace; allow only letters/spaces/hyphens;
|
||||
reject newlines and any prompt-control tokens. Because we only interpolate `[subject]` into a fixed
|
||||
prompt, prompt-injection surface is small — but still strip anything that could break out of the
|
||||
phrase.
|
||||
- **Allowlist-first (recommended for kids):** maintain a curated allowlist of safe, drawable subjects
|
||||
(animals, objects, food, vehicles, nature, shapes…). If the subject is on the allowlist → proceed
|
||||
automatically. This is the safest posture and the literature's recommendation for child contexts.
|
||||
- **Blocklist backstop:** a denylist of profanity, violence, weapons, drugs, sexual, hateful, and
|
||||
self-harm terms (and obfuscations) rejects anything obviously unsafe that slips past.
|
||||
- **Anything else → admin review queue** (`status: 'needs_review'`): not on the allowlist and not on
|
||||
the blocklist → a human approves before it generates or goes live. Keeps the door open to new
|
||||
subjects without exposing kids to raw generation.
|
||||
- **Rate-limit** per user (e.g. N creations/day) to bound abuse and GPU cost.
|
||||
|
||||
**Output (the generated image):** even safe prompts can misfire. Before a lesson is playable, require a
|
||||
**preview/approve step** — minimally the creating user sees it and confirms; for stricter operators, an
|
||||
admin approval queue. (Optional later: an automated NSFW image classifier as a pre-filter.)
|
||||
|
||||
**Logging vs. privacy:** store only what's needed to run/curate the feature (subject, status,
|
||||
moderation decision). Consistent with "no analytics, data stays local."
|
||||
|
||||
## 6. From image → level (the core pipeline)
|
||||
|
||||
**What a lesson needs today:** a `Lesson` with phases. Trace phases (`construct`/`outline`/`detail`)
|
||||
use per-step `lines: string[]` of SVG elements that `LineDraw` reveals one at a time; color/light
|
||||
phases use `baseSvg` which `ColoringRunner` **rasterizes** for the fill boundary and overlay. Badge,
|
||||
`sublevel`, `subjectKey`, and a group complete the level.
|
||||
|
||||
**The gap:** ComfyUI returns *one raster PNG*. It is not pre-split into outline-vs-detail phases, and
|
||||
it is pixels, not SVG strokes. So a Created lesson can't perfectly mirror a hand-authored 3–4-phase
|
||||
subject. **Chosen approach (per §13): Option C — vectorize the line art with potrace for a
|
||||
stroke-by-stroke Trace phase, and use the raster for the Color phase (2 phases total).** The three
|
||||
options below are kept for context; the build targets C.
|
||||
|
||||
- **Option A — Raster-native, 2-phase (recommended MVP).** Treat the cleaned PNG as the lesson art.
|
||||
- **Trace** phase: show the line art faintly as a guide to trace over (reuse `TraceRunner`'s base
|
||||
layer, fed a raster instead of SVG). Optional "watch it appear" = a left-to-right wipe reveal of
|
||||
the bitmap rather than true per-stroke animation.
|
||||
- **Color** phase: `ColoringRunner` already rasterizes its template to a boundary + overlay, so it
|
||||
can take the PNG **directly** — flood-fill and overlay work as-is. This phase is essentially free.
|
||||
- For Beginner-created lessons, optionally add the existing **Light** phase (sun overlay) on top.
|
||||
- Smallest change: add a `templateUrl?` (raster) alongside `baseSvg` to `TraceRunner`/`ColoringRunner`
|
||||
and the lesson-page meta; when present, use the bitmap path.
|
||||
|
||||
- **Option B — Vectorize for true line-by-line reveal.** Run the PNG through **potrace** (pure-JS npm,
|
||||
no native build — important given our better-sqlite3 build friction) to get SVG paths, then split the
|
||||
SVG into multiple `lines` (by `M` subpaths) so `LineDraw` animates it stroke-by-stroke like the
|
||||
hand-made lessons. Quality varies with image cleanliness; treat as an enhancement over A.
|
||||
|
||||
- **Option C — Hybrid (target end state).** Vectorize for the trace reveal (B) **and** keep the raster
|
||||
for coloring (A). Optionally heuristically split paths into a simpler "outline" (large/long paths)
|
||||
vs "detail" (small/short paths) pass to approximate the two trace phases — best-effort, not perfect.
|
||||
|
||||
**Image post-processing** (server-side, before storing) with a pure-JS lib like **jimp** (avoids native
|
||||
deps) or **sharp** (faster, native): grayscale → threshold to crisp black/white → auto-trim margins →
|
||||
fit to the lessons' aspect (the runners use a 4:3 / 400×300 viewBox and an 880×660 color canvas) →
|
||||
flatten on white → downscale. Store the cleaned PNG; if vectorizing, store the derived SVG too.
|
||||
|
||||
**Badge & metadata:** Created lessons get a generic medal — a default emoji (✏️ or 🎨, or a
|
||||
creator-picked one) plus a name like "**Zebra Artist**". Reuse the existing emoji-medal `BadgeArt` and
|
||||
award-PDF machinery untouched.
|
||||
|
||||
## 7. Data model
|
||||
|
||||
No destructive changes; one new table (plus the `Role` union value):
|
||||
|
||||
```sql
|
||||
CREATE TABLE created_lessons (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
level TEXT NOT NULL, -- 'early-beginner' | 'beginner'
|
||||
subject TEXT NOT NULL, -- sanitized, e.g. 'Zebra'
|
||||
slug TEXT NOT NULL UNIQUE, -- e.g. 'created-42-zebra'
|
||||
status TEXT NOT NULL DEFAULT 'pending', -- pending|generating|needs_review|ready|failed|blocked
|
||||
image TEXT, -- cleaned line-art (data URL, like drawings/avatars)
|
||||
template_svg TEXT, -- optional vectorized paths (Option B/C)
|
||||
prompt TEXT NOT NULL, -- full prompt sent (audit)
|
||||
moderation TEXT, -- 'allowlist' | 'review' | 'blocked:<reason>'
|
||||
emoji TEXT, -- badge emoji
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
);
|
||||
```
|
||||
|
||||
**Integration with existing systems (reuse, don't fork):**
|
||||
- **Sublevel namespacing:** created lessons keep their `level` (so the unlock gate / grouping still
|
||||
apply) and use a high synthetic `sublevel = 900000 + id`, guaranteeing no collision with static
|
||||
lessons. Progress, completions, drawings, and badges then work through the *existing* tables keyed by
|
||||
`(user_id, level, sublevel)` with zero changes.
|
||||
- **Badge key:** `created-<id>`; `badgeDefs()` gains the user's ready created lessons at request time.
|
||||
- **Not in static `LESSONS`.** Add resolver helpers that the lesson page and learn page call:
|
||||
`getCreatedLesson(slug)` builds a `Lesson`-shaped object on the fly from the row (so the runners need
|
||||
no special-casing), and `listCreatedLessons(userId, level)` feeds the "My Creations" group.
|
||||
- **Images as data URLs in SQLite** matches the existing privacy-consistent storage for drawings and
|
||||
avatars (note: line-art PNGs are small once thresholded; watch row size, consider a files dir if big).
|
||||
- **Backups:** the new table rides along with the existing Litestream/SQLite backup automatically.
|
||||
|
||||
## 8. API routes (Next.js App Router)
|
||||
|
||||
- `POST /api/create` — `requireCreator`; body `{ level, subject }`. Validate + moderate (§5). On pass:
|
||||
insert row, interpolate prompt, kick off ComfyUI job, return `{ id, slug, status }`. On block: 422
|
||||
with a gentle message.
|
||||
- `GET /api/create/:id` — owner (or admin); returns `{ status, slug, image? }` for polling.
|
||||
- `GET /api/create/:id/image` — serves the stored line art (or inline data URL).
|
||||
- `POST /api/admin/create/:id` — admin approve/block for the review queue.
|
||||
- All Create routes are no-ops/404 when `COMFYUI_URL` is unset (feature disabled cleanly).
|
||||
|
||||
## 9. UI
|
||||
|
||||
- **Entry point:** a "✨ Create your own" card at the top of the Early Beginner and Beginner sections of
|
||||
the Learn page, **rendered only when `canCreate(user)`**. (Also a nav entry, optional.)
|
||||
- **Create view:** subject input + the fixed-prompt explainer ("we'll draw a coloring page of your
|
||||
subject"), a live status/progress state, then a preview with **"Start the lesson"** / **"Try again"**.
|
||||
- **My Creations group:** a `getGroupedSubjects`-style group on the level listing the user's ready
|
||||
creations, each opening into the normal trace/color runner.
|
||||
- **Gating:** non-creators never see Create (UI), and the API rejects them regardless (defense in depth).
|
||||
|
||||
## 10. Config & ops
|
||||
|
||||
- Requires a reachable ComfyUI with an image model + the saved API-format workflow JSON checked in at
|
||||
`comfyui/coloring-book.workflow_api.json` (with documented node ids for prompt + seed).
|
||||
- New deps (all optional/pure-JS preferred): `potrace` (vectorize), `jimp` or `sharp` (image cleanup).
|
||||
- Document GPU expectations, timeout, and that Create is **off by default** (blank `COMFYUI_URL`), so
|
||||
the core app and existing deploys are unaffected.
|
||||
|
||||
## 11. Privacy & principles (unchanged)
|
||||
|
||||
Self-hosted ComfyUI means the child's subject text and the image stay on the operator's own
|
||||
infrastructure — no third-party calls, consistent with "ABSOLUTELY NO DATA COLLECTED." Generated assets
|
||||
live in the operator's SQLite. The feature is fully optional and disabled unless configured.
|
||||
|
||||
## 12. Phased rollout
|
||||
|
||||
- **Phase 1 — Plumbing:** Creator role + `requireCreator` + admin can assign; env config; `created_lessons`
|
||||
table; ComfyUI client (queue → poll → fetch) behind a feature flag.
|
||||
- **Phase 2 — Lesson pipeline (Option C):** image cleanup + **potrace vectorization** → Trace (stroke
|
||||
reveal) + Color; subject **allowlist + admin review queue** + blocklist + sanitize; preview/approve;
|
||||
"My Creations" group (private to creator); badge + gallery via existing tables; rate limiting.
|
||||
- **Phase 3 — Curation:** admin **promote-to-global** (creation enters the shared curriculum with
|
||||
attribution; owner becomes system/global); review-queue UI polish.
|
||||
|
||||
## 13. Decisions (locked June 2026)
|
||||
|
||||
Settled with Dan; these drive the build:
|
||||
|
||||
1. **Phases:** a created lesson is **2-phase — Trace + Color** (one AI image isn't cleanly splittable
|
||||
into outline vs detail). Beginner creations may add the Light phase later, but the baseline is two.
|
||||
2. **Moderation:** **allowlist + admin review queue** (+ blocklist backstop + input sanitize). Curated
|
||||
safe subjects auto-pass; anything new is held for admin approval before it generates/goes live.
|
||||
3. **Line reveal:** **vectorize up front** — run the cleaned line art through **potrace** so the Trace
|
||||
phase animates **stroke-by-stroke** (via `LineDraw`) from day one, exactly like hand-made lessons.
|
||||
The raster is still used for the Color phase (`ColoringRunner` rasterizes anyway). So the pipeline is
|
||||
Option C (hybrid): **vectorized SVG for Trace + raster for Color**.
|
||||
4. **Visibility:** creations are **private to the creator by default, but admin-promotable to the global
|
||||
curriculum** (with attribution). So the data model must support a "promoted/global" flag and an
|
||||
owner of `null`/system once promoted.
|
||||
|
||||
Still open (smaller, decide at build time): **image storage** (data URLs in SQLite vs. a files dir for
|
||||
the larger line-art PNGs) and **which ComfyUI model/workflow** is the reference (affects line-art
|
||||
cleanliness, hence vectorization quality — pick a model that yields bold, closed outlines).
|
||||
|
||||
## Sources
|
||||
|
||||
- [ComfyUI API — Hosting a Workflow via API (9elements)](https://9elements.com/blog/hosting-a-comfyui-workflow-via-api/)
|
||||
- [ComfyUI API: The Complete Developer's Guide (Runflow)](https://www.runflow.io/blog/comfyui-api-developer-guide)
|
||||
- [ComfyUI API Endpoints — Complete Reference (Runflow)](https://www.runflow.io/blog/comfyui-api-endpoints)
|
||||
- [ComfyUI: Using the API, Part 1 (Medium)](https://medium.com/@yushantripleseven/comfyui-using-the-api-261293aa055a)
|
||||
- [potrace — npm (Node bitmap→SVG)](https://www.npmjs.com/package/potrace)
|
||||
- [imagetracerjs — npm (JS raster vectorizer)](https://github.com/jankovicsandras/imagetracerjs)
|
||||
- [Filtering Techniques: Blocklists and Allowlists for Safe AI Prompts (Learn Prompting)](https://learnprompting.org/docs/prompt_hacking/defensive_measures/filtering)
|
||||
- [Azure AI Content Safety — Filters and Best Practices](https://jannikreinhard.com/2026/02/18/what-azure-ai-content-safety-does-and-why-it-matters/)
|
||||
- [Safety Filtering in AI: How to Block Harmful Model Outputs](https://www.practical-devsecops.com/glossary/safety-filtering/)
|
||||
Reference in new issue
Block a user