0 Fork 0

Files

T

Dan f79ce000e4 Add design doc for Create feature and Creator role

2026-06-29 14:11:09 +00:00

17 KiB

Raw Blame History

DrawIt — Design: "Create" (AI-generated custom lessons) + the Creator role

Designed June 2026. Purpose: let a privileged user type a subject (e.g. "Zebra"), have a self-hosted ComfyUI server generate a coloring-book line-art image, and turn that image into a real DrawIt lesson with the usual design elements. This is a design/research document — nothing is built yet. It assumes the current curriculum/runner architecture documented in RESEARCH-beginner-level.md and RESEARCH-early-beginner-expansion.md.

1. Goal & summary

A new "Create" option (visible only to a new Creator role) appears on the Early Beginner and Beginner levels. The Creator types a subject; the app sends a fixed prompt to a configurable ComfyUI endpoint; ComfyUI returns black-and-white line art; DrawIt processes that image into a playable lesson (trace + color phases, badge, progress, gallery — the same machinery every other lesson uses).

The fixed prompt (subject interpolated):

A coloring book page of a [subject], clean black-and-white line art, bold crisp outlines, simple composition, large open areas to color, white background, no shading, no grayscale, no color, no gradients, no heavy background detail, printable page.

Two hard problems drive the design: (a) an AI returns one raster image, but our lessons are multi-phase, SVG-step constructions; and (b) this is a children's app accepting free-text that feeds an image generator, so safety/moderation is mandatory, not optional.

2. The Creator role

Today Role = "admin" | "learner" (src/lib/types.ts); the DB users.role is already a free-text column defaulting to learner, and the first registered account is forced to admin. Adding a role is small and backward-compatible:

Type: Role = "admin" | "learner" | "creator". A Creator has all Learner permissions plus the Create ability. Admins implicitly have it too.
No schema change — role is already TEXT. Existing rows stay learner/admin.
Assignment: only an admin grants it, via the existing admin users API (src/app/api/admin/users/[id]/route.ts) — add "creator" to the allowed set (today it coerces to admin/learner). Signup is unchanged (first = admin, everyone else = learner).
Server guard: add requireCreator() in src/lib/session.ts (mirrors requireAdmin()): returns the user iff role === "creator" || role === "admin". A tiny canCreate(user) helper feeds the UI. Every Create endpoint must call this server-side — never trust the client.

3. End-to-end user flow

Creator opens Early Beginner or Beginner → taps ✨ Create your own.
Types a subject ("Zebra"). Client does light validation; server does the real moderation (§5).
Server creates a created_lessons row (status: 'pending'), interpolates the subject into the fixed prompt, and queues the ComfyUI workflow (status: 'generating').
Client polls status (or subscribes). ComfyUI renders (seconds to ~1 min on a GPU box).
Server fetches the image, post-processes it (§6), derives the lesson assets, sets status: 'ready' (or 'needs_review' if moderation requires admin approval).
The lesson appears under a "My Creations" group on that level and plays through the normal runners (trace → color), earning a badge and saving to the gallery like any lesson.

4. ComfyUI integration

Why ComfyUI / self-hosted: it keeps DrawIt's privacy promise — the subject text and image never leave a server the operator runs. No third-party AI cloud. The endpoint is configurable so each self-hoster points at their own GPU box (or none, disabling Create).

Config (env, mirroring the existing SMTP pattern in .env.example):

COMFYUI_URL=            # e.g. http://127.0.0.1:8188 — blank disables the Create feature
COMFYUI_API_KEY=        # optional, if the box is behind an auth proxy
COMFYUI_WORKFLOW=./comfyui/coloring-book.workflow_api.json   # API-format workflow template
COMFYUI_TIMEOUT_MS=120000
COMFYUI_PROMPT_NODE=6   # id of the positive CLIP-text node to inject the prompt into

API calls (ComfyUI's HTTP API):

POST {COMFYUI_URL}/prompt with { prompt: <workflow_api_json>, client_id }. We load the saved API-format workflow (exported from ComfyUI's "Save (API format)"), and set the positive prompt node's inputs.text to our interpolated prompt (and randomize the seed node). Returns prompt_id.
Poll GET {COMFYUI_URL}/history/{prompt_id} until the entry exists; read outputs[node].images[] → { filename, subfolder, type }. (Optionally subscribe to GET /ws?clientId= for live progress to show a friendly "drawing your zebra…" bar.)
GET {COMFYUI_URL}/view?filename=…&subfolder=…&type=output → the PNG bytes.

Async model: generation is slow, so treat it as a job. created_lessons.status is the state machine: pending → generating → (needs_review) → ready | failed | blocked. The POST route kicks off generation server-side and returns immediately; the client polls GET /api/create/:id. On a single self-hosted node an in-process async task is fine; document that horizontal scaling would want a real queue. Always set a timeout and a retry-once policy; surface failed with a kid-friendly message.

5. Safety & moderation (mandatory — this is a kids' app)

Free text → image generation for children is the riskiest part. Layered defenses, input + output:

Input (the subject string):

Sanitize: trim; cap length (~30 chars); collapse whitespace; allow only letters/spaces/hyphens; reject newlines and any prompt-control tokens. Because we only interpolate [subject] into a fixed prompt, prompt-injection surface is small — but still strip anything that could break out of the phrase.
Allowlist-first (recommended for kids): maintain a curated allowlist of safe, drawable subjects (animals, objects, food, vehicles, nature, shapes…). If the subject is on the allowlist → proceed automatically. This is the safest posture and the literature's recommendation for child contexts.
Blocklist backstop: a denylist of profanity, violence, weapons, drugs, sexual, hateful, and self-harm terms (and obfuscations) rejects anything obviously unsafe that slips past.
Anything else → admin review queue (status: 'needs_review'): not on the allowlist and not on the blocklist → a human approves before it generates or goes live. Keeps the door open to new subjects without exposing kids to raw generation.
Rate-limit per user (e.g. N creations/day) to bound abuse and GPU cost.

Output (the generated image): even safe prompts can misfire. Before a lesson is playable, require a preview/approve step — minimally the creating user sees it and confirms; for stricter operators, an admin approval queue. (Optional later: an automated NSFW image classifier as a pre-filter.)

Logging vs. privacy: store only what's needed to run/curate the feature (subject, status, moderation decision). Consistent with "no analytics, data stays local."

6. From image → level (the core pipeline)

What a lesson needs today: a Lesson with phases. Trace phases (construct/outline/detail) use per-step lines: string[] of SVG elements that LineDraw reveals one at a time; color/light phases use baseSvg which ColoringRunner rasterizes for the fill boundary and overlay. Badge, sublevel, subjectKey, and a group complete the level.

The gap: ComfyUI returns one raster PNG. It is not pre-split into outline-vs-detail phases, and it is pixels, not SVG strokes. So a Created lesson can't perfectly mirror a hand-authored 3–4-phase subject. Chosen approach (per §13): Option C — vectorize the line art with potrace for a stroke-by-stroke Trace phase, and use the raster for the Color phase (2 phases total). The three options below are kept for context; the build targets C.

Option A — Raster-native, 2-phase (recommended MVP). Treat the cleaned PNG as the lesson art.
- Trace phase: show the line art faintly as a guide to trace over (reuse TraceRunner's base layer, fed a raster instead of SVG). Optional "watch it appear" = a left-to-right wipe reveal of the bitmap rather than true per-stroke animation.
- Color phase: ColoringRunner already rasterizes its template to a boundary + overlay, so it can take the PNG directly — flood-fill and overlay work as-is. This phase is essentially free.
- For Beginner-created lessons, optionally add the existing Light phase (sun overlay) on top.
- Smallest change: add a templateUrl? (raster) alongside baseSvg to TraceRunner/ColoringRunner and the lesson-page meta; when present, use the bitmap path.
Option B — Vectorize for true line-by-line reveal. Run the PNG through potrace (pure-JS npm, no native build — important given our better-sqlite3 build friction) to get SVG paths, then split the SVG into multiple lines (by M subpaths) so LineDraw animates it stroke-by-stroke like the hand-made lessons. Quality varies with image cleanliness; treat as an enhancement over A.
Option C — Hybrid (target end state). Vectorize for the trace reveal (B) and keep the raster for coloring (A). Optionally heuristically split paths into a simpler "outline" (large/long paths) vs "detail" (small/short paths) pass to approximate the two trace phases — best-effort, not perfect.

Image post-processing (server-side, before storing) with a pure-JS lib like jimp (avoids native deps) or sharp (faster, native): grayscale → threshold to crisp black/white → auto-trim margins → fit to the lessons' aspect (the runners use a 4:3 / 400×300 viewBox and an 880×660 color canvas) → flatten on white → downscale. Store the cleaned PNG; if vectorizing, store the derived SVG too.

Badge & metadata: Created lessons get a generic medal — a default emoji (✏️ or 🎨, or a creator-picked one) plus a name like "Zebra Artist". Reuse the existing emoji-medal BadgeArt and award-PDF machinery untouched.

7. Data model

No destructive changes; one new table (plus the Role union value):

CREATE TABLE created_lessons (
  id           INTEGER PRIMARY KEY AUTOINCREMENT,
  user_id      INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
  level        TEXT NOT NULL,                 -- 'early-beginner' | 'beginner'
  subject      TEXT NOT NULL,                 -- sanitized, e.g. 'Zebra'
  slug         TEXT NOT NULL UNIQUE,          -- e.g. 'created-42-zebra'
  status       TEXT NOT NULL DEFAULT 'pending', -- pending|generating|needs_review|ready|failed|blocked
  image        TEXT,                          -- cleaned line-art (data URL, like drawings/avatars)
  template_svg TEXT,                          -- optional vectorized paths (Option B/C)
  prompt       TEXT NOT NULL,                 -- full prompt sent (audit)
  moderation   TEXT,                          -- 'allowlist' | 'review' | 'blocked:<reason>'
  emoji        TEXT,                           -- badge emoji
  created_at   TEXT NOT NULL DEFAULT (datetime('now'))
);

Integration with existing systems (reuse, don't fork):

Sublevel namespacing: created lessons keep their level (so the unlock gate / grouping still apply) and use a high synthetic sublevel = 900000 + id, guaranteeing no collision with static lessons. Progress, completions, drawings, and badges then work through the existing tables keyed by (user_id, level, sublevel) with zero changes.
Badge key: created-<id>; badgeDefs() gains the user's ready created lessons at request time.
Not in static LESSONS. Add resolver helpers that the lesson page and learn page call: getCreatedLesson(slug) builds a Lesson-shaped object on the fly from the row (so the runners need no special-casing), and listCreatedLessons(userId, level) feeds the "My Creations" group.
Images as data URLs in SQLite matches the existing privacy-consistent storage for drawings and avatars (note: line-art PNGs are small once thresholded; watch row size, consider a files dir if big).
Backups: the new table rides along with the existing Litestream/SQLite backup automatically.

8. API routes (Next.js App Router)

POST /api/create — requireCreator; body { level, subject }. Validate + moderate (§5). On pass: insert row, interpolate prompt, kick off ComfyUI job, return { id, slug, status }. On block: 422 with a gentle message.
GET /api/create/:id — owner (or admin); returns { status, slug, image? } for polling.
GET /api/create/:id/image — serves the stored line art (or inline data URL).
POST /api/admin/create/:id — admin approve/block for the review queue.
All Create routes are no-ops/404 when COMFYUI_URL is unset (feature disabled cleanly).

9. UI

Entry point: a "✨ Create your own" card at the top of the Early Beginner and Beginner sections of the Learn page, rendered only when canCreate(user). (Also a nav entry, optional.)
Create view: subject input + the fixed-prompt explainer ("we'll draw a coloring page of your subject"), a live status/progress state, then a preview with "Start the lesson" / "Try again".
My Creations group: a getGroupedSubjects-style group on the level listing the user's ready creations, each opening into the normal trace/color runner.
Gating: non-creators never see Create (UI), and the API rejects them regardless (defense in depth).

10. Config & ops

Requires a reachable ComfyUI with an image model + the saved API-format workflow JSON checked in at comfyui/coloring-book.workflow_api.json (with documented node ids for prompt + seed).
New deps (all optional/pure-JS preferred): potrace (vectorize), jimp or sharp (image cleanup).
Document GPU expectations, timeout, and that Create is off by default (blank COMFYUI_URL), so the core app and existing deploys are unaffected.

11. Privacy & principles (unchanged)

Self-hosted ComfyUI means the child's subject text and the image stay on the operator's own infrastructure — no third-party calls, consistent with "ABSOLUTELY NO DATA COLLECTED." Generated assets live in the operator's SQLite. The feature is fully optional and disabled unless configured.

12. Phased rollout

Phase 1 — Plumbing: Creator role + requireCreator + admin can assign; env config; created_lessons table; ComfyUI client (queue → poll → fetch) behind a feature flag.
Phase 2 — Lesson pipeline (Option C): image cleanup + potrace vectorization → Trace (stroke reveal) + Color; subject allowlist + admin review queue + blocklist + sanitize; preview/approve; "My Creations" group (private to creator); badge + gallery via existing tables; rate limiting.
Phase 3 — Curation: admin promote-to-global (creation enters the shared curriculum with attribution; owner becomes system/global); review-queue UI polish.

13. Decisions (locked June 2026)

Settled with Dan; these drive the build:

Phases: a created lesson is 2-phase — Trace + Color (one AI image isn't cleanly splittable into outline vs detail). Beginner creations may add the Light phase later, but the baseline is two.
Moderation: allowlist + admin review queue (+ blocklist backstop + input sanitize). Curated safe subjects auto-pass; anything new is held for admin approval before it generates/goes live.
Line reveal: vectorize up front — run the cleaned line art through potrace so the Trace phase animates stroke-by-stroke (via LineDraw) from day one, exactly like hand-made lessons. The raster is still used for the Color phase (ColoringRunner rasterizes anyway). So the pipeline is Option C (hybrid): vectorized SVG for Trace + raster for Color.
Visibility: creations are private to the creator by default, but admin-promotable to the global curriculum (with attribution). So the data model must support a "promoted/global" flag and an owner of null/system once promoted.

Still open (smaller, decide at build time): image storage (data URLs in SQLite vs. a files dir for the larger line-art PNGs) and which ComfyUI model/workflow is the reference (affects line-art cleanliness, hence vectorization quality — pick a model that yields bold, closed outlines).

17 KiB Raw Blame History Unescape Escape