Automated Quality Management

Project log

1. Experience22 Jul 2026

   Product Thinking for Claralens

   My first move is always the same: learn. I search, research, and read as much as I can about the requirements — what others in the industry are doing, what we actually need to build, and where we can step… more

   ◆ Automated Quality ManagementDesignAI Collaboration & JudgmentCross-Functional CommunicationData AnalyticsProduct ThinkingSystems ThinkingUser Research & EvidenceVisual & Interaction CraftClaude CodeFigma
2. Work log24 Jul 2026

   "Call" → "Interaction"

   Initial approach. The BRD's atomic unit was a "call." Every dashboard, list, and report was built around it.

   Where it broke. During stakeholder clarification the scope expanded: Claralens has to grade SMS, WhatsApp, and email too — not just voice. "Call" quietly excludes three of the four channels, and any IA built on it would need re-architecting the moment text channels arrived.

   Resolution. Renamed the atomic unit to Interaction (channel: voice | sms | whatsapp | email; actor: human | bot). Every surface became a lens over one interaction corpus. Evidence anchoring got two renderers behind one contract — timestamp for voice, message ID for text — so adding a channel never means adding an app.

   ◆ Automated Quality ManagementResearchData AnalyticsProduct ThinkingAI Collaboration & JudgmentCross-Functional CommunicationSystems ThinkingUser Research & EvidenceVisual & Interaction CraftClaude CodeFigma
3. Work log25 Jul 2026

   Permissions aren't a settings page.

   In a QA tool, who-can-see-what isn't a checkbox screen you bolt on at the end. It decides what every other screen even contains.

   An auditor overrides scores. A QA manager runs calibration. An exec reads aggregates and should never touch a raw override. A tenant admin configures templates but has no business seeing an individual agent's grade. A superadmin lives in a separate shell.

   So before wireframing, I mapped every role against every surface as a matrix — full access, read-only, or none. Not for security theater. Because an exec's "detail view" and an auditor's "detail view" are different screens wearing the same name: one can flip a checkpoint and see the trail; the other can't see the button.

   Drawing that grid early kills a whole class of bug — the read-only user shown a control that errors on click, the exec accidentally exposed to data they shouldn't have. You can't design the screen honestly until you know who's standing in front of it.

   The permission matrix is a design input, not a compliance afterthought. It reshapes everything it touches.

   ◆ Automated Quality ManagementDesignAI Collaboration & JudgmentCross-Functional CommunicationData AnalyticsProduct ThinkingSystems ThinkingUser Research & EvidenceVisual & Interaction CraftClaude CodeFigma
4. Work log27 Jul 2026

   One Detector Library (Checkpoint vs Moment)

   Initial approach. The existing platform had scattered concepts — checklists, highlights, entities, signals — plus 143 live detectors.

   Where it broke. Overlapping vocabulary meant overlapping screens. Nobody could say cleanly what was scored and what was merely observed.

   Resolution. Collapsed everything into one unified Detector Library. A detector assigned to a template renders as a scored Checkpoint; an unassigned detector firing in the wild renders as an observed Moment (unscored). Detectors are typed — rule (deterministic) vs instructional (AI) — versioned, and pinned to each evaluation. Retired: checklist, highlight, entity, signal. This unification later flipped the entire build order (see §6).

   Two more locks that came out of the same work:

   • All three detector states must be authored — pass, fail, and not applicable. "Not applicable" is a first-class state that changes the scoring denominator, not an afterthought.
   • Overrides happen only at the checkpoint leaf, recomputing upward with a delta preview before commit (e.g. parameter 25 → 50 · total 62 → 68). Parent-level overrides break explainability; showing the consequence before the click turns a reflex into a considered judgment.

   ◆ Automated Quality ManagementAIDesignResearchUser Research & EvidenceAI Collaboration & JudgmentCross-Functional CommunicationData AnalyticsProduct ThinkingSystems ThinkingVisual & Interaction CraftClaude CodeFigma
5. Work log28 Jul 2026

   The build order flipped under me

   I'd sequenced Claralens the obvious way: start with the Evaluation Detail View. It's the heart of the product — the screen where a grade gets read, disputed, overridden. Build the heart first.

   Then I wrote the object model down as a plain dependency list and argued with it. The heart fell apart.

   You can't render a detail view without a template to score against. You can't author a template without detectors to assign into it — because a Checkpoint is a detector assigned to a template. The dependency chain runs the exact opposite of intuition:

   Detector Library → Template Editor → Evaluation Detail View.

   The most important screen in the product literally cannot exist until its foundations do. The "heart" ships last, not first.

   I only caught this because I forced the model into a table before touching wireframes. In prose it was invisible — everything sounded shippable in any order. In a dependency list it was obvious in ten seconds.

   Write the object model as a list and stress-test the order before you draw. It's a table you can rewrite in a minute or a set of wireframes you rebuild for a week.

   ◆ Automated Quality ManagementDesignResearchSystems ThinkingAI Collaboration & JudgmentCross-Functional CommunicationData AnalyticsProduct ThinkingUser Research & EvidenceVisual & Interaction CraftClaude CodeFigma
6. LearningTIL28 Jul 2026

   How to design the IA for uploading data?

   The brief listed a dozen CCaaS integrations — Genesys, Five9, the usual roster. The implied task: build a data-upload flow. I didn't. Not because it wasn't needed, but because "how data gets in" isn't one problem. It's three completely different… more

   ◆ Automated Quality ManagementResearchUser Research & EvidenceAI Collaboration & JudgmentCross-Functional CommunicationData AnalyticsProduct ThinkingSystems ThinkingVisual & Interaction CraftClaude CodeFigma
7. Work log29 Jul 2026

   Correct Judgement Heuristics

   When I made "not applicable" a real checkpoint state — alongside pass and fail — it quietly broke the scoring math.

   Say a parameter has five checkpoints and one genuinely doesn't apply to this call. Score it out of 5 and you've penalized the agent for something that was never in scope. The honest denominator is 4, not 5. "Not applicable" isn't a silent zero — it's an exclusion, and exclusions change the denominator.

   So the logic changed: N/A checkpoints drop out of the denominator entirely, rather than counting as fails. The parameter scores 1 of 4. The header reads "14 of 17 scored," with the three excluded checkpoints shown explicitly — not hidden to make the number look cleaner.

   That visibility is the entire point. An agent disputing the grade can walk the exact chain: this checkpoint was N/A, here's why, that's why the parameter is out of 4, that's why your total lands where it does.

   The correction wasn't hard to make. It was hard to notice — the math looks correct right up until you realize a third state was silently doing a fail's job in the denominator.

   ◆ Automated Quality ManagementDesignResearchData AnalyticsAI Collaboration & JudgmentCross-Functional CommunicationProduct ThinkingSystems ThinkingUser Research & EvidenceVisual & Interaction CraftClaude CodeFigma
8. Work log18 Aug 2026

   "Not enough data" is a state, not a blank

   Initial approach. Building the Claralens analytics layer, I defaulted to the obvious: every insight card shows a score. A number in every cell felt like completeness.

   Where it broke. On small or sparse interaction sets, a raw score misleads. Five interactions scoring 100% isn't a good agent — it's five interactions. Ranking on numbers like that manufactures false confidence, and in BFSI that confidence gets acted on.

   Resolution. I made emptiness explicit. "Not enough data" is now a first-class UI state — never a blank cell, never a zero standing in for absence. Template Health became a prerequisite gate: if the instrument isn't sound, nothing downstream is trusted. Ranking uses the Wilson lower bound with a minimum-N threshold, so a position has to be earned. Every number renders alongside its coverage and denominator, with NA checkpoints excluded from the denominator rather than counted as failures.

   The principle underneath: the dashboard's job isn't to look full. It's to say when it knows something and when it doesn't. A confident wrong number costs more than an honest gap.

   ◆ Automated Quality ManagementDesignUser Research & EvidenceAI Collaboration & JudgmentCross-Functional CommunicationData AnalyticsProduct ThinkingSystems ThinkingVisual & Interaction CraftClaude CodeFigma
9. Work log22 Aug 2026

   I stopped designing the dashboard and started designing the decision

   Initial approach. The brief was the one every designer knows: research the best quality dashboards, study what the category ships, design the strongest insights page I could. So I did — benchmarked, collected patterns, had a good-looking page forming.

   Where it broke. Midway I stopped trusting the dashboard heuristics. A dashboard isn't a gallery of the best-looking charts; it's the front end of a decision. The question that mattered wasn't "what should this page show" — it was "how does a user analyse this, share it, and act on it, and how does my presentation decide which of those they can even do." Framed that way, half of my well-researched page was decoration.

   Resolution. I anchored the insights layer to an analytics lifecycle — Ask → Prepare → Process → Analyze → Share → Act — and used it as a gate, not a diagram. The rule: an insight ships only if it names a real question (Ask) and a real decision it drives (Act). If a metric changes no one's behaviour, it isn't an insight — it's decoration, and it goes to drill-down or nowhere. That one test killed most of what research had produced. Prepare and Process became the buildability gate: data has to exist with its denominator or it doesn't ship. Share stopped being a phase-three nicety and became a v1 requirement — an insight you can't export never reaches the person who acts on it.

   What I'm proudest of isn't a cleaner page, it's a mechanism. Every insight now carries the same spine — what question, whose decision, what data, what action. Most QM dashboards drown you in scoreable numbers. This one refuses to show a number that doesn't earn a decision. Fewer metrics, each load-bearing.

   ◆ Automated Quality ManagementDesignResearchData AnalyticsSystems ThinkingUser Research & EvidenceVisual & Interaction CraftAI Collaboration & JudgmentCross-Functional CommunicationProduct ThinkingClaude CodeFigma