Chapter Evaluator Skill

Evaluate educational chapters through dual lenses: the

Student Experience

(engagement, clarity, confidence) and the

Teacher Perspective

(pedagogy, objectives, assessment). Output structured analysis with ratings, gaps, and actionable improvements.

When to Use

Analyzing a chapter's overall quality before publication

Identifying why content "feels off" (too short, boring, disconnected)

Planning improvements to existing chapters

Comparing chapters against quality standards

User asks to "evaluate", "review", "analyze", or "assess" a chapter

Evaluation Process

Step 1: Gather Chapter Content

Read all lesson files in the chapter directory:

ls

-la

<

chapter-path

>

/*.md

|

grep

-v

summary

|

grep

-v

README

|

grep

-v

quiz

For each lesson file, extract:

YAML frontmatter (learning objectives, cognitive load, skills, layer)

Word count

Section structure (headings)

Try With AI prompts

Hands-on exercises

Code examples

Step 2: Student Perspective Analysis

Evaluate as a beginner encountering this content for the first time.

2.1 Engagement Score (1-10)

Score

Criteria

9-10

Compelling hook, real-world relevance clear, I want to keep reading

7-8

Interesting enough, some engaging moments, minor dry spots

5-6

Functional but forgettable, reads like documentation

3-4

Boring, walls of text, no compelling reason to continue

1-2

Would abandon after first section

Check for:

Opening hook (does first paragraph grab attention?)

Real-world scenarios (why does this matter to ME?)

Story/narrative flow vs disconnected facts

Visual breaks (diagrams, tables, code blocks)

Pacing variety (concept → hands-on → concept)

Comparative Value

(vs alternatives like VS Code/Copilot)

2.2 Length Assessment

Verdict

Criteria

Too Short

Missing examples, concepts unexplained, abrupt endings, "I don't understand"

Just Right

Each concept has sufficient depth, examples clarify, natural flow

Too Long

Repetitive explanations, over-elaborated points, could cut 30%+

Word count benchmarks:

Conceptual lesson: 1,000-1,400 words

Hands-on lesson: 1,200-1,600 words

Installation/setup: 800-1,200 words (focused)

Capstone: 1,400-1,800 words

2.3 Clarity Score (1-10)

Score

Criteria

9-10

Crystal clear, no re-reading needed, "aha" moments

7-8

Mostly clear, occasional re-read for complex parts

5-6

Understandable with effort, some confusing sections

3-4

Frequently confused, missing context, jargon unexplained

1-2

Cannot follow, assumes knowledge I don't have

Check for:

Jargon introduced before defined

Logical flow between paragraphs

Transitions between sections

Prerequisites assumed vs stated

Safety Checks:

No concatenated commands or risky copy-pastes

2.4 Hands-On Effectiveness (1-10)

Score

Criteria

9-10

Clear steps, achievable, builds confidence, "I did it!"

7-8

Mostly clear, minor ambiguity, successful completion likely

5-6

Workable but confusing steps, may need to troubleshoot

3-4

Missing steps, unclear what to do, likely to get stuck

1-2

Cannot complete without external help

Check for:

Step-by-step instructions (numbered, clear)

Expected output/results shown

Troubleshooting guidance

Connection to concepts just learned

2.5 Progression Clarity (1-10)

Score

Criteria

9-10

Clear path from start to mastery, each lesson builds on previous

7-8

Generally progressive, minor jumps between lessons

5-6

Some logical progression, noticeable gaps

3-4

Disconnected lessons, unclear how they relate

1-2

Random ordering, no clear learning path

Check for:

Opening connections ("In Lesson N-1, you learned X. Now...")

Running example threaded through chapter

Skills building on each other

Clear "what's next" at lesson end

2.6 Confidence Score (1-10)

Score

Criteria

9-10

"I can definitely do this now" - ready to apply independently

7-8

"I mostly understand and could figure out the rest"

5-6

"I kind of get it but would need help applying it"

3-4

"I'm confused about when/how to use this"

1-2

"I have no idea what I just read"

Check for:

Practice opportunities before moving on

Verification steps ("you should see X")

Real-world application examples

"Try it yourself" prompts

Step 3: Teacher Perspective Analysis

Evaluate as an instructional designer assessing pedagogical soundness.

3.1 Learning Objectives Quality (1-10)

Score

Criteria

9-10

SMART objectives, measurable, aligned to content and assessment

7-8

Clear objectives, mostly measurable, good alignment

5-6

Objectives present but vague or partially aligned

3-4

Weak objectives, not measurable, poor alignment

1-2

Missing or meaningless objectives

Check for:

Bloom's taxonomy verb alignment (Remember → Create)

Measurable criteria ("can explain", "can create", "can distinguish")

Assessment method specified

Objectives actually taught in lesson content

3.2 Cognitive Load Management (1-10)

Score

Criteria

9-10

Appropriate concepts for level, well-scaffolded, no overload

7-8

Generally appropriate, minor overload moments

5-6

Some cognitive overload, too many concepts at once

3-4

Significant overload, concepts piled without consolidation

1-2

Overwhelming, no chance of retention

Benchmarks by proficiency:

A1-A2: 3-5 new concepts per lesson

B1-B2: 5-7 new concepts per lesson

C1-C2: 7-10 new concepts per lesson

Check for:

New concepts counted in frontmatter

Concepts introduced one at a time

Practice before new concept introduced

Chunking of complex procedures

3.3 Scaffolding Quality (1-10)

Score

Criteria

9-10

Perfect progression, each concept builds on previous, no gaps

7-8

Good scaffolding, minor jumps that students can bridge

5-6

Some scaffolding gaps, requires prior knowledge not taught

3-4

Significant gaps, assumes knowledge not in prerequisites

1-2

No scaffolding, concepts appear randomly

Check for:

Prerequisites listed and actually prerequisite

Concepts introduced before used

Increasing complexity curve

Prior knowledge activated before new content

3.4 Pedagogical Layer Appropriateness (1-10)

Layer

Expected Characteristics

L1 (Foundation)

Manual-first, understand before automate, no AI shortcuts

L2 (Collaboration)

AI as Teacher/Student/Co-Worker, learning through interaction

L3 (Intelligence)

Pattern recognition, creating reusable intelligence (skills/subagents)

L4 (Orchestration)

Capstone, combining components, spec-driven development

Check for:

Layer declared in frontmatter

Content matches layer expectations

Layer progression through chapter (L1 → L2 → L3 → L4)

No premature automation (L3 content in early lessons)

3.5 Try With AI Effectiveness (1-10)

Score

Criteria

9-10

Prompts directly extend lesson, specific, build skills

7-8

Good prompts, mostly connected to content

5-6

Generic prompts, loosely connected

3-4

Copy-paste prompts, don't match lesson

1-2

Missing or irrelevant prompts

Check for:

2-3 prompts per lesson (not 1, not 5+)

Prompts reference lesson content specifically

Progressive difficulty across prompts

"What's you're learning" explanations present

3.6 Assessment/Verification Quality (1-10)

Score

Criteria

9-10

Clear verification at each step, students know if they succeeded

7-8

Good verification for most exercises

5-6

Some verification, students may be unsure of success

3-4

Weak verification, students can't tell if they're on track

1-2

No verification, students have no idea if they succeeded

Check for:

"Expected output" shown for commands

"You should see X" confirmations

Error states explained

End-of-lesson checkpoint

Dimension Criticality & Publication Gate

CRITICAL

Not all dimensions are equally important for publication. Use this gate to determine if content is ready.

Gate Dimensions (MUST BE 7+)

These dimensions BLOCK publication if below 7/10. Fix these first.

Dimension

Why Critical

Remediation

Clarity

If unclear, nothing works. Confused students abandon.

Use

technical-clarity

skill

Scaffolding

Poor progression breaks learning. Students can't build on prior knowledge.

Use

concept-scaffolding

skill

Layer Appropriateness

Wrong layer means students lack prerequisites or are under-challenged.

Redesign layer; check prerequisites

Important Dimensions (6+)

These should be strong but minor issues are fixable.

Dimension

Target

Remediation

Engagement

6+

Add more worked examples and interactive elements

Learning Objectives

6+

Use

learning-objectives

skill

Assessment/Verification

6+

Add verification steps; clarity checks

Cognitive Load

6+

Reduce concepts per lesson; add practice

Enhancement Dimensions (5+)

These are nice-to-have; publication doesn't require perfection here.

Progression Clarity (5+)

Hands-On Effectiveness (5+)

Confidence (5+)

Try With AI Effectiveness (5+)

Publication Decision Logic

Use this decision tree AFTER scoring all dimensions:

IF any gate dimension (Clarity, Scaffolding, Layer) < 7:

→ REVISE: Content not ready

→ Fix the failing dimension(s)

→ Re-evaluate

ELSE IF (Engagement < 6) AND (Hands-On < 6):

→ CONDITIONAL PASS: Functional but needs improvement

→ Content is usable; improvements recommended

→ Can publish with revision plan

ELSE IF any important dimension (Objectives, Assessment, Load) < 5:

→ CONDITIONAL PASS: Missing elements but learnable

→ Flag for revision; can publish

ELSE:

→ PASS ✅: Ready for publication

→ All gate dimensions 7+

→ Most important dimensions 6+

Example Decision

Chapter Evaluation Results:

Clarity: 8 ✅

Scaffolding: 7 ✅

Layer Appropriateness: 8 ✅

Engagement: 5 (below ideal)

Cognitive Load: 7 ✅

Learning Objectives: 6 ✅

Assessment: 7 ✅

Decision

PASS ✅ — All gate dimensions 7+. Engagement is low, but structure is solid. Recommend: Add more compelling examples in next revision.

Step 4: Gap Analysis

After scoring, identify specific missing elements:

Content Gaps

Missing examples (concept taught but not demonstrated)

Missing hands-on (theory without practice)

Missing "why" (what but not why it matters)

Missing troubleshooting (happy path only)

Missing transitions (lessons don't connect)

Structural Gaps

Missing opening hook

Missing running example continuity

Missing "What's Next" closure

Missing visual elements (all text, no diagrams/tables)

Missing code examples for technical content

Pedagogical Gaps

Objectives not assessed

Cognitive overload unaddressed

Layer mismatch (content doesn't match declared layer)

Prerequisites not actually prerequisite

Try With AI prompts disconnected from content

Step 5: Generate Improvement Recommendations

For each gap, provide:

Problem

What's missing or wrong

Impact

How it affects learning (high/medium/low)

Fix

Specific action to address

Effort

Estimated work (low: <30min, medium: 30-90min, high: >90min)
Priority: 1 (critical), 2 (important), 3 (nice-to-have) Output Format Generate analysis in this structure:

Chapter Evaluation: [Chapter Name]

Executive Summary [1 paragraph: Overall quality assessment, key strengths, critical issues, recommendation]

Student Analysis

|

|
|
Engagement
|
X/10
|
[One-line summary]
|
|
Length
|
[Short/Right/Long]
|
[One-line summary]
|
|
Clarity
|
X/10
|
[One-line summary]
|
|
Hands-On
|
X/10
|
[One-line summary]
|
|
Progression
|
X/10
|
[One-line summary]
|
|
Confidence
|
X/10
|
[One-line summary]
|
**
Overall Student Experience
**: X/10

Detailed Findings [Specific observations per dimension with examples from content]

Student Pain Points 1. [Specific issue from student perspective] 2. [Specific issue from student perspective] ...

Teacher Analysis

|

|
|
Learning Objectives
|
X/10
|
[One-line summary]
|
|
Cognitive Load
|
X/10
|
[One-line summary]
|
|
Scaffolding
|
X/10
|
[One-line summary]
|
|
Layer Appropriateness
|
X/10
|
[One-line summary]
|
|
Try With AI
|
X/10
|
[One-line summary]
|
|
Assessment
|
X/10
|
[One-line summary]
|
**
Overall Pedagogical Quality
**: X/10

Detailed Findings [Specific observations per dimension with examples from content]

Pedagogical Concerns 1. [Specific issue from teacher perspective] 2. [Specific issue from teacher perspective] ...

Gap Analysis

|

| | [Missing element] | L0X | High/Med/Low | ...

|

| | [Missing element] | L0X | High/Med/Low | ...

|

| | [Missing element] | L0X | High/Med/Low | ...

Improvement Recommendations

Priority 1 (Critical) |

|

| | 1 | [Issue] | [Action] | Low/Med/High | L0X | ...

Priority 2 (Important) |

|

| | 1 | [Issue] | [Action] | Low/Med/High | L0X | ...

Priority 3 (Nice-to-Have) |

|

| | 1 | [Issue] | [Action] | Low/Med/High | L0X | ...

Publication Decision

|

| | 🚧 BLOCK if <7 | Clarity | X/10 | ✅/❌ | | 🚧 BLOCK if <7 | Scaffolding | X/10 | ✅/❌ | | 🚧 BLOCK if <7 | Layer Appropriateness | X/10 | ✅/❌ |

Publication Verdict

Status

[PASS ✅ | CONDITIONAL | REVISE]
**
Recommendation
**: [Ready for publication | Fix gates first | Needs revision plan]

Next Steps

If PASS:

-

[ ] Ready for publication

-

[ ] Note: Optional improvements in Priority 3 section above

If CONDITIONAL:

-

[ ] Content is functional

-

[ ] Recommended: Address Priority 1 issues in next iteration

-

[ ] Can publish now; plan revision cycle

If REVISE:

-

[ ] STOP: Fix gate dimensions first

-

[

] [

Gate dimension 1

]

[Specific action]

[
] [
Gate dimension 2
]: [Specific action]

[ ] Use remediation skills: [skill-1, skill-2]
[ ] Re-evaluate after fixes

Summary Metrics | Metric | Value | |

|

| | Total Lessons | X | | Average Word Count | X | | Student Score | X/10 | | Teacher Score | X/10 | | Overall Score | X/10 | | Gate Pass? | Yes/No | | Critical Issues | X | | Estimated Fix Time | X hours | Quality Reference Compare evaluated chapters against high-quality reference lessons. The skill should automatically identify and read a reference lesson from Part 1 or Part 6 for comparison when available. Reference lesson patterns to look for: 01-agent-factory-paradigm/01-digital-fte-revolution.md 33-introduction-to-ai-agents/01-what-is-an-ai-agent.md Resources references/ See references/ for detailed rubrics: student-rubric.md - Detailed student perspective evaluation criteria teacher-rubric.md - Detailed teacher perspective evaluation criteria word-count-benchmarks.md - Word count guidelines by lesson type

chapter-evaluator

安装

|

|

|

|

|

|

|

|

|

|

|

|

|

|

|

|

|

|

|

|

|

|

|

|

|

[Specific action]

[Specific action]

[ ] Use remediation skills: [skill-1, skill-2]

|