Skill Development Guide for creating and refining skills in the skills/ directory of this repository. Skills are packaged instructions that teach AI agents how to work with Phoenix features. Directory Structure skills/ {skill-name}/ # kebab-case, prefixed with "phoenix-" for Phoenix features SKILL.md # Required: skill definition and index README.md # Optional: human-oriented overview rules/ # Optional: detailed rule files {prefix}-{topic}.md {prefix}-{topic}-{lang}.md Existing Skills Skill Purpose Has Rules phoenix-tracing OpenInference instrumentation and span types Yes (31 files) phoenix-evals Evaluator development and validation Yes (33 files) phoenix-cli CLI debugging and analysis No (single SKILL.md) Creating a New Skill 1. Plan the Scope Determine whether the skill needs rule files: Single SKILL.md (like phoenix-cli ): Self-contained topics, command references, single-workflow tools SKILL.md + rules/ (like phoenix-tracing , phoenix-evals ): Multi-faceted topics with language-specific guides, multiple workflows, or extensive reference material 2. Create SKILL.md Every skill requires a SKILL.md with YAML frontmatter:

name : { skill - name } description : { What it does. When to use it. Include trigger phrases. } license : Apache - 2.0 metadata : author : oss@arize.com version : "1.0.0" languages : Python , TypeScript

omit if language-agnostic

Frontmatter rules:

name

kebab-case, match directory name

description

Third person, specific, includes trigger terms

license

Always

Apache-2.0

metadata.author

Use

oss@arize.com

for Phoenix skills

metadata.version

Semver string (e.g.,

"1.0.0"

)

metadata.languages

Only include if skill has language-specific content

metadata.internal

Set to true for skills not intended for public listing (e.g., skills in .agents/skills/ ). Omit for public skills in skills/ . 3. Structure the SKILL.md Body SKILL.md serves as the index and entry point . The agent reads this first and navigates to rule files as needed. Required sections: Section Purpose Title (

Name

) Skill name with brief description Quick Reference Table mapping tasks to rule files Rule Categories Table of prefix patterns Optional sections (include as relevant): Section Purpose When to Apply Trigger scenarios Workflows Step-by-step paths through rules Key Principles Core decision-making guidance Common Attributes Reference tables How to Use Navigation guidance References External docs and API links 4. Organize Rule Files Naming Conventions Pattern: {prefix}-{topic}[-{language}].md Prefix categories (reuse these across skills): Prefix Purpose Example setup- Installation, configuration setup-python.md fundamentals- Core concepts, reference fundamentals-overview.md instrumentation- Auto/manual setup instrumentation-auto-python.md span- Span type specifications span-llm.md evaluators- Evaluator types, patterns evaluators-code-python.md experiments- Datasets, running experiments experiments-running-typescript.md production- Deployment, monitoring production-python.md annotations- Feedback, scoring annotations-overview.md validation-* Calibration, testing validation-calibration-python.md Language suffixes: -python.md — Python-specific content -typescript.md — TypeScript-specific content No suffix — Language-agnostic or overview (e.g., span-llm.md , evaluators-overview.md ) Overview files: Use -overview.md suffix for conceptual introductions (e.g., fundamentals-overview.md , production-overview.md ). Flat Structure All rule files go directly in rules/ — no subdirectories. Use prefixes for organization, not folders. rules/ setup-python.md # Good: flat with prefix setup-typescript.md span-llm.md # Good: no language suffix (language-agnostic) evaluators-code-python.md # Good: prefix-topic-language 5. Write Rule Files Standard Structure

Title Brief description of what this rule covers.

Metadata (optional) | Field | Value | |

|

| | Priority | 1 (Critical) | | Setup Time | 5 minutes |

Quick Start / Basic Pattern Minimal working example.

Detailed Sections Expanded content with examples.

See Also

related-rule.md — Brief description - External docs Code Examples Always use fenced blocks with language tags: python , typescript , bash , json Show working, copy-pasteable examples Include both minimal and production-ready patterns when relevant Cross-References Reference other rule files by filename: setup-python.md , span-llm.md Reference external docs with full URLs Keep references one level deep (SKILL.md → rule file, not rule → rule → rule) Refining Existing Skills Adding a Rule File Choose the correct prefix from the table above (or establish a new one) Follow the naming convention: {prefix}-{topic}[-{language}].md Add an entry in SKILL.md under Quick Reference and Rule Categories Cross-reference related rules in the new file's "See Also" section Updating SKILL.md When adding content to SKILL.md, keep it as an index — move detailed content to rule files. SKILL.md should stay under 500 lines. Improving Consistency Common issues to fix when refining: Issue Fix Missing metadata table Add Priority, Setup Time if applicable Inconsistent headings Standardize to

Quick Start

,

See Also

Inline code dumps Extract to rule files, link from SKILL.md Missing cross-references Add

See Also

with related rules Vague descriptions Make frontmatter description specific with trigger terms Quality Checklist SKILL.md Frontmatter has all required fields ( name , description , license , metadata ) metadata.internal: true is set for skills in .agents/skills/ ; omitted for skills in skills/ Description is third person, specific, includes trigger terms Quick Reference table maps tasks to rule files Rule Categories table lists all prefixes used Under 500 lines No detailed content that belongs in rule files Rule Files Follow {prefix}-{topic}[-{language}].md naming Have a clear

Title

and brief description Include working code examples with language tags Cross-reference related rules in

See Also

Use consistent heading structure Overall Skill Flat rules/ directory (no subdirectories) Consistent terminology throughout all files Language-specific content split into -python.md / -typescript.md files Language-agnostic content has no language suffix No time-sensitive information (no dates, version caveats) Anti-Patterns Putting too much in SKILL.md. SKILL.md is an index. If a section exceeds ~30 lines of content, extract it to a rule file. Deep reference chains. Rule files should not require reading other rule files to be useful. Each rule should be self-contained enough to act on independently. Generic rule names. Use evaluators-code-python.md not code-eval.md . Prefixes enable discovery via ls rules/{prefix}-* . Mixing languages in one file. Split into -python.md and -typescript.md unless the content is truly language-agnostic. Forgetting SKILL.md updates. Every new rule file must be reflected in the SKILL.md Quick Reference and Rule Categories tables. Workflow Summary 1. Plan scope → single SKILL.md or SKILL.md + rules/? 2. Create directory: skills/{skill-name}/ 3. Write SKILL.md with frontmatter and index 4. Create rules/ directory (if needed) 5. Write rule files following naming conventions 6. Update SKILL.md index to reference all rules 7. Run quality checklist