contract-tester

仓库: vladm3105/aidoc-flow-framework
安装量: 49
排名: #15043

安装

npx skills add https://github.com/vladm3105/aidoc-flow-framework --skill contract-tester

contract-tester

Description: API contract validation, breaking change detection, and consumer-driven contract testing

Category: API Quality Assurance

Complexity: Medium-High (schema validation + contract compatibility)

Purpose

Validate API implementations against CTR (Contract) documents, detect breaking changes, and ensure provider-consumer compatibility. Prevents API contract violations before deployment.

Capabilities 1. CTR YAML Schema Validation Validate YAML structure against schema Check required fields completeness Verify data type correctness Validate enum values Check format specifications (email, URL, UUID) 2. OpenAPI/AsyncAPI Compliance Convert CTR to OpenAPI 3.0/3.1 spec Validate against OpenAPI schema AsyncAPI validation for event-driven APIs JSON Schema validation GraphQL schema validation 3. Contract Versioning Semantic versioning validation (MAJOR.MINOR.PATCH) Version compatibility checking Deprecation warnings Migration path validation 4. Breaking Change Detection Removed endpoints/operations Changed request/response schemas Removed required fields Changed field types Stricter validation rules Changed authentication requirements 5. Mock Server Generation Generate mock API server from CTR Realistic response generation Configurable response delays Error scenario simulation State management for stateful APIs 6. Consumer-Driven Contract Testing Pact contract testing Provider verification Consumer contract generation Bidirectional contract testing Contract evolution tracking 7. Contract Compatibility Matrix Provider version compatibility Consumer version support Deprecation timeline tracking Migration requirements 8. API Documentation Generation Generate API documentation from CTR Interactive API explorer Code examples in multiple languages Authentication guides Contract Testing Workflow graph TD A[CTR Document] --> B[Parse YAML] B --> C{Valid YAML?} C -->|No| D[Report Parse Error] C -->|Yes| E[Schema Validation]

E --> F{Schema Valid?}
F -->|No| G[Report Schema Errors]
F -->|Yes| H[Version Check]

H --> I{Version Valid?}
I -->|No| J[Report Version Error]
I -->|Yes| K[Breaking Change Detection]

K --> L{Previous Version Exists?}
L -->|No| M[First Version]
L -->|Yes| N[Compare Versions]

N --> O{Breaking Changes?}
O -->|Yes| P[Report Breaking Changes]
O -->|No| Q[Compatible Change]

M --> R[Generate Mock Server]
Q --> R
P --> R

R --> S[Run Contract Tests]
S --> T{Tests Pass?}
T -->|No| U[Report Test Failures]
T -->|Yes| V[Generate Documentation]

V --> W[Validation Complete]

D --> W
G --> W
J --> W
U --> W

Usage Instructions Validate CTR Document contract-tester validate --file ctrs/CTR-USER-V1.yaml

Output:

=== Contract Validation Report: CTR-USER-V1 ===

Status: PASS

Schema Validation: ✓ - Service name: user-service - Version: 1.0.0 - Protocol: REST - Endpoints: 5 - Models: 3

OpenAPI Compliance: ✓ - Valid OpenAPI 3.1 specification - All schemas properly defined - Authentication specified (Bearer JWT)

Version Check: ✓ - Semantic versioning: 1.0.0 - No previous version (initial release)

Breaking Changes: N/A (initial release)

Warnings (1): - Endpoint DELETE /users/{id} lacks rate limiting specification → Recommendation: Add rate limiting to prevent abuse

Summary: Contract is valid and ready for implementation

Detect Breaking Changes contract-tester diff \ --previous ctrs/CTR-USER-V1.yaml \ --current ctrs/CTR-USER-V2.yaml

Output:

=== Breaking Change Analysis: V1 → V2 ===

BREAKING CHANGES (3):

  1. [CRITICAL] Removed endpoint
  2. DELETE /users/{id}
  3. Impact: Consumers using this endpoint will fail

  4. Migration: Use DELETE /users/{id}/deactivate instead

  5. [CRITICAL] Changed response type

  6. Endpoint: GET /users/{id}
  7. Field: created_at
  8. V1: string (format: date-time)
  9. V2: integer (unix timestamp)
  10. Impact: Date parsing will fail in consumers

  11. Migration: Update consumers to handle unix timestamp

  12. [MAJOR] Added required field

  13. Endpoint: POST /users
  14. Field: email_verified (boolean, required)
  15. Impact: Existing POST requests missing this field will fail
  16. Migration: Update all POST requests to include email_verified

NON-BREAKING CHANGES (2):

  1. [MINOR] Added optional field
  2. Endpoint: GET /users/{id}
  3. Field: last_login (string, optional)

  4. Impact: None (backward compatible)

  5. [PATCH] Fixed typo in description

  6. Model: User
  7. Field: username
  8. Impact: Documentation only

RECOMMENDATION: Version should be 2.0.0 (MAJOR bump due to breaking changes) Current version: 1.1.0 ❌ Suggested version: 2.0.0 ✓

Generate Mock Server contract-tester mock \ --file ctrs/CTR-USER-V1.yaml \ --port 8080 \ --response-delay 100ms

Mock server features:

Realistic data generation based on schemas Configurable response delays Error scenario injection Request validation Response logging

Test against mock server

curl http://localhost:8080/api/users/123

Response

{ "user_id": "123e4567-e89b-12d3-a456-426614174000", "username": "john_doe", "email": "john.doe@example.com", "created_at": "2025-01-15T10:30:00Z", "status": "active" }

Run Consumer Contract Tests contract-tester test-consumer \ --contract ctrs/CTR-USER-V1.yaml \ --consumer user-frontend \ --provider user-service \ --output reports/contract-tests/

Generated Pact test:

import pytest from pact import Consumer, Provider, Like, EachLike, Term

@pytest.fixture def pact(): return Consumer('user-frontend').has_pact_with( Provider('user-service'), pact_dir='./pacts' )

def test_get_user_contract(pact): """Verify GET /users/{id} contract""" expected_response = { 'user_id': Like('123e4567-e89b-12d3-a456-426614174000'), 'username': Like('john_doe'), 'email': Term(r'.+@.+..+', 'john.doe@example.com'), 'created_at': Like('2025-01-15T10:30:00Z'), 'status': Term(r'active|inactive|suspended', 'active') }

(pact
 .given('user with id 123 exists')
 .upon_receiving('a request for user 123')
 .with_request('GET', '/api/users/123')
 .will_respond_with(200, body=expected_response))

with pact:
    result = user_api.get_user('123')
    assert result.status_code == 200
    assert 'user_id' in result.json()

Verify Provider Implementation contract-tester test-provider \ --contract ctrs/CTR-USER-V1.yaml \ --provider-url http://localhost:8000 \ --output reports/provider-verification/

CTR Document Structure Complete CTR Example

ctrs/CTR-USER-V1.yaml

contract_id: CTR-USER-V1 version: 1.0.0 service_name: user-service protocol: REST base_path: /api

metadata: description: User management API contract owner: backend-team consumers: - user-frontend - admin-dashboard traceability: requirements: - REQ-USER-001 - REQ-USER-002 specifications: - SPEC-API-V1

authentication: type: bearer scheme: JWT token_location: header header_name: Authorization

endpoints: - operation_id: get_user method: GET path: /users/{id} summary: Retrieve user by ID parameters: - name: id in: path required: true schema: type: string format: uuid responses: 200: description: User found content: application/json: schema: $ref: '#/components/schemas/User' 404: description: User not found content: application/json: schema: $ref: '#/components/schemas/Error' 401: description: Unauthorized rate_limit: requests: 100 window: 60s

  • operation_id: create_user method: POST path: /users summary: Create new user request_body: required: true content: application/json: schema: $ref: '#/components/schemas/CreateUserRequest' responses: 201: description: User created content: application/json: schema: $ref: '#/components/schemas/User' 400: description: Invalid request content: application/json: schema: $ref: '#/components/schemas/Error' 409: description: User already exists

  • operation_id: update_user method: PUT path: /users/{id} summary: Update user parameters:

    • name: id in: path required: true schema: type: string format: uuid request_body: required: true content: application/json: schema: $ref: '#/components/schemas/UpdateUserRequest' responses: 200: description: User updated content: application/json: schema: $ref: '#/components/schemas/User' 404: description: User not found

  • operation_id: delete_user method: DELETE path: /users/{id} summary: Delete user parameters:

    • name: id in: path required: true schema: type: string format: uuid responses: 204: description: User deleted 404: description: User not found

  • operation_id: list_users method: GET path: /users summary: List all users parameters:

    • name: page in: query schema: type: integer minimum: 1 default: 1
    • name: limit in: query schema: type: integer minimum: 1 maximum: 100 default: 20 responses: 200: description: User list content: application/json: schema: type: object properties: users: type: array items: $ref: '#/components/schemas/User' pagination: $ref: '#/components/schemas/Pagination'

components: schemas: User: type: object required: - user_id - username - email - created_at - status properties: user_id: type: string format: uuid description: Unique user identifier username: type: string minLength: 3 maxLength: 20 pattern: '^[a-zA-Z0-9_]+$' description: Unique username email: type: string format: email description: User email address full_name: type: string maxLength: 100 description: User's full name created_at: type: string format: date-time description: Account creation timestamp updated_at: type: string format: date-time description: Last update timestamp status: type: string enum: [active, inactive, suspended] description: Account status

CreateUserRequest:
  type: object
  required:
    - username
    - email
    - password
  properties:
    username:
      type: string
      minLength: 3
      maxLength: 20
      pattern: '^[a-zA-Z0-9_]+$'
    email:
      type: string
      format: email
    password:
      type: string
      minLength: 8
      maxLength: 128
    full_name:
      type: string
      maxLength: 100

UpdateUserRequest:
  type: object
  properties:
    email:
      type: string
      format: email
    full_name:
      type: string
      maxLength: 100
    status:
      type: string
      enum: [active, inactive, suspended]

Error:
  type: object
  required:
    - error_code
    - message
  properties:
    error_code:
      type: string
      description: Machine-readable error code
    message:
      type: string
      description: Human-readable error message
    details:
      type: object
      description: Additional error details

Pagination:
  type: object
  required:
    - page
    - limit
    - total
  properties:
    page:
      type: integer
      description: Current page number
    limit:
      type: integer
      description: Items per page
    total:
      type: integer
      description: Total number of items
    has_next:
      type: boolean
      description: Whether there are more pages

Breaking Change Rules MAJOR Version Changes (X.0.0)

Removed Endpoints

V1

DELETE /users/{id}

V2 (BREAKING)

Endpoint removed

Removed Required Fields

V1

required: [username, email, password]

V2 (BREAKING)

required: [username, email] # password removed

Changed Field Types

V1

user_id: type: string

V2 (BREAKING)

user_id: type: integer

Stricter Validation

V1

username: type: string

V2 (BREAKING)

username: type: string pattern: '^[a-zA-Z0-9_]+$' # New constraint

Changed Authentication

V1

authentication: type: basic

V2 (BREAKING)

authentication: type: bearer scheme: JWT

MINOR Version Changes (x.Y.0)

New Endpoints

V1.0.0

GET /users

V1.1.0 (Non-breaking)

GET /users GET /users/search # New endpoint

New Optional Fields

V1.0.0

properties: username: ...

V1.1.0 (Non-breaking)

properties: username: ... nickname: ... # New optional field

Relaxed Validation

V1.0.0

username: minLength: 5

V1.1.0 (Non-breaking)

username: minLength: 3 # Less strict

PATCH Version Changes (x.y.Z) Documentation Updates Bug Fixes in Descriptions Typo Corrections Example Updates Contract Compatibility Matrix Provider Version Compatibility:

V1.0.0 consumers can use: - V1.0.0, V1.0.1, V1.1.0, V1.2.0 providers ✓ - V2.0.0 providers ❌ (breaking changes)

V1.1.0 consumers can use: - V1.1.0, V1.1.1, V1.2.0 providers ✓ - V1.0.0 providers ⚠️ (missing optional features) - V2.0.0 providers ❌ (breaking changes)

V2.0.0 consumers can use: - V2.0.0, V2.0.1, V2.1.0 providers ✓ - V1.x.x providers ❌ (incompatible)

Tool Access

Required tools:

Read: Read CTR YAML files Write: Generate test files and documentation Bash: Execute validation and testing tools Grep: Search for contract patterns

Required libraries:

pyyaml: YAML parsing jsonschema: Schema validation openapi-spec-validator: OpenAPI validation pact-python: Consumer-driven contract testing faker: Mock data generation Integration Points With doc-flow Validate CTR documents during generation Ensure traceability to requirements Update documentation on contract changes With test-automation Generate contract tests automatically Integrate with test suites Track contract test coverage With code-review Validate API implementation against contracts Detect contract violations in code Suggest contract-compliant fixes With devops-flow Contract validation in CI/CD pipelines Block deployments with breaking changes Generate API gateway configurations Best Practices Semantic versioning: Follow MAJOR.MINOR.PATCH strictly Backward compatibility: Prefer additive changes Deprecation notices: Warn before removing features Consumer feedback: Involve consumers in contract changes Documentation: Clear migration guides for breaking changes Testing: Test all contract changes before release Versioning strategy: Support N-1 versions minimum Change log: Maintain comprehensive contract changelog Success Criteria 100% contract validation coverage Zero undetected breaking changes <5 minute contract validation time 100% consumer-provider compatibility in same MAJOR version All breaking changes documented with migration paths Mock servers available for all contracts All CTR documents include complete Document Control sections with: Project metadata (name, version, date, owner, preparer, status) Document Revision History table tracking all contract changes Notes Contract tests run in CI/CD before deployment Breaking changes require MAJOR version bump Provider must maintain backward compatibility within MAJOR version Consumers should handle minor version differences gracefully Contract changes trigger automatic notification to consumers

← 返回排行榜