arize-trace

仓库: arize-ai/arize-skills
安装量: 45
排名: #16346

安装

npx skills add https://github.com/arize-ai/arize-skills --skill arize-trace
Arize Trace Skill
Concepts
Trace
= a tree of spans sharing a
context.trace_id
, rooted at a span with
parent_id = null
Span
= a single operation (LLM call, tool call, retriever, chain, agent)
Session
= a group of traces sharing
attributes.session.id
(e.g., a multi-turn conversation)
Use
ax spans export
to download trace data. This is the only supported command for retrieving spans.
Resolving project for export:
The
PROJECT
positional argument accepts either a project name or a base64 project ID. When using a name,
--space-id
is required. If you hit limit errors or
401 Unauthorized
when using a project name, resolve it to a base64 ID: run
ax projects list --space-id SPACE_ID -l 100 -o json
, find the project by
name
, and use its
id
as
PROJECT
.
Exploratory export rule:
When exporting spans or traces
without
a specific
--trace-id
,
--span-id
, or
--session-id
(i.e., browsing/exploring a project), always start with
-l 50
to pull a small sample first. Summarize what you find, then pull more data only if the user asks or the task requires it. This avoids slow queries and overwhelming output on large projects.
Default output directory:
Always use
--output-dir .arize-tmp-traces
on every
ax spans export
call. The CLI automatically creates the directory and adds it to
.gitignore
.
Prerequisites
Three things are needed:
ax
CLI, an API key (env var or profile), and a space ID. A project name or ID is also needed but usually comes from the user's message.
Install ax
Verify
ax
is installed and working before proceeding:
Check if
ax
is on PATH:
command -v ax
(Unix) or
where ax
(Windows)
If not found, check common install locations:
macOS/Linux:
test -x ~/.local/bin/ax && export PATH="$HOME/.local/bin:$PATH"
Windows: check
%APPDATA%\Python\Scripts\ax.exe
or
%LOCALAPPDATA%\Programs\Python\Scripts\ax.exe
If still not found, install it (requires shell access to install packages):
Preferred:
uv tool install arize-ax-cli
Alternative:
pipx install arize-ax-cli
Fallback:
pip install arize-ax-cli
After install, if
ax
is not on PATH:
macOS/Linux:
export PATH="$HOME/.local/bin:$PATH"
Windows (PowerShell):
$env:PATH = "$env:APPDATA\Python\Scripts;$env:PATH"
If
ax --version
fails with an SSL/certificate error:
macOS:
export SSL_CERT_FILE=/etc/ssl/cert.pem
Linux:
export SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
Windows (PowerShell):
$env:SSL_CERT_FILE = "C:\Program Files\Common Files\SSL\cert.pem"
(or use
python -c "import certifi; print(certifi.where())"
to find the cert bundle)
ax --version
must succeed before proceeding. If it doesn't, stop and ask the user for help.
Verify environment
Run a quick check for credentials:
macOS/Linux (bash):
ax
--version
&&
echo
"--- env ---"
&&
if
[
-n
"
$ARIZE_API_KEY
"
]
;
then
echo
"ARIZE_API_KEY: (set)"
;
else
echo
"ARIZE_API_KEY: (not set)"
;
fi
&&
echo
"ARIZE_SPACE_ID:
${ARIZE_SPACE_ID
:-
(not set)}
"
&&
echo
"--- profiles ---"
&&
ax profiles show
2
>
&1
Windows (PowerShell):
ax
--
version
;
Write-Host
"--- env ---"
;
Write-Host
"ARIZE_API_KEY:
$
(
if
(
$env
:ARIZE_API_KEY
)
{ '(set)' } else { '(not set)' })"
;
Write-Host
"ARIZE_SPACE_ID:
$env
:ARIZE_SPACE_ID"
;
Write-Host
"--- profiles ---"
;
ax profiles show 2>&1
Read the output and proceed immediately
if either the env var or the profile has an API key. Only ask the user if
both
are missing. Resolve failures:
No API key in env
and
no profile →
AskQuestion
"Arize API key ( https://app.arize.com/admin

API Keys)" Space ID unknown → AskQuestion , or run ax projects list -o json --limit 100 --space-id $ARIZE_SPACE_ID and present as selectable options Project unclear → ask, or run ax projects list -o json --limit 100 and search for a match IMPORTANT: --space-id is required when using a human-readable project name as the PROJECT positional argument. It is not needed when using a base64-encoded project ID. If you hit 401 Unauthorized or limit errors when using a project name, resolve it to a base64 ID first (see "Resolving project for export" in Concepts). Export Spans: ax spans export The primary command for downloading trace data to a file. By trace ID ax spans export PROJECT_ID --trace-id TRACE_ID --output-dir .arize-tmp-traces By span ID ax spans export PROJECT_ID --span-id SPAN_ID --output-dir .arize-tmp-traces By session ID ax spans export PROJECT_ID --session-id SESSION_ID --output-dir .arize-tmp-traces Flags Flag Type Required Description --trace-id string mutex Filter: context.trace_id = 'X' --span-id string mutex Filter: context.span_id = 'X' --session-id string mutex Filter: attributes.session.id = 'X' PROJECT string (positional) yes (or $ARIZE_DEFAULT_PROJECT ) Project name or base64 ID (positional arg, not a flag) --space-id string yes (when PROJECT is a name) Space ID; required to resolve project names. Not needed when using a base64 project ID. Also required when using --all (Arrow Flight). --days int no Lookback window (default: 30) --start-time string no Override start (ISO 8601) --end-time string no Override end (ISO 8601) --output-dir string no Output directory (default: .arize-tmp-traces ; ensure it is gitignored — see above) --stdout bool no Print JSON to stdout instead of file --all bool no Use Arrow Flight for bulk export (see below) Exactly one of --trace-id , --span-id , --session-id is required. Output is a JSON array of span objects. File naming: {type}{id}/spans.json . Bulk export with --all (Arrow Flight) By default, ax spans export uses the REST API which is limited to 500 spans per page and capped by --limit . Pass --all to switch to Arrow Flight for streaming bulk export with no span limit. ax spans export PROJECT_ID --space-id SPACE_ID --filter "status_code = 'ERROR'" --all --output-dir .arize-tmp-traces REST vs Flight trade-offs: REST (default): Lower friction -- no Arrow/Flight dependency needed, uses standard HTTPS ports, works through any corporate proxy or firewall. Limited to 500 spans per page. Flight ( --all ): Required for bulk export beyond 500 spans. Uses gRPC+TLS on a separate host/port which some corporate networks may block. When to use --all : Exporting more than 500 spans Downloading full traces with many child spans Large time-range exports Agent auto-escalation rule: If a REST export returns exactly the number of spans requested by -l (or 500 if no limit was set), the result is likely truncated. Increase -l or re-run with --all to get the full dataset — but only when the user asks or the task requires more data. Requirements for --all : --space-id is required (Flight uses space_id + project_name , not project_id ) --limit is ignored when --all is set Networking notes for --all : Arrow Flight connects to flight.arize.com:443 via gRPC+TLS -- this is a different host from the REST API ( api.arize.com ). On internal or private networks, the Flight endpoint may use a different host/port. Configure via: ax profile: flight_host , flight_port , flight_scheme Environment variables: ARIZE_FLIGHT_HOST , ARIZE_FLIGHT_PORT , ARIZE_FLIGHT_SCHEME The --all flag is also available on ax traces export , ax datasets export , and ax experiments export with the same behavior (REST by default, Flight with --all ). Export Traces: ax traces export Export full traces -- all spans belonging to traces that match a filter. Uses a two-phase approach: Phase 1: Find spans matching --filter (up to --limit via REST, or all via Flight with --all ) Phase 2: Extract unique trace IDs, then fetch every span for those traces

Explore recent traces (start small with -l 50, pull more if needed)

ax traces export PROJECT_ID -l 50 --output-dir .arize-tmp-traces

Export traces with error spans (REST, up to 500 spans in phase 1)

ax traces export PROJECT_ID --filter "status_code = 'ERROR'" --stdout

Export all traces matching a filter via Flight (no limit)

ax traces
export
PROJECT_ID --space-id SPACE_ID
--filter
"status_code = 'ERROR'"
--all
Flags
Flag
Type
Default
Description
PROJECT
string
required
Project name or base64 ID (positional arg)
--filter
string
none
Filter expression for phase-1 span lookup
--space-id
string
none
Space ID; required when
PROJECT
is a name or when using
--all
(Arrow Flight)
--limit, -l
int
50
Max number of traces to export
--days
int
30
Lookback window in days
--start-time
string
none
Override start (ISO 8601)
--end-time
string
none
Override end (ISO 8601)
--output-dir
string
.
Output directory
--stdout
bool
false
Print JSON to stdout instead of file
--all
bool
false
Use Arrow Flight for both phases (see spans
--all
docs above)
-p, --profile
string
default
Configuration profile
How it differs from
ax spans export
ax spans export
exports individual spans matching a filter
ax traces export
exports complete traces -- it finds spans matching the filter, then pulls ALL spans for those traces (including siblings and children that may not match the filter)
Filter Syntax Reference
SQL-like expressions passed to
--filter
.
Common filterable columns
Column
Type
Description
Example Values
name
string
Span name
'ChatCompletion'
,
'retrieve_docs'
status_code
string
Status
'OK'
,
'ERROR'
,
'UNSET'
latency_ms
number
Duration in ms
100
,
5000
parent_id
string
Parent span ID
null for root spans
context.trace_id
string
Trace ID
context.span_id
string
Span ID
attributes.session.id
string
Session ID
attributes.openinference.span.kind
string
Span kind
'LLM'
,
'CHAIN'
,
'TOOL'
,
'AGENT'
,
'RETRIEVER'
,
'RERANKER'
,
'EMBEDDING'
,
'GUARDRAIL'
,
'EVALUATOR'
attributes.llm.model_name
string
LLM model
'gpt-4o'
,
'claude-3'
attributes.input.value
string
Span input
attributes.output.value
string
Span output
attributes.error.type
string
Error type
'ValueError'
,
'TimeoutError'
attributes.error.message
string
Error message
event.attributes
string
Error tracebacks
Use CONTAINS (not exact match)
Operators
=
,
!=
,
<
,
<=
,
>
,
>=
,
AND
,
OR
,
IN
,
CONTAINS
,
LIKE
,
IS NULL
,
IS NOT NULL
Examples
status_code = 'ERROR'
latency_ms > 5000
name = 'ChatCompletion' AND status_code = 'ERROR'
attributes.llm.model_name = 'gpt-4o'
attributes.openinference.span.kind IN ('LLM', 'AGENT')
attributes.error.type LIKE '%Transport%'
event.attributes CONTAINS 'TimeoutError'
Tips
Prefer
IN
over multiple
OR
conditions:
name IN ('a', 'b', 'c')
not
name = 'a' OR name = 'b' OR name = 'c'
Start broad with
LIKE
, then switch to
=
or
IN
once you know exact values
Use
CONTAINS
for
event.attributes
(error tracebacks) -- exact match is unreliable on complex text
Always wrap string values in single quotes
Workflows
Debug a failing trace
ax traces export PROJECT_ID --filter "status_code = 'ERROR'" -l 50 --output-dir .arize-tmp-traces
Read the output file, look for spans with
status_code: ERROR
Check
attributes.error.type
and
attributes.error.message
on error spans
Download a conversation session
ax spans export PROJECT_ID --session-id SESSION_ID --output-dir .arize-tmp-traces
Spans are ordered by
start_time
, grouped by
context.trace_id
If you only have a trace_id, export that trace first, then look for
attributes.session.id
in the output to get the session ID
Export for offline analysis
ax spans
export
PROJECT_ID --trace-id TRACE_ID --output-dir .arize-tmp-traces
--stdout
|
jq
'.[]'
Span Column Reference (OpenInference Semantic Conventions)
Core Identity and Timing
Column
Description
name
Span operation name (e.g.,
ChatCompletion
,
retrieve_docs
)
context.trace_id
Trace ID -- all spans in a trace share this
context.span_id
Unique span ID
parent_id
Parent span ID.
null
for root spans (= traces)
start_time
When the span started (ISO 8601)
end_time
When the span ended
latency_ms
Duration in milliseconds
status_code
OK
,
ERROR
,
UNSET
status_message
Optional message (usually set on errors)
attributes.openinference.span.kind
LLM
,
CHAIN
,
TOOL
,
AGENT
,
RETRIEVER
,
RERANKER
,
EMBEDDING
,
GUARDRAIL
,
EVALUATOR
Where to Find Prompts and LLM I/O
Generic input/output (all span kinds):
Column
What it contains
attributes.input.value
The input to the operation. For LLM spans, often the full prompt or serialized messages JSON. For chain/agent spans, the user's question.
attributes.input.mime_type
Format hint:
text/plain
or
application/json
attributes.output.value
The output. For LLM spans, the model's response. For chain/agent spans, the final answer.
attributes.output.mime_type
Format hint for output
LLM-specific message arrays (structured chat format):
Column
What it contains
attributes.llm.input_messages
Structured input messages array (system, user, assistant, tool).
Where chat prompts live
in role-based format.
attributes.llm.input_messages.roles
Array of roles:
system
,
user
,
assistant
,
tool
attributes.llm.input_messages.contents
Array of message content strings
attributes.llm.output_messages
Structured output messages from the model
attributes.llm.output_messages.contents
Model response content
attributes.llm.output_messages.tool_calls.function.names
Tool calls the model wants to make
attributes.llm.output_messages.tool_calls.function.arguments
Arguments for those tool calls
Prompt templates:
Column
What it contains
attributes.llm.prompt_template.template
The prompt template with variable placeholders (e.g.,
"Answer {question} using {context}"
)
attributes.llm.prompt_template.variables
Template variable values (JSON object)
Finding prompts by span kind:
LLM span
Check
attributes.llm.input_messages
for structured chat messages, OR
attributes.input.value
for serialized prompt. Check
attributes.llm.prompt_template.template
for the template.
Chain/Agent span
Check
attributes.input.value
for the user's question. Actual LLM prompts are on child LLM spans.
Tool span
Check attributes.input.value for tool input, attributes.output.value for tool result. LLM Model and Cost Column Description attributes.llm.model_name Model identifier (e.g., gpt-4o , claude-3-opus-20240229 ) attributes.llm.invocation_parameters Model parameters JSON (temperature, max_tokens, top_p, etc.) attributes.llm.token_count.prompt Input token count attributes.llm.token_count.completion Output token count attributes.llm.token_count.total Total tokens attributes.llm.cost.prompt Input cost in USD attributes.llm.cost.completion Output cost in USD attributes.llm.cost.total Total cost in USD Tool Spans Column Description attributes.tool.name Tool/function name attributes.tool.description Tool description attributes.tool.parameters Tool parameter schema (JSON) Retriever Spans Column Description attributes.retrieval.documents Retrieved documents array attributes.retrieval.documents.ids Document IDs attributes.retrieval.documents.scores Relevance scores attributes.retrieval.documents.contents Document text content attributes.retrieval.documents.metadatas Document metadata Reranker Spans Column Description attributes.reranker.query The query being reranked attributes.reranker.model_name Reranker model attributes.reranker.top_k Number of results attributes.reranker.input_documents. Input documents (ids, scores, contents, metadatas) attributes.reranker.output_documents. Reranked output documents Session, User, and Custom Metadata Column Description attributes.session.id Session/conversation ID -- groups traces into multi-turn sessions attributes.user.id End-user identifier attributes.metadata.* Custom key-value metadata. Any key under this prefix is user-defined (e.g., attributes.metadata.user_email ). Filterable. Errors and Exceptions Column Description attributes.exception.type Exception class name (e.g., ValueError , TimeoutError ) attributes.exception.message Exception message text event.attributes Error tracebacks and detailed event data. Use CONTAINS for filtering. Evaluations and Annotations Column Description annotation..label Human or auto-eval label (e.g., correct , incorrect ) annotation..score Numeric score (e.g., 0.95 ) annotation..text Freeform annotation text Embeddings Column Description attributes.embedding.model_name Embedding model name attributes.embedding.texts Text chunks that were embedded Troubleshooting Problem Solution ax: command not found Check ~/.local/bin/ax ; if missing: uv tool install arize-ax-cli (requires shell access to install packages). Then export PATH="$HOME/.local/bin:$PATH" SSL: CERTIFICATE_VERIFY_FAILED macOS: export SSL_CERT_FILE=/etc/ssl/cert.pem . Linux: export SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt . Windows: $env:SSL_CERT_FILE = (python -c "import certifi; print(certifi.where())") No such command on a subcommand that should exist The installed ax is outdated. Reinstall from the local workspace: uv tool install --force --reinstall /path/to/arize/sdk/python/arize-ax-cli (requires shell access to install packages) No profile found Follow "Resolve credentials" in Prerequisites to auto-discover or prompt for the API key 401 Unauthorized with valid API key You are likely using a project name without --space-id . Add --space-id SPACE_ID , or resolve to a base64 project ID first: ax projects list --space-id SPACE_ID -l 100 -o json and use the project's id . No spans found Expand --days (default 30), verify project ID Filter error Check column name spelling, wrap string values in single quotes Timeout on large export Use --days 7 to narrow the time range Save Credentials At session end, if the user manually provided an API key, space ID, or project ID (not loaded from an existing profile), offer to save them to ~/.arize/config.toml . Use AskQuestion with "Yes, save them" / "No thanks". Skip if all values were already in the profile. Read the existing file (or create it), add/update only the new fields, and write it back: [ auth ] api_key = "THE_API_KEY" [ defaults ] space_id = "THE_SPACE_ID" project = "THE_PROJECT_NAME"
← 返回排行榜