web

Web Scraping & Search Paid web crawling and search API at https://web.surf.cascade.fyi . Crawl costs $0.005 USDC, search costs $0.01 USDC per call via x402 on Solana. Use the x_payment tool for all requests. Use this when the built-in web_fetch tool fails or is blocked (anti-bot, JS-rendered pages, paywalled content). Endpoints Crawl Web Pages Fetch and extract content from any URL. Automatically escalates through fetching tiers if blocked: fast HTTP -> headless browser -> stealth browser with proxy. x_payment({ "url": "https://web.surf.cascade.fyi/v1/crawl", "method": "POST", "params": "{\"url\": \"https://example.com\", \"format\": \"markdown\"}" }) Body parameters: Param Type Default Description url string - Single URL to crawl (required if no urls ) urls string[] - Multiple URLs to bulk crawl (required if no url ) format string markdown Output format: markdown , html , or text selector string - CSS selector to extract specific elements proxy boolean false Force proxy usage (auto-enabled on stealth tier) Single URL response: { "status" : 200 , "content" : [ "# Page Title\n\nPage content in markdown..." ] , "url" : "https://example.com/" } Bulk URL response: [ { "status" : 200 , "content" : [ "..." ] , "url" : "https://example.com/" } , { "status" : 200 , "content" : [ "..." ] , "url" : "https://other.com/" } ] Cascade escalation: The crawler automatically tries increasingly powerful methods: get - Fast HTTP request (cheapest, handles most static sites) fetch - Headless Chromium (for JS-rendered pages) stealthy_fetch - Stealth browser with proxy (bypasses Cloudflare, anti-bot) Escalation triggers: HTTP 403/429/503, empty content, or content under 100 characters. Search the Web Search the web using Exa's search API. Returns titles, URLs, and text snippets. x_payment({ "url": "https://web.surf.cascade.fyi/v1/search", "method": "POST", "params": "{\"query\": \"x402 protocol crypto payments\", \"num_results\": 10}" }) Body parameters: Param Type Default Description query string required Search query num_results 1-20 5 Number of results to return Response: { "results" : [ { "title" : "Page Title" , "url" : "https://example.com/page" , "snippet" : "Relevant text excerpt from the page..." } ] } Usage Patterns Scrape a page as markdown x_payment({ "url": "https://web.surf.cascade.fyi/v1/crawl", "method": "POST", "params": "{\"url\": \"https://docs.solana.com\", \"format\": \"markdown\"}" }) Extract specific elements with a CSS selector x_payment({ "url": "https://web.surf.cascade.fyi/v1/crawl", "method": "POST", "params": "{\"url\": \"https://news.ycombinator.com\", \"format\": \"html\", \"selector\": \".titleline\"}" }) Bulk crawl multiple URLs x_payment({ "url": "https://web.surf.cascade.fyi/v1/crawl", "method": "POST", "params": "{\"urls\": [\"https://example.com\", \"https://httpbin.org/get\"], \"format\": \"text\"}" }) Search and then crawl top results First search: x_payment({ "url": "https://web.surf.cascade.fyi/v1/search", "method": "POST", "params": "{\"query\": \"ERC-8004 agent identity standard\", \"num_results\": 5}" }) Then crawl the most relevant result: x_payment({ "url": "https://web.surf.cascade.fyi/v1/crawl", "method": "POST", "params": "{\"url\": \"https://eips.ethereum.org/EIPS/eip-8004\", \"format\": \"markdown\"}" }) Cost Crawl: $0.005 USDC per call (single or bulk) Search: $0.01 USDC per call All payments on Solana mainnet. Each request is a separate call. For bulk crawls, one payment covers all URLs in the batch. Errors HTTP Meaning 400 Invalid parameters (check body format) 402 Payment required (handled automatically by x_payment) 429 Too many concurrent requests (retry later) 502 Upstream crawl/search error