{"openapi":"3.1.0","info":{"title":"Competitor Listing Scraper API","version":"1.0.0","description":"Give it a brand's shop / collection / category page and it finds every product listed there, then visits each product page and extracts **title, price, MRP, rating (and count) and images**.\n\n**How discovery works**\n1. If the site is Shopify or WooCommerce, the public catalog JSON is used (fast, complete, exact compare-at prices).\n2. Otherwise a headless browser opens the page and handles both listing styles: it scrolls and clicks \"Load more\" until no new products appear (infinite scroll), and follows \"Next\" / `rel=next` links or `?page=N` (pagination).\n\n**How product pages are read**: JSON-LD → microdata → OpenGraph → review-widget markup (Judge.me, Yotpo, Loox, Stamped, Okendo…) → DOM heuristics (struck-through prices, \"MRP ₹…\" text, gallery images). Pages that need JavaScript are rendered in the browser.\n\n**Typical flow**: `POST /api/scrape` → poll `GET /api/jobs/{id}` until `status` is `completed` → read `products` (or download `/api/jobs/{id}/results.csv`). Products are appended as they finish, so you can read partial results while a job runs.\n\nIf the server has an `API_KEY` set, send it as `X-API-Key` header (or `?api_key=`)."},"servers":[{"url":"https://comp-listing.greygoose.baby"}],"components":{"securitySchemes":{"ApiKey":{"type":"apiKey","in":"header","name":"X-API-Key"}},"schemas":{"ScrapeOptions":{"type":"object","properties":{"mode":{"type":"string","enum":["auto","pagination","infinite"],"default":"auto","description":"`auto` scrolls / clicks \"Load more\" on each page *and* follows Next-page links. `pagination` only follows Next links (or `?page=N`). `infinite` only scrolls the first page."},"usePlatformApi":{"type":"boolean","default":true,"description":"Use Shopify `/products.json` or WooCommerce Store API for discovery when the site runs on those platforms (much faster and complete). Set false to force a browser crawl of exactly the page you gave."},"render":{"type":"string","enum":["auto","always","never"],"default":"auto","description":"Use the headless browser on product pages. `auto` renders only when the static HTML lacks title/price/images or has a JS review widget whose rating is missing."},"maxProducts":{"type":"integer","default":500,"maximum":5000},"maxPages":{"type":"integer","default":50,"maximum":500,"description":"Maximum listing pages to follow (pagination)."},"maxScrolls":{"type":"integer","default":60,"maximum":300,"description":"Maximum scroll / load-more rounds per listing page (infinite scroll)."},"scrollDelayMs":{"type":"integer","default":1500,"description":"Wait after each scroll for new products to load."},"concurrency":{"type":"integer","default":4,"maximum":8,"description":"Product pages fetched in parallel."},"delayMs":{"type":"integer","default":0,"description":"Politeness delay before each product request."},"productLinkSelector":{"type":"string","description":"CSS selector for product links/cards on the listing page, if auto-detection picks up the wrong links. e.g. `.product-card a`"},"nextPageSelector":{"type":"string","description":"CSS selector for the \"Next page\" control."},"loadMoreSelector":{"type":"string","description":"CSS selector for a \"Load more\" button."}}},"Product":{"type":"object","properties":{"url":{"type":"string"},"title":{"type":["string","null"]},"price":{"type":["number","null"],"description":"Current selling price."},"mrp":{"type":["number","null"],"description":"MRP / list / compare-at price. Equals `price` when no discount is shown."},"currency":{"type":["string","null"],"example":"INR"},"discountPercent":{"type":"number","example":25},"rating":{"type":["number","null"],"example":4.6},"ratingCount":{"type":["integer","null"],"example":128},"images":{"type":"array","items":{"type":"string"}},"brand":{"type":["string","null"]},"sku":{"type":["string","null"]},"availability":{"type":["string","null"],"example":"InStock"},"priceRange":{"type":"object","properties":{"low":{"type":"number"},"high":{"type":"number"}},"description":"Present when variants have different prices."},"renderedWithBrowser":{"type":"boolean"},"_sources":{"type":"object","description":"Which source each field came from (platform-api, json-ld, microdata, meta, review-widget, dom, rendered)."},"error":{"type":"string","description":"Present when the page could not be scraped."}}},"Job":{"type":"object","properties":{"id":{"type":"string","format":"uuid"},"status":{"type":"string","enum":["queued","running","completed","failed","cancelled"]},"url":{"type":"string"},"options":{"type":"object","properties":{"mode":{"type":"string","enum":["auto","pagination","infinite"],"default":"auto","description":"`auto` scrolls / clicks \"Load more\" on each page *and* follows Next-page links. `pagination` only follows Next links (or `?page=N`). `infinite` only scrolls the first page."},"usePlatformApi":{"type":"boolean","default":true,"description":"Use Shopify `/products.json` or WooCommerce Store API for discovery when the site runs on those platforms (much faster and complete). Set false to force a browser crawl of exactly the page you gave."},"render":{"type":"string","enum":["auto","always","never"],"default":"auto","description":"Use the headless browser on product pages. `auto` renders only when the static HTML lacks title/price/images or has a JS review widget whose rating is missing."},"maxProducts":{"type":"integer","default":500,"maximum":5000},"maxPages":{"type":"integer","default":50,"maximum":500,"description":"Maximum listing pages to follow (pagination)."},"maxScrolls":{"type":"integer","default":60,"maximum":300,"description":"Maximum scroll / load-more rounds per listing page (infinite scroll)."},"scrollDelayMs":{"type":"integer","default":1500,"description":"Wait after each scroll for new products to load."},"concurrency":{"type":"integer","default":4,"maximum":8,"description":"Product pages fetched in parallel."},"delayMs":{"type":"integer","default":0,"description":"Politeness delay before each product request."},"productLinkSelector":{"type":"string","description":"CSS selector for product links/cards on the listing page, if auto-detection picks up the wrong links. e.g. `.product-card a`"},"nextPageSelector":{"type":"string","description":"CSS selector for the \"Next page\" control."},"loadMoreSelector":{"type":"string","description":"CSS selector for a \"Load more\" button."}}},"createdAt":{"type":"string","format":"date-time"},"startedAt":{"type":["string","null"]},"finishedAt":{"type":["string","null"]},"elapsedMs":{"type":"integer"},"progress":{"type":"object","properties":{"phase":{"type":"string","enum":["queued","discovering","scraping","completed","failed","cancelled"]},"discovered":{"type":"integer"},"scraped":{"type":"integer"},"failed":{"type":"integer"},"page":{"type":"integer"}}},"platform":{"type":["string","null"],"enum":["shopify","woocommerce","magento","bigcommerce","wix","squarespace","generic",null]},"discoveryMethod":{"type":["string","null"],"enum":["shopify-api","woocommerce-api","browser-crawl",null]},"strategy":{"type":["object","null"],"description":"For browser crawls: pagesVisited, usedInfiniteScroll, usedPagination, usedPageParamGuess."},"productCount":{"type":"integer"},"products":{"type":"array","items":{"$ref":"#/components/schemas/Product"}},"error":{"type":["string","null"]}}},"Error":{"type":"object","properties":{"error":{"type":"string"}}}}},"security":[{"ApiKey":[]},{}],"paths":{"/api/scrape":{"post":{"summary":"Start a full scrape (listing → every product page)","description":"Creates an async job and returns it immediately (HTTP 202). Pass `?wait=true` to block until it finishes (up to `timeout` seconds, default 300) — handy for small catalogs.","parameters":[{"name":"wait","in":"query","schema":{"type":"boolean"}},{"name":"timeout","in":"query","schema":{"type":"integer","default":300,"maximum":900},"description":"Seconds to wait when wait=true."}],"requestBody":{"required":true,"content":{"application/json":{"schema":{"allOf":[{"type":"object","required":["url"],"properties":{"url":{"type":"string","description":"Shop / collection / category / search page URL."},"webhookUrl":{"type":"string","description":"Optional: POSTed the finished job (with products) when it ends."}}},{"$ref":"#/components/schemas/ScrapeOptions"}]},"examples":{"shopify":{"summary":"Shopify collection","value":{"url":"https://www.allbirds.com/collections/mens","maxProducts":50}},"generic":{"summary":"Generic site, infinite scroll","value":{"url":"https://example-shop.com/shop","mode":"infinite","maxProducts":200}},"custom":{"summary":"Custom selectors","value":{"url":"https://example-shop.com/category/shoes","productLinkSelector":".product-tile a.title","nextPageSelector":"a.pagination-next"}}}}}},"responses":{"200":{"description":"Job finished (wait=true)","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Job"}}}},"202":{"description":"Job created","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Job"}}}},"400":{"description":"Error","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"401":{"description":"Error","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Error","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/jobs":{"get":{"summary":"List recent jobs (without products)","responses":{"200":{"description":"Jobs","content":{"application/json":{"schema":{"type":"array","items":{"$ref":"#/components/schemas/Job"}}}}}}}},"/api/jobs/{id}":{"get":{"summary":"Job status, progress and results","parameters":[{"name":"id","in":"path","required":true,"schema":{"type":"string"}},{"name":"offset","in":"query","schema":{"type":"integer","default":0}},{"name":"limit","in":"query","schema":{"type":"integer"}},{"name":"products","in":"query","schema":{"type":"boolean","default":true},"description":"Set false to get status only."}],"responses":{"200":{"description":"Job","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Job"}}}},"404":{"description":"Error","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}},"delete":{"summary":"Cancel a queued or running job","parameters":[{"name":"id","in":"path","required":true,"schema":{"type":"string"}}],"responses":{"200":{"description":"Job","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Job"}}}},"404":{"description":"Error","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/jobs/{id}/results.csv":{"get":{"summary":"Download results as CSV","parameters":[{"name":"id","in":"path","required":true,"schema":{"type":"string"}}],"responses":{"200":{"description":"CSV (images joined with \" | \")","content":{"text/csv":{}}},"404":{"description":"Error","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/jobs/{id}/results.json":{"get":{"summary":"Download products as a JSON array","parameters":[{"name":"id","in":"path","required":true,"schema":{"type":"string"}}],"responses":{"200":{"description":"Products","content":{"application/json":{"schema":{"type":"array","items":{"$ref":"#/components/schemas/Product"}}}}},"404":{"description":"Error","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/listing":{"post":{"summary":"Discover product URLs only (synchronous)","description":"Runs just the listing phase and returns product URLs. Useful to check detection before a full scrape.","requestBody":{"required":true,"content":{"application/json":{"schema":{"allOf":[{"type":"object","required":["url"],"properties":{"url":{"type":"string"}}},{"$ref":"#/components/schemas/ScrapeOptions"}]},"example":{"url":"https://www.allbirds.com/collections/mens","maxProducts":20}}}},"responses":{"200":{"description":"Discovered products","content":{"application/json":{"schema":{"type":"object","properties":{"platform":{"type":"string"},"discoveryMethod":{"type":"string"},"strategy":{"type":["object","null"]},"count":{"type":"integer"},"urls":{"type":"array","items":{"type":"string"}}}}}}},"400":{"description":"Error","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/product":{"post":{"summary":"Scrape a single product page (synchronous)","requestBody":{"required":true,"content":{"application/json":{"schema":{"type":"object","required":["url"],"properties":{"url":{"type":"string"},"render":{"type":"string","enum":["auto","always","never"],"default":"auto"}}},"example":{"url":"https://www.allbirds.com/products/mens-tree-runners"}}}},"responses":{"200":{"description":"Product","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Product"}}}},"400":{"description":"Error","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"422":{"description":"Error","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/health":{"get":{"summary":"Health check","security":[],"responses":{"200":{"description":"OK"}}}}}}