All projects matching “deduplicate”

Academic and scientific paper search API. Search research papers and scholarly literature across arXiv, PubMed, OpenAlex, Crossref, and Semantic Scholar in one call, deduplicated into a ranked result set: title, authors, year, venue, abstract, citation count, DOI, and direct PDF link. Query by keyword or restrict to one source; filter by year range; sort by relevance, recency, or citations. Keyles

All links on a page, absolute and deduplicated, split into internal and external.

Read any public web page as clean Markdown: main-content extraction (navigation, footers, scripts and hidden elements removed), headings, lists, tables, code blocks, links (absolute, deduplicated), images, plus title, description, canonical URL, language, author and publish date, and a content hash + fetch time as evidence. JSON and plain-text URLs are returned as-is. No JavaScript rendering. $0.0

Japanese address normalization (住所正規化): standardize a JP address into clean prefecture / city / town / chome-banchi components. Fixes fullwidth/halfwidth digits, old-form kanji, and chome-banchi notation variants, and resolves the 7-digit postal code. Returns a 0-1 confidence score so agents can flag low-quality input. Use for address validation, cleansing, deduplication, and canonicalization befo

Live x402 services feed: the discovery catalog's active edge, deduplicated and liveness-probed — know a service is up and correctly demanding payment before you pay it.

Top crypto stories of the last N hours, deduplicated across 13 outlets and ranked by number of distinct outlets reporting them, then recency: headline, outlets, links, firstSeen, asset/theme tags.

Search open U.S. federal contract opportunities (solicitations, presolicitations, sources sought) from a copy synced from SAM.gov several times a day. Use when you need current government RFPs or bids matching a title keyword, 6-digit NAICS code, small-business set-aside or place-of-performance state. Returns deduplicated JSON: notice ID, title, agency, NAICS, set-aside, posted date, response dead

Scientific and academic paper search: research papers and scholarly literature from arXiv, PubMed, OpenAlex, Crossref and Semantic Scholar in one call, deduplicated and retraction-aware. Each hit: DOI, authors, venue, snippet, citation count and citations per year, type, open-access PDF link. Sort by relevance, recency, citations or impact. Keyless. For literature review and citing research.

Data Cleaner - Clean, validate, standardize and deduplicate tabular datasets, preserve originals, and produce Excel, PDF, JSON or TXT export reports with full change logs.

Crypto news feed, $0.0008 USDC per call: deduplicated headlines from 37 public sources (CoinDesk, Cointelegraph, The Block, Decrypt, DL News, Blockworks, Ethereum/Solana/Base blogs, Uniswap/Aave/Arbitrum/Optimism forums, GitHub releases of geth, reth, agave, Bitcoin Core). Each item: title, source url, source, published_at, topics, tickers, summary. Filters: topics, tickers, q, type. since cursor

JSON canonicalization — canonicalize structured data with RFC 8785 JCS; return canonical JSON text, UTF-8 byte length, and SHA-256 hash for integrity checks, signature preparation, and deduplication. Includes an EIP-712 signed attestation. Use when an agent needs an idempotency key, cache key, dedup hash, or verifiable commitment. $0.002 per call. Free verification at POST /v1/verify.

Verify an on-chain payment without trusting any RPC: did this tx transfer at least this amount of a token (default USDC) to this payee, optionally from this payer? Proven by Plumb (Base via proof-backed L1 checkpoints, Ethereum via a light client, XRP Ledger via a quorum of Ripple's validator…