All projects matching “token overlap”

Compare text using deterministic, case-insensitive bag-of-words token-overlap similarity for lexical duplicate screening. Lowercase each input and tokenize with Python Unicode regex \w+ (letters, digits and underscores). Count repeated tokens. Similarity = sum(min(count_a(token),count_b(token))) / max(words_a,words_b,1), rounded to four decimal places. Returns exactly similarity in [0,1], words_a

Tokenizes each claim and its supplied evidence, removes a small declared English stop-word list, and reports the fraction of unique claim tokens present in evidence. Coverage below a supplied threshold is flagged. This is lexical grounding screening only: overlap does not prove truth, low overlap does not prove hallucination, and the service does not retrieve or validate evidence.