Screaming Frog crawler is such a powerful tool, it’s perhaps the best paid tool for SEO with a very economic cost, considering what it offers!
What about the ‘semantic search’ feature in the new version?
New version comes with multiple handy features, at top of them is ‘semantic search’ which imitates Google search and produces a cosine similarity score for a given search query.
Instead of relying only on exact keyword matching, it leverages NLP to understand the contextual meaning behind context in such a way that is much closer to how Google actually reads and interprets web pages nowadays.
For advanced SEO specialists who already leverage semantic entities and think about SEO in terms of semantic relevance, this feature is a gamechanger!
How to activate the semantic search feature in Screaming Frog Spider?
- Connect it first with Claude (or chatgpt) API which I believe requires extra few payment, set up the prompt which in this case would be “extract embedding from page content“
- Run a full crawl with custom extraction to scrape the ‘body-text’ of the website’s pages using CSS element or XPATH
- Activate crawl analysis
It doesn’t only help to see how much pages cover topics/queries in depth, but it also flags semantically similar pages that might be competing for the same query at search, the thing that cause what we call ‘keyword cannibalization’ in SEO terms.
Conclusion:
The addition of semantic search as a quite new feature is not just a regular upgrade; it reflects a genuine shift toward how modern SEO should be practiced. Whether you’re conducting content SEO audits, flagging cannibalization issues, thinking about cleaning your website index from decaying content, or attempting to align your content with how Google actually processes content, this feature is definitely an added-value for you.