| spider trap | Spider trap refers to a set of links that are used to cause a web crawler to create an infinite number of pages which dont exist on the site or cause a poorly constructed crawler to crash. Spider trap may be used intentionally or unintentionally and are created to catch the spambots. |
| follow | The default state of a link, follow links pass PageRank. |
| site-wide link | A hyperlink that appears on every webpage, usually due to its position on the header, side, or footer sections. |
| crawl friction | Noun phrase. The obstructions: inadvertent or intentional: a Website creates to efficient crawling. Examples include redirects, blocking resources in robots.txt, poor site structure, embedding too many URLs in a page, etc. |
| average volume | Average monthly search volume for a queried term. |
