Also called spider or crawler. A program which scans and analyzes HTML documents for further processing (e.g. for a search engine). During the scanning process, the program may also automatically follow hypertext references in the documents, so that the complete document tree of a web-site can be caught.