Each pass adds new sites to the catalogue and informs the next round of searches. Specialist tools extend the search on selected sites.
Discovery feeds the broad search. Collected links feed back into discovery. Search results inform priorities, which guide later searches. Specialist tools extend coverage in parallel. Content from both searches goes through processing and indexing.
01 / Discovery
Discover and classify new sites
The discovery engine is designed to run around the clock. It follows links between sites, examines their content and classifies it to build up the catalogue of sources.
Continuous discovery expands the scope of the search over time.
02 / Broad search
Explore the full catalogue
The general-purpose engine aims for broad coverage of catalogued sites. It distributes searches across domains, retrieves accessible content and discovers new pages.
Collected links feed into new searches, so each pass can turn up new leads.
03 / Prioritisation
Focus resources on useful leads
The system uses search results to adjust its priorities. For example, it distinguishes images unique to a page’s content from shared elements such as navigation graphics.
Less relevant sites and groups of pages can be skipped or explored less often.
Computing resources can then go to the most useful searches.
04 / In parallel
Extend coverage with specialist tools
Some platforms organise their content into galleries, albums or videos that need different handling.
Dedicated tools supplement the broad search by collecting material in ways that suit each site’s structure.
05 / Processing
Turn content into usable data
Broad and specialist searches
Collected images and videos go through several steps: calculating digital fingerprints, detecting duplicates, indexing content and recording where it came from.
This prepares the data for matching and makes it possible to trace the sources of the material found.