|
Let us show you how OrbitSpider will crawl your site
New added links are processed by OrbitSpider each time when it begins new cycle.
If you provide correct link and if your site contains search engine friendly content it will appear among search results in about an hour.
It's just a demo which was designed to demonstrate the basic principles of OrbitSpider's work.
So it works with some constraints to avoid too long delays between site adding and its appearing in search results.
These constarints are:
-
it doesn't process outbound links;
-
it processes maximum 50 pages per domain during one crawling cycle;
-
it takes maximum 100 links per page during one crawling cycle;
-
it takes maximum 200 links from one site;
-
it doesn't recrawl pages which were processed once;
Please note:
Only the top-level page from a host is necessary; you do not need to submit each individual page.
OrbitSpider will find the rest.
|