Search - All Tools - Fetch Tool - Parse Tool - Dashboard

Search Engine Dashboard

This is where you can peek at all the internals of the search engine. Currently you can see all new URLs scheduled for adding with more coming soon.

Currenlty about one page is fetched every minute. The whole process of fetching and indexing a page takes around 6 seconds. I have to limit the amount of time spent indexing as I'm running on a shared hosting account and I don't wan't to overload the server.

System Status

Current time is: Mon, 08 Sep 2008 03:41:44 -0500
Pages in index: 49157 (updated every day)
Links in index: 2883175 (updated every day)
Keywords in index: 15044848 (updated every day)
Pages in fetch queue: 147
New bot-discovered URLs in queue: 500
New user submitted URLs in queue: 8345

Spider Generated URL Queue

This is where all new URLs found by the spider while parsing pages goes.

The User Submitted URL Queue

This is where all user submitted URLs go. The fetcher will take one site from this queue every time it runs, fetch it, parse it and add it to the index if all goes well.

Add Your URL:

URLs from the Spider URL Queue are validated and put into the fetch queue URLs from the User URL Queue are validated and put into the fetch queue

Fetch Queue

This is pages queued to be fetched. They can be new pages to be fetched for the first time or old pages that needs to be refreshed. There is 147 pages in the queue, the first 500 is shown here.

URLs from the Spider URL Queue are validated and put into the fetch queue

Log of last fetch

Here the spider logs what it did during the last run

Find Another Website

Home - Dashboard - Add URL

Byholm.com - Casual Clothing - Search - Directory - eBooks - eCards - eBiz Tutorial - Bible Info - Mount St. Helens - Scam Info