Sublime directory Surf the web anonymous Pagerank Monitor


How can i allow the spiders to crawl my site ...

lfm1012
Thu 14 July 2011, 08:38 am GMT +0200
I have a site. I have checked the inner page was crawled one month back, usually the page is taking more time to crawl.

C.Rebecca
Thu 14 July 2011, 01:36 pm GMT +0200
For improving the crawler's visit frequency on your website
- Submit your Sitemap to google webmaster tools
- Get backlinks from popular blogs which frequently get crawled
- Ping the search engines regularly

DuncanCox
Fri 15 July 2011, 06:20 am GMT +0200
Hey!! Rebeca
Thanks for the tips. But I want to ask what is Sitemap ?

reihaan
Fri 15 July 2011, 01:36 pm GMT +0200
you can set the crawl rate from google master, please check it. from there you can learn alot of things.

ambi
Mon 18 July 2011, 02:08 pm GMT +0200

You have to change your meta tag like
<meta name="robots" content="index, follow" />

<meta name="revisit-after" content="3 Days" />
Then after you ? ping your site ? through ? pingomatic ? any other sites. defiantly your site crawing rate is increase.


C.Rebecca
Wed 3 August 2011, 01:38 pm GMT +0200
Hey!! Rebeca
Thanks for the tips. But I want to ask what is Sitemap ?
There are two types of sitemaps - html (for human visitors) and xml (for search engines). In my post, I was talking about XML sitemaps...though both are equally important. An XML is a well-structured collection of URLs of your web-pages.

An XML sitemap:
- Ensures better visibility by search engines
- Enables you to rely less on external links that will bring search engines to your site.
- Helps with broken internal links or orphaned pages that cannot be reached in other way
- Alerts search engines to any changes/additions to your site
- Helps you choose pages that you don???t want to index

palmgeo
Fri 19 August 2011, 09:09 am GMT +0200
Continue with some small changes in the website so that search engine could crawl your website more fast.

myseopandit
Fri 23 September 2011, 07:38 am GMT +0200
If you want your webpages to be crawled by Search engine spiders, You should not robots.txt file, and also do off page submissions for frequent crawling, If not update your web pages content.

Plusplushosting-m
Mon 26 September 2011, 03:16 pm GMT +0200
Google is a fully automated search engine that uses software known as "spiders" to crawl the web on a regular basis and find sites to add to our index.

Create sitemap - it will help to get indexed in Google.

Highave
Mon 10 October 2011, 08:47 am GMT +0200
submit your site on google site map, book mark with some high pr do follow bookmarking sites.

Archive for SMF v1.00 by N.P. Valid XHTML 1.0 Transitional