22, March 2010

Links and robots - webmaster forum

 
Webdigity webmaster forums
[ Home | Help | Search | Forum's Shop | Archive | Login | Register | Webmaster Directory ]
Webdigity Webmaster Forums  >  Design and Layout  >  Web Page Design  >  HTML & XHTML (Moderator: Meth0d)
Topic: Links and robots
« previous next »
Pages: [1] Print

Author Topic: Links and robots  (Read 1249 times)
Global Moderator
Internet Junkie
*****
Gender: Male
Posts: 1525
6359 credits
Members referred : 8


Gimme all your cookies!!!


« on: Apr 14, 2006, 07:04:57 am »

I have a script on my error404 page that emails me the file that was not found, the referring page and the user agent that requested the file. Recently I have been receiving the following messages:

Quote
File not found: www. site .com/'http: //www. link .com'     [no spaces obviously]
Refering page:
User agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

So there is no referring page... obviously because it is the google bot, but my question is this... Why is the google bot adding the external link to the site domain with single quotes around it? I presume that the bot sees <a href='...' >, with single quotes and for some reason is adding the link to the main site link and trying to see if that page exists. I have changed as manu links as possible to <a href="..." > now so that there are double quotes aroung the url to try and prevent this.

Anyone have any idea why this bot is behaving like this?   Huh


Last blog : Canonical URL Links / Tags
I am a metal monkey!
Administrator
Community Supporter ?
Jedai Sword Master
*****
Gender: Male
Posts: 5660
45593 credits
Members referred : 3



« Reply #1 on: Apr 14, 2006, 11:35:09 am »

This is very strange for G. I think that this might propably be a spam bot that uses the googlebot headers, or a bug of the googlebot 2.

But anyway if you don't have a link to ww. site .com/'http: //www. link .com'  in your site there is no problem.

Also have in mind that the gbot continues to visit non existent pages for a long time.

Trial and Error my two best teachers Cool
Join us @ facebook or twitter

Last blog : Butterfly Marketing 2.0
Global Moderator
Community Supporter ?
Jedai Sword Master
*****
Gender: Male
Posts: 6690
34708 credits
Members referred : 374


It's time to use PHP5!


« Reply #2 on: Apr 14, 2006, 03:25:09 pm »

Right, the Googlebit spiders your site without any referer information, check the corresponding ip address. I don't think that this one is from Google.

Olaf


Last blog : A new Wordpress theme for our blog
Trackback URI for this entry : http://www.webdigity.com/trackback.php?topic=2264
Tags : google domains spam Bookmark this thread : Digg Del.icio.us Dzone more....

Pages: [1] Print 
Webdigity Webmaster Forums  >  Design and Layout  >  Web Page Design  >  HTML & XHTML (Moderator: Meth0d)
Topic: Links and robots
« previous next »
Jump to:
User Area
Welcome, Guest. Please login or register.
Did you miss your activation email?
Mar 22, 2010, 12:59:55 am





Login with username, password and session length

Donate to our community, and get a permanent link back to your site!

Donate to our community, and get a permanent link back to your site!


Forum Statistics
Total Posts: 44.238
Total Topics: 8.626
Total Members: 8.245
Tutorials : 58
Resources : 929
Designs : 361
Latest Member: clickonportal

25 Guests, 4 Users online :

10 users online today:



Readers

Web Design Gallery · Whois Lookup · Pagerank · Tag Browsing · Lo-fi version · Syndication · Webmaster forum history · Advertise
Developed by HumanWorks © 2005 - 2010 Webdigity webmaster community · sublime directory
Webdigity Webmaster Forums | Powered by SMF 1.0.12. © 2001-2005, Lewis Media. All Rights Reserved.