7, October 2008

Links and robots - webmaster forum

 
Webdigity webmaster forums
This forum shares its ad revenue with its members!
[ Home | Help | Search | Forum's Shop | Archive | Login | Register | Webmaster Directory ]
Webdigity Webmaster Forums  >  Design and Layout  >  Web Page Design  >  HTML & XHTML (Moderator: Meth0d)
Topic: Links and robots
« previous next »
Pages: [1] Print

Author Topic: Links and robots  (Read 695 times)
Global Moderator
Internet Junkie
*****
Gender: Male
Posts: 1523
6847 credits
Members referred : 8


Gimme all your cookies!!!


« on: Apr 14, 2006, 07:04:57 AM »

I have a script on my error404 page that emails me the file that was not found, the referring page and the user agent that requested the file. Recently I have been receiving the following messages:

Quote
File not found: www. site .com/'http: //www. link .com'     [no spaces obviously]
Refering page:
User agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

So there is no referring page... obviously because it is the google bot, but my question is this... Why is the google bot adding the external link to the site domain with single quotes around it? I presume that the bot sees <a href='...' >, with single quotes and for some reason is adding the link to the main site link and trying to see if that page exists. I have changed as manu links as possible to <a href="..." > now so that there are double quotes aroung the url to try and prevent this.

Anyone have any idea why this bot is behaving like this?   Huh


Last blog : Site of the Month - August 2007
I am a metal monkey!
Administrator
Community Supporter ?
Jedai Sword Master
*****
Gender: Male
Posts: 8102
41569 credits
Members referred : 3



« Reply #1 on: Apr 14, 2006, 11:35:09 AM »

This is very strange for G. I think that this might propably be a spam bot that uses the googlebot headers, or a bug of the googlebot 2.

But anyway if you don't have a link to ww. site .com/'http: //www. link .com'  in your site there is no problem.

Also have in mind that the gbot continues to visit non existent pages for a long time.

Trial and Error my two best teachers Cool
Join us @ facebook Visit through proxy

Last blog : Current Events + Big Sites = Easy Money
Global Moderator
Community Supporter ?
Jedai Sword Master
*****
Gender: Male
Posts: 6343
38878 credits
Members referred : 374


It's time to use PHP5!


« Reply #2 on: Apr 14, 2006, 03:25:09 PM »

Right, the Googlebit spiders your site without any referer information, check the corresponding ip address. I don't think that this one is from Google.

Olaf


Last blog : Upload images for usage in TinyMCE
Trackback URI for this entry : http://www.webdigity.com/trackback.php?topic=2264
Tags : google domains spam Bookmark this thread : Digg Del.icio.us Dzone more....

Topic sponsors:
Get a permanent link here for $1.99!


Pages: [1] Print 
Webdigity Webmaster Forums  >  Design and Layout  >  Web Page Design  >  HTML & XHTML (Moderator: Meth0d)
Topic: Links and robots
« previous next »
Jump to:
User Area
Welcome, Guest. Please login or register.
Did you miss your activation email?
Oct 07, 2008, 05:17:43 PM





Login with username, password and session length

Donate to our community, and get a permanent link back to your site!

Donate to our community, and get a permanent link back to your site!


Forum Statistics
Total Posts: 36.835
Total Topics: 7.546
Total Members: 4.121
Tutorials : 56
Resources : 143
Designs : 220
Latest Member: mani

27 Guests, 3 Users online :

24 users online today:



Readers

Web Design Gallery · Whois Lookup · Pagerank · Tag Browsing · Lo-fi version · Syndication · Webmaster forum history · Advertise
Developed by HumanWorks © 2005 - 2008 Webdigity webmaster community · sublime directory
Webdigity Webmaster Forums | Powered by SMF 1.0.12. © 2001-2005, Lewis Media. All Rights Reserved.