Nikolas
Tue 6 June 2006, 10:43 pm GMT +0200
I just saw the who is online list, and we have 40 guests from the same ip range (38.98.19.XX) with user agent Snapbot/1.0
Does anyone knows this bot?
Does anyone knows this bot?
olaf
Wed 7 June 2006, 08:49 am GMT +0200
I have the visitors too,
it's a new search engine: http://www.snap.com/
We reviewed the site here at this coomunity...
it's a new search engine: http://www.snap.com/
We reviewed the site here at this coomunity...
Nikolas
Wed 7 June 2006, 11:04 am GMT +0200
I thought that it might be from snap.com, but I am not really sure.
This is too aggresive to be a legit bot (yesterday we got up to 50 visitors from them at the same time)
This is too aggresive to be a legit bot (yesterday we got up to 50 visitors from them at the same time)
vbignacio
Wed 7 June 2006, 11:36 am GMT +0200
"This is too aggresive to be a legit bot..."
why? do you suspect that this bot is up to something fishy? like what for instance?
why? do you suspect that this bot is up to something fishy? like what for instance?
Nikolas
Wed 7 June 2006, 11:39 am GMT +0200
It could be a spam bot like those who fetch email adresses.
olaf
Wed 7 June 2006, 11:58 am GMT +0200
I have regular visits from snapbot, max 10 a day and the IP address range is 66.234.139.xxx
Nikolas
Wed 7 June 2006, 12:06 pm GMT +0200
Maybe I should ask the snap to see this thread and tell us what they think
olaf
Wed 7 June 2006, 01:58 pm GMT +0200
Maybe I should ask the snap to see this thread and tell us what they think
sounds like a good ideaNikolas
Wed 7 June 2006, 02:01 pm GMT +0200
I have email the company.
I will let you know if they reply my email.
I will let you know if they reply my email.
Nikolas
Wed 7 June 2006, 06:01 pm GMT +0200
1st reply :
Now I've asked them regarding the ip ranges that we saw. I will keep you posted on this :)
BTW those folks are very good. They replied in only 4 hours.
Quote
Nick,
Yes, that is our bot. It is not malicious. It is just designed to build
an index of the Web for Snap.com.
Thanks for checking in with us and please get back to us if you have any
more questions.
Brad
Snap Customer Service
Yes, that is our bot. It is not malicious. It is just designed to build
an index of the Web for Snap.com.
Thanks for checking in with us and please get back to us if you have any
more questions.
Brad
Snap Customer Service
Now I've asked them regarding the ip ranges that we saw. I will keep you posted on this :)
BTW those folks are very good. They replied in only 4 hours.
Nikolas
Wed 7 June 2006, 07:58 pm GMT +0200
Here is the second letter (too fast again :) )
Quote
Nick,
Okay, I have some answers for you.
Yes, those are our IPs you are seeing.
The cause of the "aggressiveness" of our crawling is because we are
using multiple servers to do crawling, and while each server has an
adequate delay between requests, at times they may - purely by chance -
get in the same rhythm, and cause multiple requests in a very small time
frame.
We recognize this is a problem and are working on a fix for it right now.
I apologize for the inconvenience. It really is not intended to be
malicious, and we're working to improve our crawler so it does not cause
this problem.
Thanks much for writing, and please write back if you have further
questions.
Brad
Snap Customer Service
Okay, I have some answers for you.
Yes, those are our IPs you are seeing.
The cause of the "aggressiveness" of our crawling is because we are
using multiple servers to do crawling, and while each server has an
adequate delay between requests, at times they may - purely by chance -
get in the same rhythm, and cause multiple requests in a very small time
frame.
We recognize this is a problem and are working on a fix for it right now.
I apologize for the inconvenience. It really is not intended to be
malicious, and we're working to improve our crawler so it does not cause
this problem.
Thanks much for writing, and please write back if you have further
questions.
Brad
Snap Customer Service
olaf
Wed 7 June 2006, 08:56 pm GMT +0200
good working people (I think Google did the same in the beginning...)
Nikolas
Wed 21 June 2006, 05:33 pm GMT +0200
Forgot to post you the last email I got which is this :
Quote
Nick,
I?ve been keeping an eye on important incoming user feedback and saw your e-mails, and then visited your forum.
We appreciate your members? dialog while our ?young? indexing systems get their kinks worked out. Your forum members are obviously professionals in this area, and we?re happy to see that even though we may be hitting some site(s) harder than we could be periodically, your community is aware of Snap.com and willing to give our bots the benefit of the doubt for now. As Brad stated, we are aware of this issue and working to resolve it (timeline currently unknown).
I wanted to share with you news about a couple of things you might be interested in. First, we?re working to build a URL submission system where anyone can submit URLs of their web pages, for free, into the Snap Index. This should be live in about 4-6 weeks. I?ll make a personal note to notify you when this goes live (unless you tell me otherwise).
Secondly, as some of your members appear to be familiar with Snap, you might tell them about our ?Launch Snap? contest we?re running right now at http://blog.snap.com . Given that we just launched a major update to the search engine in May, we?re looking for winning ideas on how to promote it ? without the resources and budget of the ?big guys.? The contest only has a few days left, but we?re giving away $5,000 to the winner (and $5,000 to the referring blog, if applicable), and it might be a worthy cause for someone you know.
I hope my e-mail isn?t unwelcome; I?m not trying to advertise Snap so much as I?m thanking you for administering a friendly and fair forum that?s willing to give Snap, the new kid on the block (and a small company) the fighting chance that we think it deserves.
-Brian
Project Manger, Snap.com
I?ve been keeping an eye on important incoming user feedback and saw your e-mails, and then visited your forum.
We appreciate your members? dialog while our ?young? indexing systems get their kinks worked out. Your forum members are obviously professionals in this area, and we?re happy to see that even though we may be hitting some site(s) harder than we could be periodically, your community is aware of Snap.com and willing to give our bots the benefit of the doubt for now. As Brad stated, we are aware of this issue and working to resolve it (timeline currently unknown).
I wanted to share with you news about a couple of things you might be interested in. First, we?re working to build a URL submission system where anyone can submit URLs of their web pages, for free, into the Snap Index. This should be live in about 4-6 weeks. I?ll make a personal note to notify you when this goes live (unless you tell me otherwise).
Secondly, as some of your members appear to be familiar with Snap, you might tell them about our ?Launch Snap? contest we?re running right now at http://blog.snap.com . Given that we just launched a major update to the search engine in May, we?re looking for winning ideas on how to promote it ? without the resources and budget of the ?big guys.? The contest only has a few days left, but we?re giving away $5,000 to the winner (and $5,000 to the referring blog, if applicable), and it might be a worthy cause for someone you know.
I hope my e-mail isn?t unwelcome; I?m not trying to advertise Snap so much as I?m thanking you for administering a friendly and fair forum that?s willing to give Snap, the new kid on the block (and a small company) the fighting chance that we think it deserves.
-Brian
Project Manger, Snap.com
olaf
Wed 21 June 2006, 09:26 pm GMT +0200
hm... this contest sound like the campaign from the "Echolist Directory" a year ago. I don't wanna compare both with each other but I didn't like this way of marketing. Because I was not a winner I got nothing. Echolist is getting a lots of links with result no one is using the site and these people getting no visitors from them.
I support at the moment the swickis's from eurekster, why ? because I get lots of visits from them and the swicki pages are very well listed at Google.
I support at the moment the swickis's from eurekster, why ? because I get lots of visits from them and the swicki pages are very well listed at Google.
YMC
Wed 21 June 2006, 10:53 pm GMT +0200
I support at the moment the swickis's from eurekster, why ? because I get lots of visits from them and the swicki pages are very well listed at Google.
What is that? Is it something mainly for European sites?
olaf
Wed 21 June 2006, 11:02 pm GMT +0200
What is that? Is it something mainly for European sites?
it's a US company, check the swicki inside my signature...