Sublime directory Surf the web anonymous Pagerank Monitor


getting Alexa information for free

olaf
Tue 27 June 2006, 07:32 am GMT +0200
Hello,

I need to retrieve some data from the alexa page on my own website, its about this data (for example):
http://www.alexa.com/site/site_stats/signup?site_url=www.webdigity.com&widget=s&style=a&submitted=true&mode=stats&amzn_id=

In know its possible to use their webservices but they are not free (your have to oay if you reach a  number of requests).

do you know a fast way to retrieve them a different, legal way? (using php)

Nikolas
Tue 27 June 2006, 10:37 am GMT +0200
First, I don't think it is legal, but I guess if you want to do this anyway, you should grab the content of this page :

http://www.alexa.com/data/details/main?q=&url=webdigity.com

Check the code of that page, I think it is easy to grab the ranking from there.

olaf
Tue 27 June 2006, 11:06 am GMT +0200

Quote
First, I don't think it is legal, but I guess if you want to do this anyway, you should grab the content of this page :
You're right this is my next problem, but I'm affraid the the free impressions are nit enough...


Quote
Check the code of that page, I think it is easy to grab the ranking from there.
I thought about that, but it takes to much loading time. (keep in mind that the alexa site is not so fast)

Nikolas
Tue 27 June 2006, 11:16 am GMT +0200
You can't deal with the loading time. Of course if you will use CURL it will be much faster, but you can't do something for crawling faster the site

olaf
Tue 27 June 2006, 11:25 am GMT +0200
You can't deal with the loading time. Of course if you will use CURL it will be much faster, but you can't do something for crawling faster the site
the fastest way is using their web services, but 10.000 free requests are not so much...
100.000 addtional requests are 15USD

Nikolas
Tue 27 June 2006, 12:39 pm GMT +0200
Then I guess there is no other solution than using their site.

olaf
Tue 27 June 2006, 12:41 pm GMT +0200
Then I guess there is no other solution than using their site.
it looks like, maybe I have to cache the requests inside a database...

Nikolas
Tue 27 June 2006, 12:47 pm GMT +0200
Quote
maybe I have to cache the requests inside a database...

That's a must, but have in mind that this can produce overhead as you will have many records with varchar keys

olaf
Tue 27 June 2006, 12:48 pm GMT +0200
Quote
maybe I have to cache the requests inside a database...

That's a must, but have in mind that this can produce overhead as you will have many records with varchar keys

is there an other way? I think I have to index the the web addresses to make it quick

Nikolas
Tue 27 June 2006, 12:53 pm GMT +0200
You can create a table like this :

domain
tld
rank

Example row :

webdigity
com
44.000

Of course the two first rows will be primary key, so the indexing will be much faster, and it will be able to handle many thousands records with a light server.

olaf
Tue 27 June 2006, 12:57 pm GMT +0200
You can create a table like this :

domain
tld
rank

Example row :

webdigity
com
44.000

Of course the two first rows will be primary key, so the indexing will be much faster, and it will be able to handle many thousands records with a light server.
yes this is what i thought, but how about a table with more then 100.000 records?

Nikolas
Tue 27 June 2006, 01:03 pm GMT +0200
Big tables will work much better with mysql 5, but anyway don't be afraid, as mysql is very well optimized.

BTW what are you going to with this?

olaf
Tue 27 June 2006, 01:05 pm GMT +0200
Big tables will work much better with mysql 5, but anyway don't be afraid, as mysql is very well optimized.

BTW what are you going to with this?
I created this Google module to promote my thumbnail service (soon):
http://fusion.google.com/add?moduleurl=http%3A//www.finalwebsites.com/webpage_information.xml

Nikolas
Tue 27 June 2006, 01:08 pm GMT +0200
BTW I just added the module, and I see that it does not accepts urls without http://

Maybe you should make it work without it too.

olaf
Tue 27 June 2006, 01:10 pm GMT +0200
BTW I just added the module, and I see that it does not accepts urls without http://

Maybe you should make it work without it too.
yes it's a first version, but you are getting an error message ;)

Archive for SMF v1.00 by N.P. Valid XHTML 1.0 Transitional