Googlebot: Proof
10:10 Tuesday 5 Jul 05
Googlebot has free rein here, just like other bots.
It has crawled ~20meg of data.
Go search for “codingvisions” – Tamba2 is #1
Go search for “freshlypressed” – Tamba2 is #1
Go search for “tamba2″ – this domain does not exist.
Both of those domains have a PR of 0, yet this blog has a PR of -
The links to those other two domains exist in only one place – here in the sidebar. Now if Google can crawl my site and put those two search results at #1, yet also deny that this domain exists that must surely show that my domain is blacklisted. I have asked Google for clarification (though I do not expect them to acknowledge anything and to use the bot as an excuse).Now … if they are taking my data but not using it, why should I let them ?
More: Google
Comments (2) --
Read (1535)
1
It’ll probably take at least a month or two after you allow them in robots.txt to end up back in search results.
13:26 Tuesday 5 Jul 05
2
And this is what I do not understand.
They have been in, and all they have used is two links. Now if those two links did not get #1 in Google last week but they do today, why is my site not listed anywhere else at all ?
If they want to take 20meg of data just to return 2 url’s, that seems just….. odd / wrong.
13:30 Tuesday 5 Jul 05