Comments on: Google: One week in

By: Mark

Mark — Mon, 11 Jul 2005 13:32:55 +0000

I have mailed Google numerous times.
They assure me I am not banned or penalised – but they would do that.

Google have had every possible opportunity to address my concerns – they are actively choosing not to.

It would be a pleasure to post that I had got it wrong and i was back in G’s search results. Somehow though, this won’t happen.

By: Angsuman Chakraborty

Angsuman Chakraborty — Mon, 11 Jul 2005 09:40:52 +0000

Follow up with Google on this. I had contacted them a couple of times. It appears they do take action to redress your issues. My issues were addressed.
Ask them directly why you have been banned. Reply to their automated response with detailed information. Point them to your blog entries too.

By: MacManX.com » Blogroll Dive: 7/11/05

MacManX.com » Blogroll Dive: 7/11/05 — Mon, 11 Jul 2005 07:03:31 +0000

[...] Bryan is forced to clean install Windows XP after attempting to use a Belkin Wireless G USB Network Adapter. Owen compares MovableType v3.2 to WordPress v1.5. Michael releases K2 (Kubrick v2) to the public as an alpha release. Brian is back with a new addition to his family. Where are those baby pictures? Khaled ruminates on the recent incident in London. Craig has returned, and he is finally feeling “normal”. Orson discusses “stretching” your brain through video games. Angsuman asks, “Is PHP secure?” Mark shares his feelings on Live 8. Tom discovers that Microsoft may be embracing web standards. And, Mark has a visit from the site-stealing Googlebot. [...]

By: Mark

Mark — Sat, 09 Jul 2005 17:13:03 +0000

Andy – that is a very wierd result.. and one I don’t understand right now.

By: Andy Skelton

Andy Skelton — Sat, 09 Jul 2005 17:09:28 +0000

I searched Google for a fairly unique string on your footer ("header graphic me") and it produced one result. Why on earth would Google spider your site via this proxy?

By: Mark

Mark — Sat, 09 Jul 2005 16:29:21 +0000

If Else – they have a month. What Google have done is added two urls to their database which are listed here and only here, so given that they will list a url which has only one other website pointing to it, I cannot see why my site is not listed unless it is banned.
They tell me in an email that:

Although our robots may visit your site, we cannot guarantee that your pages will be thoroughly crawled or indexed. For site indexing and ranking recommendations, please visit http://www.google.com/webmasters/guidelines.html

I did, and submitted a sitemap. The domain is xhtml valid. It is linked.
That “cannot guarantee” is their get-out clause.

What bugs me is NOT the fact they will not list me, but that they take my data anyway. THAT is the issue here, not my PR / linkage / inclusion or whatever. If they will not list me, why crawl me ? Their bots are not stupid.
It’s taken as a deal I guess … they crawl and display your data under what they would term “fair use” and for that, you get traffic. You “spend” bandwidth on the bot to get traffic. So if I’m “spending”, what do I get ?

By: If Else

If Else — Sat, 09 Jul 2005 13:15:51 +0000

When your page is spidered, it doesn’t immediately enter the google listings. I’d be patient and give it some time before jumping to conclusions.

By: Mark

Mark — Sat, 09 Jul 2005 08:31:56 +0000

I actually do not give a damn if I am banned – I am perfectly aware of why this happened and I will use that header image permanently if I do not get a reasonable answer.

What annoys me (I am being SO polite this morning) is what I see as the pointless theft of my data. I can afford the bandwidth, but it’s a principle here. What is also annoying – and this should disturb others – is that I cannot see how I can exclude Googlebot yet allow others in – they do NOT obey the standard and write the bot to act in a way to be sure they get what others can. that is insidious behaviour.

By: gpshewan

gpshewan — Fri, 08 Jul 2005 23:54:03 +0000

At a guess I’d say googlebot finds your site because it is heaviy linked by other people. The crawler searches around your site and pulls what information it finds and then dumps it for submission to the engine … but it looks like you’re banned. So when it comes to submitting all the information that googlebot found to the directory it all just get’s dumped.

I’d say ban checking isn’t part of googlbots remit. So it just trawls sites for information as a drone. Only when it comes to loading up the directory does the ban check come into play and your site ends up in the proverbial bin.

It may sound harsh, but if you’ve ended up on Googles ban list they aren’t going to give two hoots about your feelings.

I’ve got similar problems with Yahoo as well … but it’s probably down to an htaccess or robots issue – even though I’m sure I have it rght … just can’t be bothered to become a htaccess/robots guru right now to resolve it.

By: Bryan

Bryan — Fri, 08 Jul 2005 23:53:18 +0000

That kinda ticks me off…what’s going on? I’d like to know, that’s for sure.