Web Apps Since 2004.

Is your website boring?  Kick it up a notch.  Encodable apps easily drop into your website to make it modern, fun, and interactive, with features like file uploads, user accounts, paid subscriptions, protected pages, live chat, visitor logging, mailing lists, and more.
All apps include:
• Money-back guarantee
• No monthly fees
• Free tech support
• Easy setup (we can even do it for you!)

Live Search Referrer Spam

# Filed on Dec 3, 2007 by AnthonyDiSante 2 replies

Microsoft’s Live Search is really starting to irritate me.

As this log snippet from VisitorLog shows, I get about 30 separate hits per day from hosts named livebot-65-55-*-*.search.live.com.  The vast majority of them are bots, not real humans, as evidenced by the fact that they have no screen resolution (and therefore no screen), which while not a guarantee of botness, is a pretty strong sign of it, especially when combined with other bot-like characteristics such as having "livebot" in the hostname.

So far this is all OK.  However, the bot’s USER_AGENT string is set to IE7/Win2003, which is bogus [the full string is: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.2; .NET CLR 1.1.4322)].  It’s clearly a bot, and possibly a spider, so it’s not a real IE7 browser; it should identify itself with an accurate user-agent string like a responsible internet citizen.

And what’s worse is that, unlike most bots/spiders, this one actually has a non-null HTTP_REFERER string: it claims to have come from Microsoft’s Live Search engine, searching for extremely generic single terms like "files" for which my site isn’t even in the first 10 pages of search results.  The only logical conclusion I see here is that Microsoft is doing some serious referer spamming to get hits back to its Live Search pages.

This has been going on since November 21st, but for months before that, the exact same thing had been happening except that the hostname was bl2sch1082113.phx.gbl (or similar) instead of livebot-65-55-*-*.search.live.com.

Now don’t get me wrong: spiders are good.  I like spiders crawling my sites, and I’d really like for Google to have some strong competition in the search space.  But faking your user-agent and spoofing the referer field with bogus data aren’t good practices for a search engine.  Someone please tell me there is a valid explanation for this.

Comments:

01. Jun 26, 2008 at 01:50am by AnthonyDiSante:

Six months later and Microsoft is still using this scummy technique.  Now they’re also using hostnames of the form msnbot-65-55-211-113.search.msn.com.

02. Jul 7, 2008 at 02:13pm by Malik:

Yeah from 2 months they are doing this in every day...
They are using lot of generic keywords about your site.

Reply to this message here:

Your name
Email (why?)
Website (if you have one)
Subject
search posts:

home | archives ]

Shopping Cart

Your cart is empty.

Client Quotes

Thank you VERY much for all of your help.  You've really impressed me.  We have support agreements for other software that costs thousands of dollars / year (just for the support), and most of them aren't as helpful as you have been.
– Keith Y.
Our members think your software is fantastic...  I would recommend your software and your company to anyone.  Thanks for all your help.  It has been a pleasure dealing with you.
– Tommy A.
Thanks again for a great product and great support - beyond expectations.
– Greg S.
Do you know how rare it is to have a "canned" shopping cart that can easily do complex pricing options on a single item?  Basically, they don't exist!  I have looked.  Everywhere!  And the few that might even come close to CornerStore's functionality cost a fortune!
– Tashina P.
FileChucker is helping drive the backend of several high profile entertainment sites for people like Shania Twain and Dolly Parton.  We're also using it to drive backend file uploads for a multi-billion dollar banking institution.  It's a great product.  We've tried other "chucking" upload solutions with progress bars using flash and php, but nothing works as reliably as FileChucker.
– Michael W.
The work, the thought and the organization you put into this app is incredible.
– Bruce C.
The amount of customization in the program is incredible.  I was able to integrate it into my existing page layout relatively simply.  I was also able to easily customize the look/feel to match the current site.
– Jason M.
Nice script, it's saving the day on our project.
– Aaron W.
Why didn't I just do this from the get-go?  So much easier.  Thanks for your work.  FileChucker makes my work easier.
– Dominic M.
FileChucker is a great drop-in solution for file uploads, and worth every penny of its very reasonable cost.  Encodable's support is excellent to boot.
– Loren A.
I just want to say you guys really stand alone in that you have a quality product and you provide genuine customer service.  It's sad but those qualities are seldom found separately, much less together.  Thanks again for your time and help.
– Alex S.
I just wanted to say that yours is the first product that I've tested so far that hasn't failed on handling uploads.  This is going to work for a print company, so they are handling nothing but large files and all the other solutions I've tried so far have not been reliable.  So far yours has been 100% successful in my tests.
– Kevin H.
Just one word: Fantastic.  10-minute job to plug FileChucker into my app, and it now works a treat.  It's through the hard work by people like yourselves that make my job so much easier.  Congratulations on an outstanding product... Many many thanks.
– Sean F.
FileChucker is working great...  Clients love it.  Vendors love it.  We love it.
– Gerry W.
I just installed the demo of your product and got it up and running in no time.  I searched high and low for a decent login script and thank God I found yours.
– Adrian F.
I looked all over trying to find a simple cgi script.  I found that FileChucker was by far the best.  If you have issues with your hosting service's php.ini max upload size then this is the way to go.  Looking forward to future enhancements.
– Bob C.
You've done a wonderful job with FileChucker and UserBase, and they have made a big difference to how our website runs.
– Nicholas H.