SearchRank Newsletter Issue # 16 – August 2005

Welcome to Issue # 16. Once again I am a bit late with this issue as it normally comes out on the first Tuesday of the month. I should just shoot for the first week of the month instead as things always seem to get in the way. At any rate I’ve got a great issue today with some SEO/SEM questions and answers as well some of the top search stories that occurred last month. The featured article this month entitled “Avoiding Search Engine Penalties or Getting Out From Under Them,” explores the issue of getting penalized or banned by a search engine and what to do if that happens to you. Let’s get to it.

In this issue…

• SEM/SEO QUESTIONS & ANSWERS
• AVOIDING SEARCH ENGINE PENALTIES OR GETTING OUT FROM UNDER THEM
• SEARCH ENGINE HEADLINES
• CLOSING COMMENTS

SEM/SEO QUESTIONS & ANSWERS

Here are a few questions that were asked on some of the forums I am privileged to moderate for which I was able to provide answers for.

301 Permanent Redirects

Q: We have a couple of thousand hotels listed on our site and available for booking. These pages have been online for a few months now and are listing nicely in Google. However we were unhappy with a few aspects of the design on these pages and so  have redesigned them. We have also renamed some of the files and organized them into different folders (before we had them all listed in the root and that was just unmanageable). If we remove the link to the old pages on our site and upload the new pages is there a chance we could receive a duplicate content penalty? So there would be no links to the old pages anymore, only links to the new ones. We don’t really want to have to remove the old pages altogether until the new ones are listed because we don’t want to lose any business while the new pages find their way into the Google index.

A: Just set up 301 redirects from the old pages to the new and Google will pick these up. You can then remove the old pages from your server. Users (and Google) will be redirected to the new pages. If on a Unix/Linux server, you can do this in your .htaccess file but if Windows, I think it needs to be accomplished through IIS. Not sure on Windows though cause I don’t host my stuff on Windows environments. We had a similar situation a couple of years ago where we moved a bunch of files in the root to specific directories and it worked beautifully with Google.

Are Backlinks the Determining Factor?

Q: Is it true that after getting on the first page of Google, it is the number of links that pretty much determines who gets the number 1 rank and which one gets number 10? For one key phrase my site fluctuates between 9 and 10 and the only difference I can see between the sites above me is the number of backward links they have. Have you come across sites that have fewer backlinks with better ranking, especially once the site makes it under rank 10 for a certain phrase?

A: The quality of the web site itself is also a determining factor – how well it has been optimized for prime placement in search engines, the content that exists on the site, its size, its age, etc. I have seen sites with virtually no backlinks place better then those with many backlinks simply because the sites were better optimized and of much better quality.

Disappeared From Yahoo – What To Do?

Q: My website disappeared from Yahoo’s radar about 3 weeks ago. I decided that a knee jerk reaction was not required and I have sat and patiently waited to no avail. The site is still not appearing on Yahoo despite showing quite a few links to it. The site is also still in DMOZ. Do you have any ideas or email addresses I can write to in order to rectify this with Yahoo?

A: The email I use to communicate to Yahoo regarding indexing problems or possible penalization or banning from their index is reportsearchspam@yahoo-inc.com. First you should investigate to see if you have done anything that violates their guidelines. It seems that Yahoo mostly does not like when one participates in link farms or links to others sites that have been banned or penalized, duplicate content and garbage pages that only exist to run AdSense on. Those are the main things I’d look for.

Flash Animation & Search Engine Crawlers

Q: Does flash animation hurt the ability to properly optimize a site for the search engines? I have heard that the flash intro can “stop a spider” in its tracks and keep it from properly spidering the site and I would love to know if you can have some flash and still have the site optimized.

A: As long as you have some kind of text link that allows a person or crawler to bypass the Flash intro, you should be okay. It is when you have an all Flash intro with no actual “a href” type of link to get beyond the intro that you will likely stop a crawler in its tracks. As far as doing an “all Flash” site, this you can do but you should provide an html version as well.

AVOIDING SEARCH ENGINE PENALTIES OR GETTING OUT FROM UNDER THEM

There is not a week that goes by that I do not come across someone posting on a forum stating that they have been penalized or even banned by Google, Yahoo or another search engine. Most claims do not have much merit. The individual just assumes they have been penalized because their site is not ranking well for search terms they think it should or they don’t know how to check if their site is indexed in a search engine. Even seeing a grayed out PageRank meter in the Google toolbar can lead people to assume the unthinkable – “I’ve been banned!” However there are times when a web site has indeed come under some kind of penalty or has been completely removed from a search index for one reason or another. In this article we will look at ways to avoid such penalties as well as ways to redeem yourself if you have experienced this misfortune.

Before we take a look at some of the things that can get a site penalized, I want to make the following statement: “Do not depend solely upon the search engines for your survival.” While search engines can be a great source of traffic, you cannot rely on them alone. They can be unpredictable beasts and you have no control over them. One month you could be generating great traffic to your site via the organic search results. Then the search engine changes the algorithm and your site drops out of site. So does your traffic and you find yourself hurting in the worst possible way. Many web site owners experienced this when Google made a major change to their algorithm back in November 2003, an update often referred to as “The Florida Update” due to the catastrophic effect it had on a very large number of websites. Some folks actually went belly up because of this.

Therefore seek other ways to attract potential customers whether that be through pay per click and sponsorship advertising on the Internet or traditional means of advertising such as print, radio, television and word of mouth. As the old adage goes, “Don’t throw all your eggs in one basket.”

A Look At Search Engine Penalties

Let’s talk about search engine penalties. What are they? Google, Yahoo and MSN have compiled their own lists of “do’s and don’ts” and I am sure other search engines have their own as well. However these merely serve as guidelines and may not include everything that can initiate a penalty from a search engine. Here are some of the most common things that can generate a search engine penalty:

• Cloaking (or Code Swapping) – the process of submitting one thing to a search engine and then displaying something else to the end user. Sometimes a web page is optimized for a particular search engine or phrase, and then the page or code is swapped when an end user clicks on a search listing. This is done to either prevent others from imitating the success of the page or simply to hide an ugly text-only type of a page for one that is more stylish.
• Doorway Pages – web pages that solely exist for the purpose of ranking well in the search engine results pages (SERPs). They are typically not part of the navigational
  structure of the web site and can only be found if someone happens upon them from a search query.
• Keyword Stuffing – involves the repeated use of a word or phrase in an attempt to increase a page’s relevancy.
• Invisible Text – a technique where webmasters will insert text, which is usually repetitive use of keywords, somewhere on a page (usually at the bottom) and make the color of the text the same as the background of the page or the background of a table or cell so as not to be seen by the average Internet user.
• Tiny Text – a method of hiding and stuffing words by setting them at a small font size such as 1pt or 2pt for example.
• Page Spoofing / Meta Refresh / Redirection -  a technique where a page is developed for a particular keyword phrase and then some form of redirection is used so users cannot see the content of the page, typically a bunch of senseless text. Anyone clicking through to this page from its listing in a search engine will be automatically taken to a different page. Often, the “final” page people reach has little content relating to the keyword phrase.
• Meta Tag Stuffing – where webmasters place high traffic keywords in meta tags that are completely unrelated to a web page in order to generate more traffic.
• Page Stuffing / Duplicate Pages – where the same web page will be duplicated or slightly modified and then these variations are all submitted to a search engine. They assume that if the page is successful for a particular keyword phrase, then all of its variations can dominate a search engines top listings.
• Domain Spamming / Mirror Sites – involves having the same exact web site at different domains. While there are some legitimate reasons to have “mirror” sites, operating such sites simply to increase search engine traffic is generally considered spamming.

While these are some of the most common reasons why people find themselves suffering under a search engine penalty, there are others. Let’s look at a few of them.

Hidden Links
This is typically done by using a small 1×1 transparent gif image and then hyperlinking it somewhere. I have experienced this penalty myself and learned a hard lesson by it. True confession ahead – back in the days when Google along with their PageRank method of ranking pages began to gain popularity and before they really came out against hidden links as well as other so-called “spam” techniques, we would link back to our site using hidden links on just about every web page we touched across a variety of domains. Why? To build up link popularity of course.

Then Google began to take a more active role in combating spam, particularly the type of spam that attempted to game their PageRank algorithm. What happened to us? We were banned from Google. At the time I didn’t care much because 1.) Google wasn’t the most popular engine at the time (seems a lifetime ago) and 2.) we did not depend solely upon search engines for business (as I recommend above).

But eventually as Google gained popularity, I became concerned. We needed the traffic Google could send us. What were we to do? We actually had to go and remove all those links (what a task that was), confess our fault to Google and beg for forgiveness. We were forgiven and eventually let back in the index but the whole process took about a year. One thing gained out of that experience is that we learned how to help others get back into Google’s and other search indices once they have been penalized or banned. We also learned of course never to do that again!

Linking To Other Sites That Are Under Penalty
I have seen this happen more with Yahoo than Google. A past client of ours linked their site to a series of other sites that Yahoo had already banned. In turn they also got banned – kicked right out of the index. All these sites were linking to one another in some sort of link farm strategy. We advised the client to remove the links which they did. After this, the penalty was lifted.

Let me clarify this issue of linking to bad sites. You will never come under a penalty if a bad site links to you. Search engines understand that you do not have any control over someone else linking to you. However when you link to what a search engine considers a “bad” site, that you do have control over and while it will not always generate a penalty, it can if it is abused as was the case in the above example.

Generating Pages For the Sole Purpose of AdSense
In case you do not know, AdSense is a program Google developed for web site owners to allow Google AdWords to appear on their sites and earn a percentage of the revenue. AdSense has taken many quality and informative sites that were formerly un-profitable and made them profitable. It earns Google profit as well because they have a greater number of outlets to display their ads.

The problem with this however is that some people are abusing the privilege of being an AdSense publisher in that they are generating hundreds and sometimes even thousands of useless web pages whose sole purpose is to display AdWords in hope they will generate revenue.

Yahoo has begun to penalize or even ban sites that practice this. I have seen this occur on more than one occasion now. In fact, the client I mentioned in the example above was doing this which is part of the reason why they were penalized. Once the pages were remove, so was the penalty.

Additional Penalty Trippers
Are there other things you can do to get in trouble with a search engine? Absolutely! We have really only covered the most common ways as well as some of my own experiences. The basic rule of thumb is to build and maintain your site with your users in mind. Create the best experience for them and display your products, services or information in the best manner that will convert visitors into paying customers. All of the methods described above are intended for search engines only – not end users.

Does that mean I cannot optimize my web site? Certainly not, but a web site can be aggressively marketing via site optimization, link building, usability, etc. without resorting to the tactics mentioned above. While some of the techniques mentioned above may work for a season, there is always that chance that you will get caught and suffer because of it. You have to then ask yourself, “Is it worth the risk?” Is it worth losing valuable search engine traffic? Is it worth destroying your brand? Is it worth the time and effort it will take to reverse the penalty? In all cases I say, “No, no, no!”

How To Get Our From Under a Search Engine Penalty
What do you do if you have been penalized or even banned? There are two things you can do. One is to get a new domain and start all over but without employing the same tactics that got you penalized in the first place. That is one way but certainly not the best in my opinion because often times people have built a brand name or put extensive time into developing and marketing their sites.

First of all find out what you did to earn the penalty. Most people already know this but other times a web site owner may be using an SEO/SEM to optimize or market their sites and as such may not understand what has been done to earn them a penalty. Unfortunately thousands of customers of a large search engine optimization company called Traffic Power were affected by a Goggle search engine penalty last year. Most were completely unaware of why they had been banned from Google’s index. If you are unsure as to whether the company you hired to market your site has done something amiss, confront them directly as to what they might have done or get a second opinion from another SEM company.

Once you discover the tactic or tactics that were used, you need to remove them. If it is tons of useless doorway pages then you must delete them. If you are hiding text then it must be removed. Linking to bad sites? Get rid of the links. Whatever the violation, it has to be remedied. It is only then that you will have the opportunity the come out from under the penalty. Sometimes this can happen automatically because the penalty was simply a spam filter the engine had built into in their algorithm. Other times you have to contact the engine, admit what you did, detail the steps you have taken to reverse the violation and ask for forgiveness. After all that is done, you wait patiently. It could take as little as a month or as in my own personal experience, a full year.

Who do you contact? Contact Google help@google.com, Yahoo at reportsearchspam@yahoo-inc.com and MSN at msnbot@microsoft.com.

If you find that you have been victim of a search engine penalty, it is not the end of the world. It is possible to come out from under such a penalty. Contrary to popular belief, it is not the normal practice of search engines to penalize or ban sites but at the same time their goal is to present the most relevant search results to their users. Sometimes that means removing sites that try to “game” the system.

Written by David Wallace

SEARCH ENGINE HEADLINES

Here are some of the latest headlines related to the search engine industry.

MyJeeves 1.2 Released
If you’re looking for something to demo this weekend, Ask Jeeves has just released version 1.2 of their MyJeeves personal information management tool.

Google Simplifies & Loosens Requirements for AdWords
Google is making changes to its AdWords program intended to provide advertisers with more control over campaigns while potentially improving relevance for users.

The God of Search (and I Don’t Mean Google)
Looking for God? Google just might be able to point you in the right direction, and not just using their new local search listings.

Search Branding: Loving the Clicks That Don’t Convert
The opportunity to do branding via search creates a great opportunity to think a bit deeper about the way search is planned, bought, and measured.

CLOSING COMMENTS

That’s all I have for today. I’ll be at Search Engine Strategies in San Jose next week hanging out with all my search engine marketing buddies and hopefully learning some new things. There will be no lack of parties that go on after the conference. There’s the 4th Annual Google Dance held at Google Headquarters and then Yahoo is throwing a party they call “Search Night Off” which will be held at Paramount’s Great America. Free food, cocktails and roller coaters – yeah! Those are just two of many parties that will be held. The best part of these conferences for me is seeing all the people I know and communicate with online in person as well as meeting new folks in the industry.

After the conference it’s up to San Francisco for a little R&R. I’ll tell you how it all went in next month’s newsletter and will hopefully have a new set of articles ready from what I pick up at the conference. See you next month!