Wednesday, May 18, 2005

Search Engine Optimization Research Findings: A Client Perspective by Jimsun Lui



This research attempts to obtain business firms' views on search engine marketing. It gives valuable implications to both business and search engine marketing firms.

The research collects findings from 121 Hong Kong trading firms involving in foreign trade through telephone interview. Also, a separate focus group was held with 8 marketing managers and 1 search engine optimization (SEO) specialist. We believe the implications also apply to other geographical areas.

We have extracted part of the results to present in this article.

1. Employed SEO service or Exercised SEO Before?

103 (85%) respondents reported that they have neither employed any search engine optimization service nor self-exercised any search engine optimization service. However, 18 (15%) respondents pursued some sort of search engine marketing including search engine submission to search engine optimization.

2. Importance and Concerns of Search Engine Marketing/Search Engine Optimization

A rating scale of 1 to 5 was employed, with 5 means Very important and 1 means very unimportant. Respondents reported that they perceived that search engine marketing could be a cost-effective method to help them get new overseas customers (rating 4.7).

Respondents also considered the following factors whether they would use a search engine marketing firm: 1. Pricing (4.6), 2. Deliverables (4.8), 3. Guarantee Clause (4.0), 4. Ranking against rival firms (4.7), 5. Possibility to get to Top 10 ranking (4.1), 6. Chance to increase response rate (4.8)

3. Paid for Performance Search Engine Marketing

This question has an explanation about the meaning of paid for performance search engine marketing. The question includes pay-per-click and paid-for-inclusion plan.

Again, a rating scale of 1-5 was employed, with 5 means Very important and 1 means very unimportant.

The respondents expressed that they concerned the possible high cost (4.8), potential fraudulent clicks (4.9), delivery of message to right persons (4.7), and time required to manage the campaign (4.2).

4. Understanding of Search Engine Optimization

This question asked about their knowledge of search engine optimization techniques. Respondents were given 3 choices: 1. Yes, 2. No, 3. Don't know.

31 respondents perceived that the major task of search engine optimization was adding keywords in Title and Meta Tags while another 62 of them were not sure about the answers. 108 respondents did not know link building was an important factor that could help them improve search engine ranking. Interestingly, 72 respondents believed that program submit to 1000 search engines could increase their website traffic. 101 respondents did not realize that content analysis and improvement measures such as keyword density ratio and keyword proximity ratio played a role in search engine optimization.

In general, most respondents believed that they did not possess sufficient knowledge on search engine optimization.

5. Focus Group

Another 8 marketing managers were invited to a focus group interview. 4 of them had experience in employing search engine optimization firms, and another 4 considered using search engine optimization services but finally had not taken action. An SEO specialist was acted as a moderator in the interview.

In the interview, they generally agreed that search engine marketing could help them to generate business leads. For those who did not use SEO service expressed that they abandoned to pursue as they could not make up a decision. First, they were not familiarize with how to achieve good search engine ranking. Second, the pricing scheme offered by different SEO firms varied greatly from US$19.99 to US$5,000 up-front fee, and some also required US$500 to US$5,000 monthly service fee. However, the SEO firms could not express clearly the deliverables that clients could obtain and they also refused to clearly explain the methodologies that they would use. These facts made the interviewees very difficult to persuade their boss to use SEO service.

In addition, the interviewees expressed that they feared that SEO firms would require many website changes, and some changes like keywords stuffing in Alt Tag or put too many texts and too little graphics in web pages make the site looked unprofessional.

Another point made by interviewees was very interesting. For those who have employed SEO firms, some of them perceived that they were being kept in the dark as they did not know what the SEO firms were actually doing! When they scrutinized the changes made by SEO firms, they found that the SEO firms have just written some keyword phrases in Alt Tag, Title Tag and repeated them in Meta Tags. Then, the SEO firms only ran reports for them in the subsequent months and asked them to wait for the results. Finally, they found that they had not got any extra traffic from keyword phrases suggested by SEO firms while the SEO firms claimed that they had achieved more than what was written in the guarantee clause (e.g. 20 Top 20 rankings in major search engines). Only 1 interviewee felt satisfied to the chosen SEO firm. This was because they knew the deliverable, got desirable number of top rankings, and a piece of link exchange software called Automate Link Exchange free and they could continuously manage link building themselves after the project end. The SEO firm also provided suggestions for them to maintain ranking afterwards.

Also, participants voiced out that they also wanted to know their search engine rankings against their rival firms.

6. Implications to Business Firms

In general, it was a consensus that search engine marketing could help to generate new business leads. However, it was not a simple activity. Most business firms lacked the knowledge and tempted to use an SEO firm. However, quality of SEO firms varied. In order to select a good SEO service, business firms should define clearly their objectives. E.g. some firms might simply define that they wanted a better ranking than their rivals or that they wanted a better conversion rate. Then, it was suggested that the business firms talk with the SEO candidates and try to differentiate which one was more knowledgeable. Moreover, business firms should discuss with the selected SEO firm in order to get an agreement on the number of deliverables and project schedule.

From the research findings, many business firms were yet to optimize their websites. That means if you first optimize your website, you could have an advantage over your rivals in search engine exposure perspective.

7. Implications to SEO firms

SEO firms should spend time and clearly explain to clients the SEO techniques and methodologies that they would use. Some SEO firms might fear that they would expose their top secret techniques. However, you could not get business if your clients could not understand the SEO processes and failed to persuade their bosses. In addition, SEO firms should apply optimization techniques with a consideration of usability of websites. Do not make the site look silly or unprofessional. Also, clients would not mind your using of optimization software such as Webposition Gold and they do want to know how you would serve them.

SEO firms should assist clients to devise a measurement matrix for the campaign. This helps the decision maker to make judgment and explain to his/her colleagues and boss. Clients would feel more comfortable to do business with you.

Moreover, SEO firms should define clearly what would be delivered to clients and the delivery schedule, e.g. how many ranking reports, keyword research result, etc. In addition, SEO firms should contact clients frequently and report the progress. Clients would then understand that you were indeed working for them and you could help to set or adjust their expectations.

Finally, SEO firms should offer assistance for clients who want to terminate the relationship and to maintain rankings themselves. The idea looked ridiculous but the clients would be satisfied by such offer and would probably recommend you to other firms. After all, clients who ceased to work with you might be due to many other reasons but not your performance. Why don't you give them another candy and maintain a good relationship?

About the Author
Written by Jimsun, from Agog Digital Marketing Strategy Limited, provides professional Search Engine Optimization Positioning Service with an aim to increase website traffic and revenue. Visit http://www.agogdigital.com to learn more.

Search Engines: Different Types, Different Strategies by Terry Nicholls



There are four basic types of Search Engines:


Free Search Engines



Pay-For-Inclusion Search Engines



Pay-Per-Click (PPC) Search Engines



Directories


Because each type does things a little differently, you need to adapt your strategy to take advantage of their differences.

Free Search Engines

You can submit your pages to these engines free, but be careful. You must make sure not to over-submit (submit too often) or you'll be banned and never get listed.

Always check to see if your site is listed before submitting it.

Pay-For-Inclusion Search Engines

With this type of Search Engine, you pay to have your web site listed in their database. Pay-for-inclusion Search Engines (and the paid section of free engines) are a quick way to get listed in some major databases -- for a price, literally. The cost varies from engine to engine.

The advantages are threefold:


Faster inclusion into the Search Engine's index.



Repeated, regular spiderings.



Guaranteed continuous inclusion.


Pay-Per-Click (PPC) Search Engines

Pay-per-click Search Engines allow you to bid for keyword placement. For example, if one of your pages focuses on the topic of "fashion models," you can bid for the #1 (or any other number) placement on the first page of search results. You only pay when someone actually clicks on your ad.

There's a catch, of course. The most popular keywords have become quite expensive at Overture.com (the first and biggest PPC engine) and are rising at the others.

Directories

Directories are different from Search Engines in that they do not spider pages. Humans review each submission, visit each site, and decide what gets in.

Search engines and directories provide search results for each other. If a search turns up nothing in the directory's database of sites, it will show the search results from one of the spidered engines. All the directories use one of the major engines.

The reverse is also true. Most Search Engines also provide directory results, in addition to their own search results. All of them use one of the "Big 3" -- Yahoo!, Open Directory, or LookSmart.

Adapt Or Disappear

The difference between the types of Search Engines requires that you adapt your strategy to take maximum advantage of each engine. We'll help you with that.

For a more detailed explanation of these Search Engines, along with specific strategies and mistakes to avoid, please visit My Home-Based Business Advisor .

Terry Nicholls
My Home-Based Business Advisor
www.my-home-based-business-advisor.com

Copyrightby Terry Nicholls. All Rights Reserved.

About the Author
Terry Nicholls writes from his own experiences in trying to start his own home-based business. To benefit from his success, visit My Home-Based Business Advisor - Helping YOUR Home Business Start and Succeed for free help for YOUR home business, including ideas, startup, and expansion advice.

Creating Robots.txt File and its Importance by San Christopher



If you are thinking you have developed a truly great keyword-rich-unique-content fully optimized website for the search engines and an attracting site for the visitors - that's fine, but do you know you are missing something? A robots.txt file. Did you include it? By the way do you know what's the importance of a robots.txt file?

Success of big companies lies in keeping their confidential data a secret, hidden from all. They tell the world something and do something. This enables them to execute their future course of action easily and change plans according to the situation. Job of robots.txt file is the same. It can or cannot allow a search engine to visit some or all of your web pages. Of course a human visitor is free to visit these pages. That being the case, for the search engines your website may be different than what a visitor is seeing. If you think one or some of the pages/files aren't good enough to be visited by a particular search engine or engines you can do it. Although this is not recommended - your website should be made in such a way it should not shy away from the search engines. Nevertheless its always better to know the basics of writing robots.txt file. It will help you. We will discuss farther down - robots.txt file is important. I repeat again - don't make pages you think should be hidden from the search engines. If any search engine think you are up to some tricks, it may panelize your site causing a no-rank - in the worst case for ever!

Every search engine has a "robot" (a software program) that does the job of visiting a website. Their purpose is to "know" the website, what it is all about, gather all information about it etc. Search engine robots gather this information and bring them back to their databases to show them in their search results. So, if your site is not there in their database it never shows up in the search results.

Web Robots are sometimes referred to as Web Crawlers, or Spiders. Therefore the process of a robot visiting your website is called "Spidering" or "Crawling". When somebody says "the search engines have spidered my website," it means the search engine robots have visited their website. This robot is known by a name and has an independent IP address. This IP address is of no importance to us, but knowing their names will help since this name will be used when we create a robots.txt file. This is why the file is called "robots.txt." Given below is the list of the robots of some of the very popular search engines:

Search Engine - Robot
Alexa.com - ia_archiver
Altavista.com - Scooter (Bought by Yahoo)
UK.Altavista.com - AltaVista-Intranet (Bought by Yahoo)
Alltheweb.com - FAST-WebCrawler (Bought by Yahoo)
Excite.com - ArchitextSpider
Euroseek.net - Arachnoidea
Gendoor.com (Genealogical Search Engine) - GenCrawler
Google.com - Googlebot (http://www.google.com/bot.html)
Hotbot.com (uses Inktomi's robot) - Slurp
Inktomi.com Slurp - (slurp@inktomi.com) (Bought by Yahoo)
Infoseek.com - UltraSeek
Looksmart.com - MantraAgent
Lycos.com - Lycos_Spider_(T-Rex)
Northernlight.com - Gulliver
Nationaldirectory.com - NationalDirectory-SuperSpider
UKSearcher.co.uk - UK Searcher Spider

Writing Robots.txt:

Let's learn to write robots command. Note that there are two ways to write robots command. One is to include all the commands in a text file called "robots.txt" and another is to write robots command in the meta tag.

We will learn both ways of writing robots command.

Writing robots command in Meta tag:

There are 4 things you can tell a search engine robot when it requests (visits) your page:

1) Do not index this page - the search engines will not index the page.
2) Do not follow any links on this page - the search engines will not follow the links included in the page, i.e. they will not index any page that this page links to.
3) Do index this page - the search engines will index the page.
4) Do follow the links - the search engines will index the pages that this page links to.

Note that "index" is different than "spider". A search engine first spiders a page and then indexes it. Indexing is giving a certain importance to the page on the basis of its content, information, meta tags, link popularity with respect to the searched keyword. All this is decided at run time. When you tell search engines not to index a page, it means they know that "certain" page exists but do not rank them. That is, a no-index page will never be shown in their search results. This in any case does not mean a no-index page will not get visitors, it might get visitors indirectly from a page which links to it. Yes, no direct visitors from the search engines.

Suppose you want the search engines to index and also index (follow) its linked pages then include the following command in the Meta Tag:



Suppose you want the search engines to index a page but not follow its links then include the following command in the Meta Tag:



Suppose you do not want the search engines to index a page but follow its links then include the following command in the Meta Tag:



Suppose you do not want the search engines to either index or follow links of a particular page then include the following command in the Meta Tag:



Note:
Google makes a "Cached" of every file it spiders. It's a small snap shot of the page. Want to stop Google from doing so? Include the following Meta Tag:



Like any meta tag the above written tags should be placed in the HEAD section of an HTML page:



your title






Creating robots.txt file:

A robots.txt file is an independent file and should be written in a plain text editor like Notepad. Do not use MS-Word or any other text editor to create robots.txt. The bottom line is this file should have the extension ".txt" else it will be useless.

Let's begin. Open Notepad (it comes free with Microsoft Windows) and save the file with the name "robots.txt". Make sure that the extension is .txt.

By the way, did you note we did not use name of any robot in the meta tag! What does it indicate? Simple - by using meta you direct all the search engines to do something or not do something on a page. You do not have control over any one search engine. The solution is robots.txt.

It can always happen you do not want a particular search engine to index a page for certain reasons. In that case using a robots.txt file will help. Even though I do not recommend such a thing. The search engines get you traffic, why hate them. Stop them from doing their job and they hate you. I again repeat keep your pages smart for the search engines and welcome them. Fine, then why take the trouble to learn robots.txt? Why should you include a robots.txt file at all?

Let's suppose yours is a dynamic database site containing information of your newsletter subscribers, customers, their address, phone numbers etc. All these confidential information is kept in a separate directory called "admin". (It is recommended to keep such information in a separate directory. Handling data will be easier for you and so will be easy to keep the search engines away. We will just know how.) I am sure you would never want any unauthorized person to visit this area leave alone the search engines. It does not help the search engines either since they have nothing to do with the data or files there. Here comes the role of a robots.txt file. Write the following in the robots.txt file: (Ignore the horizontal row - they are included only to separate the commands from rest of the text.)

--------------------------------------------------------------------------------

User-agent: *
Disallow: /admin/

--------------------------------------------------------------------------------

This does not allow the spiders to index anything in the admin directory also including sub-directories if any.

The asterisk (*) mark indicates all the search engines. How do you stop a particular search engine from spidering your files or directory?

Suppose you want to stop Excite from spidering this directory:

--------------------------------------------------------------------------------

User-agent: ArchitextSpider
Disallow: /admin/

--------------------------------------------------------------------------------

Suppose you want to stop Excite and Google from spidering this directory:

--------------------------------------------------------------------------------

User-agent: ArchitextSpider
Disallow: /admin/

User-agent: Googlebot
Disallow: /admin/

--------------------------------------------------------------------------------

Files are no different. Suppose you want a file datafile.html not to be spidered by Excite:

--------------------------------------------------------------------------------

User-Agent: ArchitextSpider
Disallow: /datafile.html

--------------------------------------------------------------------------------

Similarly, you do not want it to be spidered by Google too:

--------------------------------------------------------------------------------

User-agent: ArchitextSpider
Disallow: /datafile.html

User-agent: Googlebot
Disallow: /datafile.html

--------------------------------------------------------------------------------

Suppose you want two files datafile1.html and datafile2.html not to be spidered by Excite:

--------------------------------------------------------------------------------

User-Agent: ArchitextSpider
Disallow: /datafile1.html
Disallow: /datafile2.html

--------------------------------------------------------------------------------

Can you guess what does the following mean?

--------------------------------------------------------------------------------

User-agent: ArchitextSpider
Disallow: /datafile1.html
Disallow: /datafile2.html

User-agent: Googlebot
Disallow: /datafile1.html

--------------------------------------------------------------------------------

Excite will not spider datafile1.html and datafile2.html, but Google will not spider only datafile1.html. It will spider datafile2.html and the rest of the files in the directory.

Imagine you have a file kept in a sub-directory that you wouldn't like to be spidered. What do you do? Lets suppose the sub-directory is "official" and the file is "confidential.html".

--------------------------------------------------------------------------------

User-agent: *
Disallow: /official/confidential.html

--------------------------------------------------------------------------------

I hope that's enough. A little practice is of course required. If the syntax of your robots.txt file is not written correctly, the search engines will ignore that particular command. Before uploading the robots.txt file double check for any possible errors. You should upload robots.txt file in the ROOT Directory of your server. The search engines look for robots.txt file only in the root directory else they totally ignore it. Mostly root directory is the directory where the index page is kept. In that case keep the robots.txt file in the same directory as the index file.

I know a user-friendly software that will write robots command for you (the software is introduced at the beginning of this article). It can make error-free robots.txt file very easily. This software RoboGen is a great tool. Never bother ever again to check the syntax of your robots.txt file or even write a robots.txt file yourself. RoboGen is a visual editor for Robot Exclusion Files and is easy to use. Just select files you want to be visited or not to be visited by the search engines, and it creates the robots.txt file. You can also select the search engines of your choice. RoboGen maintains a database of over 180 search engine user-agents, which are selectable from a drop down menu. It is the BEST and ONLY software on the Internet to write robots.txt file correctly and effectively. This great tool is cheaper than you expect. CLICK HERE NOW to know more!

Note: You should be able to see robots.txt file if you type the following in the address bar of your Internet browser.

http://www.your-domain.com/robots.txt

(Where your-domain is the domain name of your website. If yours is not a .com site, replace .com with the respective extension your website. For e.g. .net, .us, .org etc.)

You must be wondering whether to use Meta tag or Robots.txt or which of these is more effective!

A robots.txt correctly written is more effective than the meta tag. All search engines support robots.txt, but not all search engines support robots command written in the meta tags. I recommend that you use both so that you cover your site in both the scenarios. RoboGen will help you to write both!

One last thing - You can look in your web server log files to see what search engine robots have visited. They all leave signatures that can be detected. These signatures are nothing but name of their robots. For instance if Google has spidered your site it will leave a log file called Googlebot. This is how you know which search engine has spidered your pages and when!
About the Author
Senior Manager - Internet Promotions
http://www.searchengineoptimizationpromotion.com