TIPS and TRICKS
The ranking algorithm of Yahoo
Much has been written about Yahoo over the years. Webmasters have exchanged tips on how to get listed in the directory, while reporters have covered the company's past success and recent troubles. Yet, the algorithm of Yahoo's search has not attracted nearly as much attention for one reason or another. Speculation and general advice about it have been available, but only a few people have seriously attempted to explain how Yahoo ranks sites.
Recently, I was in progress of updating the article about getting listed in Yahoo on this site. While working on it, I realized that I really had to write something about Yahoo's ranking algorithm. Even while Yahoo is a directory and not a search engine, it does have a search feature which visitors often use. A site that ranks badly on Yahoo's search will miss out on a great deal of traffic that the directory could potentially produce.
So, what kind of elements is Yahoo's search algorithm built out of? For long, it's been claimed that including keywords in the title, description and/or URL you submit to Yahoo boosts your search ranking for those words. After Yahoo introduced click-through tracking on its pages, click popularity has also been said to be an important factor.
The latest theory, originating at WebmasterWorld, is that your Google PageRank also influences your ranking at Yahoo's search. As Google already provides the secondary "Web Pages" results, it wouldn't be a big surprise if parts of its algorithm were also being used to rank the primary "Web Site" results.
Investigating the effects of click popularity from the outside is quite hard. However, examining the other things that are said to influence your Yahoo ranking is relatively easy. In this article, we'll look at three search queries performed at Yahoo and the sites they turned up in order to determine how the algorithm works.
If you aren't interested in the details, you can skip to "Conclusions" for the juicy parts. Just keep in mind that this experiment contains a very limited amount of data, so it's not wise to form too strong opinions based on it.
The raw data is also available in a .txt file for those who want to examine it.
Table legend
The first query was the word "spoon", without quotes. The picture on the left clearly
displays what it turned up, but as you probably aren't psychic, it's a good idea to
explain the meanings of those colors, dots and lines.
Dots:
- Light green dots (
) display the
PageRank of individual sites. For example, the site displayed first in the results
had a PageRank of 7. Thus, the light green dot on the very left side of the table is
placed on the fourth line from the top. From the PageRank scale on the left you can
clearly see that the dot is in the right place. - Light blue dots (
) display whether
the site had the keyword in its title or not. If the keyword was in the title, the
dot is placed near the top of the table. If no keyword was found in the title, the
dot is placed near the bottom. As you can see, all the sites in the top ten had the
keyword in their titles and thus all the light blue dots are located at the top of the
table. - Light yellow dots (
) display whether
the site had the keyword in its description. The sites that had the keyword in the
description are marked by an yellow dot placed at the top of the table, while those
that didn't can be distinguished from the dot being located at the bottom of the
table. By quickly looking at the table, you can see that from the sites in the top 10
for this keyword, only one had the keyword in the description while nine sites
didn't. - Light grey dots (
) display whether the
site had the keyword in its URL. Should the URL contain a keyword, the dot is placed
at the top of the table. Should it not, the dot can once again be found near the
bottom.
Now, take a good look at the top part of the table, right were the dots representing the second site are located. See how the dot for the URL is actually above the table? The site in question happened to have two occurrences of the keyword in its URL, but as you can see, the scale on the right only goes up to 1.00. If the amount of keywords in the URL, description or title exceeds one, I've placed the dot "out of scale". This should help you notice the difference between the sites that have a description/URL/title that only contains the keyword once and those that have multiple keywords in those places.
- Light red dots (
) display whether the site had the
keyword in the name of the category it is listed in. This dot follows the same rules as the three
previous dots, so there probably is no need to discuss it further.
Lines
- Dark blue line (
) displays the average amount
of keywords the sites in a single part of the table had in their title. Because every site in
the top ten had one keyword in the title, this value is 1.00 for sites 1-10. For sites 15-130
the value is only 0.58 and for sites 135-144 it is 0.30. Thus, the blue line starts from the
very top, but falls fast close to the bottom. - Black line (
) displays the average amount of
keywords the sites in a single part of the table had in their URL. As you can see, this value
starts at around 0.80, then drops to 0.38 only to rise to 0.70, near to where it started. - Dark green line (
) displays the average PageRank
for the sites in a single part of the table. If PageRank influences the results, this line should
start its journey from a relatively high point, but should steadily fall until reaching its lowest
point in the bottom-10 of sites. The average PR for top 10 is 5.70, for sites 15-130 it is 4.08
and for sites 135-144 it is 2.60. - Dark red line (
) displays the average amount of
keywords the sites in a single part of the table had in the name of the category they were
listed in. Under this keyword it seems to be pretty stable, but does fall a bit when we reach
sites 135-144. - Orange line (
) displays the average amount of
keywords the sites in a single part of the table had in their description. Works just like the
other lines.
Keyword: spoon, 144 results returned
Sites 1-10, every site reviewedSites 15-130, every fifth site reviewed
Sites 135-144, every site reviewed
Title
When you look at the part of the table where sites 1-10 are located, a few things stand out. Perhaps the most significant of them is that every site has one instance of the keyword in its title.
After site 35, the keyword slowly starts to disappear from the title. It reappears near site 90. However, in the sites that rank below 90, the keyword is only present in the title as a part of another word. Titles such as "I am Spoonbender" and "Spooner Advocate" are common near the 100th place.
Upon reaching the "bottom ten", the "Keyword In Title"-average continues to drop. Not one site has the basic form of the keyword in its title, the only occurrences are as a part of another word.
Description
It's interesting to notice that only one of the top ten sites has the keyword included in its description. The average number of keywords per description creeps up to 0.3 for sites 15-130. Since the sites in the top ten only had an average of 0.1, this seems slightly odd.
This rise is soon followed by a decline, as none of the sites in places 135-144 have the keyword in their description. The average ends with a flat 0.00.
URL
Seven of the top ten sites have the keyword in their URL, which creates a very high average. This might indicate that having a keyword-rich URL is important for your ranking, but it may just as well simply be a matter of chance.
In the next section (sites 15-130), the average drops to slightly below 0.4. It rebounds and rises up to 0.70 in the bottom ten, which made me scratch my head a little. Does having the keyword in your URL influence your ranking in a negative way? Most likely not.
The increase is probably caused by the fact that in order to include your site in the results, Yahoo has to know that your site is somehow related to the word "spoon". Having the keyword in your URL, but not in your description, title or category is enough to get you listed under "spoon". However, it won't be enough to get you a decent ranking, causing the keyword-in-URL average for bottom-10 sites to jump.
Category
Three of the top ten sites have the keyword in the name of the category they are listed in. The average doesn't change much throughout the results. It is 0.3 for 1-10, 0.29 for 15-130 and 0.20 for the last ten sites. It does decline as we go downwards in ranking, but not enough to show a clear pattern.
PageRank
The PageRank theory seems to hold up well in the first section. The number one site is a PR 7, next three are PR6 and the rest are a mix of PR6 and PR5 sites. While some PR5 sites rank above PR6 sites, it is hardly enough to destroy the theory. After all, no one is claiming that PageRank is the only factor in determining your ranking at Yahoo.
After the top ten, the average keeps dropping steadily just as predicted. However, the picture is very mixed. Some sites located between rank 15 and 130 have a PR6, while others have a mere PR1. This search does not seem to indicate the presence of PR in the algorithm. However, "spoon" only produces 144 results, which reduces the reliability of this data.
Average PageRank still continues to drop and reaches its smallest value at 2.60 in the bottom ten. The PR of individual sites still does not produce a clean, smooth decline towards the end of the results. There is, however, one thing that catches the eye. The three last sites, #142, #143 and #144 all have zero PageRank. A coincidence?
Keyword: rally, 406 results returned
Sites 1-10, every site reviewed
Sites 15-150, every fifth site reviewed
Sites 397-406, every site reviewed
Title
Similarly to the keyword "spoon", the 1-10 sites for "rally" have an average 1.00 keywords in their title. The keyword average also declines in the same, steady way as it did in the last search. For sites 15-150, the average is still 0.82, but in the bottom ten it is just 0.1.
The average looks like it is falling faster towards the end of the results in this table, but it is not necessarily doing so in reality. Because no data is available for sites 150-397, the drop seems sharper than it really is.
Description
Again, the top ten sites seem to lack the keyword in their descriptions. Only two out of ten have it, which is not much more than what we saw for the word "spoon". The biggest shock is however yet to come, as the keywords-in-description average keeps on rising as we move downwards in the results. Sites 1-10 have an average of 0.2, 15-150 have an average of 0.25 and the bottom ten has a whopping average of 0.40.
This trend makes one wonder just how important it is to have keywords in your description, but I'm not ready to make any conclusions just yet.
URL
Was the 0.8 keyword average for "spoon" something that reflected the way the algorithm works, or was it just a random thing? The top ten for "rally" would seem to support the latter theory, as the average is now just 0.4.
Furthermore, the average amount of keywords per URL rises towards the bottom of the results in this search. Part of the rise can probably be explained by the same phenomenon that appeared to occur in the keyword "spoon". However, while "spoon" seemed to indicate that it would be hard to reach the top without having the keyword in your URL, "rally" gives hope to those who don't have a keyword-rich URL.
Category
Like the URL average, the category average behaves somewhat differently in this search than it did in the previous one. It starts out at a very high position in the top ten, 0.60. The average then slowly declines to 0.57 for sites 15-150 and arrives to its lowest point, 0.20, for sites 397-406.
Under "spoon", the average amount of keywords in category name was generally much lower. The reason for this is unknown, but it might have something to do with the nature of these words. "Rally" refers strongly to a specific sport that has its own categories within Yahoo. Many of these categories contain the word "rally" in their name.
"Spoon" on the other hand can refer to just about anything, from the Spoon River Community College to foods that can be eaten with a spoon to Teresa Weatherspoon, the WNBA basketball player. These sites are not very likely to be found in a category that has a name containing the word "spoon".
PageRank
Just like for the keyword "spoon", the top ten sites for "rally" have very similar PageRanks. The leading site has a PR 7, eight others have a PR6 and one has a PR5. This makes the average PR for the top ten a healthy 6.00.
When we move into the middle section, we are faced with results that initially seem very promising. Sites 15-55 all have a PR of five, which is exactly how things should be if PageRank played a part in the algorithm. While things seem to somewhat fall apart after that, the PageRanks of sites that have similar ranking are still much closer to each other than in our previous search.
"Spoon" only produced 144 results, while "rally" gave us over 400. For "spoon", the PageRanks of individual sites didn't seem to correlate with ranking very strongly, but for "rally", the image is much clearer. Is the increase in the number of results eliminating the "static" and giving us a better idea of what is going on? The next keyword should allow us to answer that question.
Additionally, the fact that the average PageRank again declines smoothly towards the end (6.00 for top ten, 4.61 for sites 15-150 and 2.90 for the bottom ten) also breathes life to the PageRank-theory. In both searches, the average has lined up in a way that is very exciting.
Keyword: tool, 10244 results returned
Sites 1-10, every site reviewed
Sites 15-150, every fifth site reviewed
Sites 300-350, every fifth site reviewed
Sites 2240-2290, every fifth site reviewed
Sites 5240-5290, every fifth site reviewed
Sites 10235-10244, every site reviewed
Title
This keyword confirms what the two previous ones already seemed to be telling us. Have the keyword in your title, or forget about reaching the top. Because the number of results returned is so large, the average amount of keywords in title per site holds steady at 1.00 for sites 1-10, 15-150 and 300-350.
It drops off at some point after that, because when we reach the 2240th place, the average is a flat 0.00. After encountering the 0.00 mark once, the average remains close to it throughout the rest of the results. Only one site in the 5240-5290 range has the keyword in the title which boosts the average back up a bit. This small rise can in my opinion be safely ignored, as it is more likely to be caused by chance than anything else.
Description
The data for "rally" made me wonder about the importance of including keywords in your description. "Tool" does nothing to crush these thoughts as nonsense, because it starts with a similar pattern as the two other searches did. The average for top ten is only 0.3, while for sites 15-150 it is slightly higher, 0.39.
The downward trend starts after that, but it appears to be relatively slow. Average for sites 300-350 is 0.36 and comes down to 0.27 for sites 2240-2290. It's fascinating to notice that the sites placed 300-350 have, on average, more keywords in their description than those ranked in the top ten.
Somewhere after rank 2290 things really get interesting. The keyword average jumps to 1.00 at 5240-5290 and is the same for the bottom ten. Again, we're likely to be seeing the same thing that happened with the URL average for the keyword "spoon".
However, one question comes to mind. Why is the "keywords in description"-average 1.00 for the bottom ten, but the "keywords in URL"-average 0.00 for the same sites? If this is indeed the result of the same phenomenon that appeared to occur for the keyword "spoon", shouldn't it be the other way around?
That is something I can't explain, but I do have a theory. I believe that because there are so many matches, the sites that only contain the keyword in the URL aren't included in the results at all.
URL
The URL average again produces a confusing pattern. It starts at 0.30 in the top ten, rises to 0.46 for sites 15-150, but begins to fall once we move further down in the results. In each of these three searches, the URL average has looked completely different. Stubbornly, it has refused to reveal just how important or unimportant it is.
Despite the lack of a distinct trend, or actually because of it, this data would seem to indicate that having the keyword in your URL isn't crucial to your ranking success. It may help, but it won't make or break you.
Category
The category average for the keyword "tool" doesn't fit in perfectly with the averages for "rally" and "spoon", but it is logical enough to give us some information. In all of the keywords, the average drops towards the end of the results and reaches its lowest point in the bottom ten.
However, while the results for this keyword confirm that being listed in a category that has a keyword-rich name is indeed good for you, it also brings up new questions. Why is the average for sites 1-10 so low, when it was relatively high under the two other keywords? Much more research is needed before this question can be answered with certainty.
Also, notice the spike in the graph around rank 2240-2290. While the average amount of keywords in the title drops, the average amount of keywords in the category name rises. After that, the amount of keywords in the category name drops and the amount of keywords in the description simultaneously goes up. I believe that these spikes are telling us something along the lines of "a keyword in the title is worth more than a keyword in the name of the category, which in turn is worth more than a keyword in the description".
PageRank
Finally, we get just what we've been waiting for. Once the number of results grows to over 10000, the static we saw under "spoon" and "rally" nearly completely disappears. There are still small differences, but the overall trend doesn't leave much room for debate. In my opinion, it seems clear that the theory is correct - a high PageRank improves your ranking at Yahoo, while a low one reduces it.
As with the previous keywords, the PR average smoothly diminishes when we move towards the right side of the table. Yet, it goes up between 2290 and 5240, only to fall to a minimal 0.9 for the last ten sites. Pay attention to how the rise in PR occurs at the same time as the average amount of keywords in the description goes up and the number of keywords in category name goes down.
I feel that this sends us an important message; PageRank is there, but it is not a demigod. You can improve your ranking by boosting your PR, but there's a limit to how high it will take you if your listing is not otherwise optimal.
Conclusions
Let's repeat the standard disclaimer once more, just in case. We only looked at three keywords and what's worse, we only looked at parts of them. Not all sites that were returned as results were reviewed. I wish I could have made a more thorough study, but even this small experiment has taken well over 15 hours of time and reporting its results has required around 4000 words.
Even while we've only scratched the surface, I believe that we've been able to learn something about Yahoo's algorithm. If you read the entire article, you already know most of the conclusions I made based on what we saw. However, if you decided to skip directly to the end, here is a summary of those conclusions:
Title
The most important thing you can do to your ranking is to include the keyword in the title you submit to Yahoo. If your title does not contain any keywords, you'll never rise high enough to collect a large amount of clicks.
Is including the keyword once in your title enough, or should you risk it and try to repeat the word? I didn't see a single site that had two or more instances of the same keyword in its title. That doesn't mean there aren't any, but I don't believe that having two keywords will give you a big advantage over having one. Also, trying to repeat the keyword might get your title edited or your submission rejected, so I wouldn't try it.
Description
This really surprised me. I started the experiment with the belief that having the keyword in the description is absolutely vital in order to gain a high ranking. However, the data seemed to prove otherwise. The top ten sites actually had less keywords in their descriptions than those listed after them.
Still, I don't believe that including keywords in your description will actually hurt you. It is likely to boost you upwards a bit, but I think the boost will be significantly smaller than what it has generally been claimed to be.
URL
No clear data here, so I won't make too strong conclusions either. Some words are in order, though. My current position is that having the keyword in your URL is worth something, but less than having the keyword in any of the other places (description, title, category).
Is it necessary to get a keyword rich domain name if you're desperate to reach the top ten? Not in my opinion, unless it will give you a valid reason to include your keyword in your title.
Category
Being listed in a category that has a keyword in its name definitely increases your ranking for that keyword. More than having the keyword in the URL and more than having the keyword in your description, but less than having the keyword in your title.
PageRank
Yes, I think that it influences your ranking. Less than having the keyword in your title or in the name of your category, but it does play a part. This gives a small glimmer of hope to everyone who has accidentally submitted a poor description or title to Yahoo. You can never correct the situation entirely, but it's good to know that at least there's something you can do even if you can't get Yahoo to change your description and title.
