How to be top of the search engines

Can some artfully placed keywords attract billions of bloggersand instant fame? Not any more, say the experts. The dark arts of the Webhave grown more subtle, and new media are not so different after all

Not long ago, my work colleague Charlie Brooker wrote a column dismissing9/11 conspiracy theories. Brooker is popular anyway, regularly attractinghundreds of comments on the Website of the Guardian newspaper which we bothwork for, but that week the site was overrun.

Thousands took to their keyboards to object. In the brave new world of digital media this is, ofcourse, exactly what we all want — lots of traffic, lots of hits, lots ofeyeballs on those ads. So the next week Brooker went further. If all ittakes is a word, or a phrase, why not pack the piece with them? His columnbegan thus: "Miley Cyrus, Angelina, Israel vs Palestine, iPhone, 9/11conspiracy, Facebook, MySpace, and Britney Spears nude."

This week it was announced that of the billions of searches typed intoYahoo.com over the last year, Britney Spears was the most popular, followedby World Wrestling Entertainment, and only then Barack Obama: one could, intheory, tweak that Brooker opening line to read: "Britney Spears, WorldWrestling Entertainment, Barack Obama, Miley Cyrus, RuneScape, JessicaAlba, Naruto, Lindsay Lohan, Angelina Jolie, and American Idol" (ie, therest of the top 10, once terms such as Google, or sex, or porn, have beenremoved), then just sit back and watch the tide come in.

Unfortunately, and with all due respect to my esteemed colleague, thingsaren't quite as simple as that. At a very basic level, we relate to theInternet in one of three ways: active, i.e., typing words into searchengines, a method limited — I use the word advisedly — by our ownimaginations; passive, i.e., checking in on the front page of the Guardian,the BBC, or the New York Times to see if our world has changed, then goingaway again; and interactive, i.e., things like the blogosphere.

In the prehistory of the Internet age — about 1995, say — that firstmethod of relating was paramount, and you could write "Britney Spears" (ormore likely "Spice Girls") 20 times in a first paragraph and stand a goodchance of having your article pop up at the top of the Yahoo results page.

But then Google came along, in 1998, and rewrote the rules. Now when youtype in a word, Google ranks the results according to how popular a site orarticle is, and it judges that not by how many hits it's had, or how manytimes a keyword appears, but how many times it's been linked to, and passedon: in effect, as Danny Sullivan, editor in chief of searchengineland.com,and an acknowledged world expert in "search", puts it, according to "thequality of links, and the context of those links".

So a piece consisting entirely of popular keywords might get lots ofhits, but it would also have a very high rate of "bounce" — people wouldexit as quickly as they'd entered, and definitely not pass it on to theirfriends, or link to it on their Website, or use it as a hyperlink in ablog. It would, in effect, be nearly invisible to a search engine such asGoogle — and, indeed, Brooker's piece doesn't show up when you type any ofthose key words in. Also, he only got the usual number of comments by doingit, instead of the 1,778 of the week before, when most of the traffic wouldhave arrived at the Guardian through communities of people discussing thecolumn, getting cross about it, and passing it on to other people who mightget cross.

'Keywords alone won't work'
Relying on keywords alone, says Scott Karp, formerly director ofdigital strategy at the Atlantic magazine, now CEO of Publish2, aWeb-based newswire, "is a losing game".

But how to win? That, of course, is the billion-dollar question, andarmies of people tackle it, every day: the dark art of manipulatingGoogle's algorithms, otherwise known as search engine optimisation (SEO),has become a big business in the last 10 years. "A common way to get a lotof links very quickly is by getting people in India or somewhere to makethem for you," says Paul Roach, the Guardian's head of SEO.

There are more aggressive, automated ways, too — scripting, usinghijacked computers to add links to blogs, hacking messageboards — but theseare referred to, in somewhat Disney-esque fashion, as "black hat" methods,and Google thoroughly disapproves: in fact, if you're caught using them,you're immediately banned. "We're what you call white hat," says Roach. "Wefollow Google's terms of service. Then again, we've got no reason not to —Google trusts the Guardian, so we generally do all right."

Other than that the answer, insofar as there is one, is internal to thepiece. "You just have to write great content," says Roach, blithely. Whichis easier said than done — and not just for the age-old reason that writinga good piece is a difficult thing to do. The fact is that a good piece onthe printed page is not quite the same as a good piece on the Internet,because the Internet is changing the way we read, and the way we write.

Ages-old argument
It is also, argued Nicholas Carr in a recent issue of the Atlantic,rewiring our brains: because we can move so fast, and our sources ofinformation are so disparate, "the Net is chipping away [our] capacity forconcentration and contemplation." Many of us might instinctively agree —but these are arguments that have been made over the millennia, fromSocrates, who objected that writing things down would damage our oralmemory, to Gutenberg's critics, who felt that the easy availability ofbooks would lead to intellectual laziness, to those who were worried thatnewspapers would cause a shallow cacophony that drowned out proper thought.The fact is that, Internet-wise, it's still early days; the jury is stillout.

It's not, for example, a question of vocabulary. "The impact of theinternet on the English language has been very, very small in terms ofgrammar and new words," says David Crystal, a linguist whose most recentbook is Txting: The gr8 db8. There are words such as mouse and click, butonly several hundred of them — which in the context of a language with amillion words is neither here nor there. And in terms of optimising search,new words can be a hindrance.

"Old words are better than new words," says Jakob Nielsen, an expert on web usability, "because they are morestraightforward and more likely to match the way people think." Karpteaches blogging, and emphasises a conversational tone - he tells hisstudents to imagine they're talking to someone just across the room fromyou.

Nielsen's general advice, for those of us who like a well-turnedsentence or a subtle argument, makes slightly depressing reading. "Stick tosimple presentation formats in all ways: a logical progression of thestory, mainly active sentences, simple words, short sentences, and a plain,scrolling page. Also, keep people looking down the page by scatteringattractive elements throughout the page in the form of subheads andbulleted lists. Plus, of course, use user-oriented keywords for bothheadings and subheads, emphasising the eternal 'what's in it for me?'perspective. Web users are selfish and brutal in rejecting material thatdoesn't immediately serve their purposes." Short pieces work. Lists workeven better. Long, thoughtful, investigative pieces don't.

Problem of generalisations
Then again, not everything on the Web is trying to impart usefulinformation as fast as possible. Crystal warns: "The problem of theInternet is the problem of generalisations. (There's such a range) from theWeb in all its fecundity to the opposite extreme of Twitter — let alone allthe interactive sites like YouTube. It's obviously making us learn lots ofnew literacies, and it's changing very, very fast. Whatever generalisationyou make today is going to be out of date tomorrow."

From a newspaper point of view, this is both an opportunity and a headache.There is, for example, the great tension between what people think theywant, and what we think they need to know. Newspapers used to be able tojust tell people things, and count on them listening. If a story was on afront page, that was what people read, and what people talked about — andeverything that wasn't on the front page got carried along with it, and wasread in its wake.

"Now," says Karp, "everything is up for grabs", and every piece has tofight for attention on its own merits. It isn't enough just to publish it:you suddenly have this "whole idea of actually having to market the news —how do you get it emailed around? How do you get it on to Twitter, or Digg(a content-sharing site)? And whose responsibility is it to flog stories?The PR department? The editorial staff? It's what bloggers do — they're outthere hustling, interacting with each other a lot. You actually have toengage in a more social dynamic."

Karp compares old and new media to the difference between standing in front of a room and giving a speech, andgoing to a party: it isn't enough just to walk into a room — you have toget stuck in there, and start talking to people, and try to work out whatthey're interested in, and what they want to hear.

Which is more possible than ever before. We can track exactly whatpeople are reading, and how many are reading it, almost in real time,through page impressions on Websites such as Google Trends.

While I was writing this piece, I knew for a fact that the most-read piece on theGuardian Website was about Roy Keane resigning as manager of the footballclub Sunderland AFC in the north of England, closely followed by a pieceabout beauty pageants. "Now, you could take a really vicious economic viewof it, and print only that type of story," says Karp. "But there are thingsthat journalism does because they're important, not because they'repopular. Which is a public service argument, not a business argument." Andin an increasingly difficult economic climate, an argument that is becomingharder to make.

Then again — you guessed it — it's not that straightforward. The wisdomof crowds may tend towards the obvious (football, Britney Spears) but itdoesn't always, and can't be predicted. Last year, a short BBC Websitepiece about a Sudanese man being required to marry a goat turned up on overa million other websites. It was the kind of piece that might get picked upby Digg, which has an average of 35 million users a month, and tendstowards the quirky, fratboy end of the spectrum (yesterday videogames, foodfights, and movie one-liners were doing well), and thus climb the searchlists — but deliberately aiming to appeal there would bend the Guardian'sprofile completely out of shape.

You could track Google Trends, and make absolutely sure you always havea story on the most popular subject that week, as some newspapersapparently do — but that way madness lies, if only because, as Sullivanpoints out, "by the time you catch up, people may not be searching for that(topic) any more." Not to mention the fact that a newspaper should reallybe setting the trends, not following them. The big spike in Britneysearches this month was in part caused by traditional media — she releasedan album, and appeared on The X Factor.

No help writing great headlines
More profitably, you can study exactly how people put words togetherwhen they're searching for things. Traditionally, and very enjoyably,newspaper headlines have tended toward the witty and playful — but it's nohelp to be writing variations on "Asian wave of death," as, reportedly, theNew York Times was in December 2004, until it was pointed out to theheadline writers that "tsunami" was actually the word people were searchingfor.

But even that wouldn't necessarily be enough. Google News doesprivilege large news organisations, but on a big story like that it has4,500 or so such organisations to choose from, and it is, to a largeextent, pot luck (otherwise known as a non-human Google algorithm) as towhich one ends up on top.

You could, to follow that party analogy, just stand in the middle of theroom and shout, tendentiously. And it might work very well, depending onhow you do it: Not long ago, Karp says, he put a piece up on his blogcalled, "Why I stopped using Twitter - because 'Twitter's a massive wasteof time'". It was actually more nuanced than that, but —I did kind ofwrite it knowing it was going to be big, as a joke: let's see if I can hackthe blogosphere. It went viral."

But that's clearly a method with diminishing returns — although, inadvertently, it also points to onerelatively failsafe way of getting lots of traffic: the Internet lovestalking about itself. For a while last week, if you put the word "pirates"into Google, it was coming up with a piece about Internet piracy instead ofSomalis because although the latter were the big story of the day, theblog-savvy tech community were spending a lot of time telling each otherabout the other sort. It is conceivable, for example — though not in theslightest bit predictable — that this piece could do better than anotherpiece about Britney Spears, or Obama.

Unless, of course, that piece was real news, and the more surprising thebetter. When I ask Sullivan how I might write a story that could beguaranteed to come top on every search engine, and every most-read list onthe planet, the answer is quite simple. "You would want to have a newsstory that was unprecedented," he says, "on a topic that would attract tonsof people and was your exclusive - like, let's say, 'Obama has decided toquit tomorrow.' You'd have huge amounts of traffic." Which is not sodifferent from traditional media after all.