Alex Steer

Advertising effectiveness, analytics and strategy / about / archive

Next up, crowdsourcing for brain surgery

1155 words

I'm going to try and get in before the crowd on this one, because the backlash will be inevitable.

The Encyclopedia Britannica is going to allow some user-generated content onto its site. However (according to the BBC News story), unlike Wikipedia, it will be maintaining very tight controls on what is allowed on. From what I can tell, only contributions that get an initial nod from the editors will be allowed onto the site, and even then it will carry a 'Britannica Checked' mark, to distinguish it from the main text.

To explain this new approach, Jorge Cauz, the president of Encyclopedia Britannica Inc., said:

'We are not abdicating our responsibility as publishers or burying it under the now-fashionable 'wisdom of the crowds... We believe that the creation and documentation of knowledge is a collaborative process but not a democratic one.'

This is clearly going to draw some fire from fans of user-generated content, who (I predict) will call it aloof and offensive. I anticipate that someone, somewhere will cite the analysis by Jim Giles, published in Nature in 2005, which purported to show that Wikipedia and EB were of comparable accuracy on science topics, even though this has been comprehensively rebutted by Britannica. (Nature's response is pretty inadequate.)

So I'm going to defend the position, and not just because I'm a curmudgeonly lexicographer. You see, what's at stake, which Cauz seems to realise, is the verifiability of text as well as its accuracy. To explain: even if the Nature study were unimpeachable, and found that Wikipedia and EB entries were of roughly equal accuracy, there's still no way of assessing the accuracy of an entry in either unless you know a lot about the subject matter. This is obviously a problem, and an encyclopedia is useless if you have to check the facts of whatever you're reading independently. Does this mean encyclopedias are useless, then? No - but it means you have to trust whoever's making them.

This is tricky, of course, but inevitable, and not just in encyclopedias. You also have to trust that your brain surgeon knows how to operate on brains, and that the person who fits your gas boiler knows enough not to blow up your house. This is what's known as the professionalization of society: we can't all do everything, so we willingly hand control over our lives to people who can do what we can't.

The modern movement towards user-generated content began, really, with blogs, in response to the idea that bloggers on the ground can share information and insight more quickly, and perhaps more accurately, than the mainstream media. This is, in large part, probably true, and 'citizen journalism' has rightly been heralded. But journalism is an odd profession. A reporter's main duty is tell people what is going on in a certain part of the world ('the here') at a given time ('the now'). Aside from the ability to write clearly, and a commitment to broad impartiality (I'm talking hard news reporters here, not comment writers), there aren't that many specific technical skills involved, except maybe shorthand. This is not to denigrate journalism - it's hard work, and hard to do well - but the entry barriers are not particularly high, and so what journalists do can be copied by bloggers.

Writing encyclopedias, like brain surgery or fitting boilers, requires a lot of technical skill. The skill is not the ability to write (though this matters) but the ability to design research: to dig out facts and verify their accuracy. Journalists do this too, but normally on less obscure subjects. This skill is important. Whether you're reading an EB or a Wikipedia entry, you have to trust that whoever wrote it is right.

So whom do you trust: an anonymous EB encyclopedist or an anonymous Wikipedia contributor? Here's a good way of deciding: starvation.

We all, unless we're very rich, have to work to live. Encyclopedists get paid for what they do. Their work is monitored by other encyclopedists, who also have to write and review entries in order to make a living. If they screw up, they get sacked. Also, since encyclopedia-writing is something you get paid for, the entry barrier is quite high: you will have to prove your ability to research before they'll let you do it. You will also, probably, be a near-specialist in a given area: botany, for example, or politics, or history, or Japan. In short, as in all jobs, there is pressure to perform. Here, the pressure is to be right. With Wikipedia, there is no pressure to be right (apart from personal pride) because nobody does it for a living. If I had to pick, with no prior knowledge and no ability to check, whose entry to trust, I'd choose the writer whose kids won't eat if the text isn't good enough.

Now, on the negative side, the point can be made that the sheer number of authors on Wikipedia creates an effective check on inaccuracy. This is true, and to a layman it seems to work. Aside from occasional vandalism (which it seems unfair to dwell on), to a layman most of the entries seem pretty accurate. In that sense, most of what's in it is 'good enough'. The problem comes when you need to be absolutely sure on a certain detail. How do you know that the research has been done using up-to-date resources, and that something important but obscure isn't missing?

You, as an amateur, can only critique an entry up to the limit of your own knowledge. On Wikipedia, all you know is that the entry has been edited by lots of other people who may not have more expertise than you. Yes, there is weight of numbers, which will allow for small corrections to aggregate, but you can only hope that some of that editing has been done by someone who really knows the subject. Even then, that person would need a lot of spare time and passion to improve an entry dramatically. At least with a professionally-edited encyclopedia you know that serious money and time have been spent by the publisher to get the best possible quality.

And yes, sometimes encyclopedists get it wrong, which is why Britannica's idea of letting people submit corrections and suggestions to the editors is a good one. Collaborative but not democratic: it reminds us that the most important thing about an encyclopedia, like a life-saving operation, is that it is done well, not that everyone gets to have a go.

# Alex Steer (24/01/2009)