SUBSCRIBE NOW WELCOME BACK. Do you want to continue reading where you left off? New Republic subscribers can pick up where they left off no matter which device they were previously using. SUBSCRIBE NOW

Go Home Failed States, Dubious Rankings

THE PLANK JUNE 30, 2009

Failed States, Dubious Rankings

I spent some time yesterday and today trying to figure out Foreign Policy magazine's ranking of failed states. Somalia, Zimbabwe, and Sudan got first, second, and third place--no surprises there. But what initially piqued my interest was the high ranking given to Kenya, a country where I just spent two weeks (on a trip sponsored by the International Reporting Project, based at Johns Hopkins). Kenya placed 14th in the study--higher than totalitarian states like North Korea and Equatorial Guinea, on the one hand, and also higher than countries like Sri Lanka (which recently concluded a bloody civil war) and Lebanon (which seems to be ever on the brink of another one). And it was just one spot below Burma (which, in addition to being a totalitarian state, is host to an ongoing ethnic insurgency). This struck me as odd.

Don't get me wrong: Kenya has a lot of problems. It is just 18 months removed from a spasm of ethnic violence that followed a disputed election. And the coalition government patched together by international mediators in order to stem the violence--the ostensible winner became president; the loser became prime minister--has proven totally dysfunctional. With 42 different ethnic groups vying for political power, plenty of Kenyans will tell you that the country could easily erupt again at any minute. As for the poverty, both rural and urban--I'm not sure how to describe it in a single sentence, except to say that it is soul-crushing.

Still, Kenya is a relatively free place. It's a democracy, with regular elections and a vigorous press that lambastes the government for transgressions large and small on a daily basis. The current government may not be a particularly impressive operation but it a) exists (unlike in Somalia); b) isn't throwing dissidents in prison (unlike North Korea, Sudan, Burma, etc); and c) was more or less chosen by the people it rules (unlike a good chunk of the governments on this list).

So when I saw the rankings (on which Foreign Policy collaborates with The Fund for Peace), my initial reaction was: What kind of metric for ranking states could possibly yield results like this? At first, I reminded myself that the term "failed state" was not synonymous with the term "cruel state." If you define the term literally--i.e., a state that is in danger of crumbling--you can see how Kenya might outrank North Korea. Given a choice, we would probably all opt to live in Nairobi over Pyongyang; but that doesn't mean the Kenyan state is more stable than the North Korean regime. In fact, it is probably less stable, since North Korea seems to have a tragically good grip on its populace. But if stability is the metric, then how does the Burmese government--which also, sadly, seems to be in no danger of collapsing--outrank Kenya? And how does North Korea, where the central government actually controls the entirety of its territory, outrank a state like Lebanon where a decent chunk of the country is controlled by a militia that takes its orders from a foreign government?

The conceptual problem with the rankings, I eventually realized, was that different criteria were pulling in opposite directions. Each state was given scores in 12 categories, then the scores were totaled. North Korea accumulated a lot of points in categories like human rights and "deligitimization of the state"--but precisely because the state allows its citizens no rights and does not care whether they find it to be legitimate, it scores relatively low on indicators like "factionalized elites" (no factions are allowed in North Korea) and "refugees" and "human flight" (few are allowed to enter or exit the country). Contrast that with a country like Kenya, which is relatively free (and therefore has a better score on human rights than most other states near the top of the list) but also has all the problems you might expect from an unstable, tottering, developing-world democracy. You could certainly argue that both states are failing. But they are failing for opposite--indeed mutually exclusive--reasons. Arranging them according to a single measure of "failure" isn't just an exercise in futility. It's completely meaningless.

Foreign Policy concedes this problem (sort of), writing that, "as Tolstoy might have put it, every failing state is failing in its own way. ... Somalia and the Democratic Republic of the Congo are failing because their governments are chronically weak to nonexistent; Zimbabwe and Burma are failing because their governments are strong enough to choke the life out of their societies." But why present states that essentially suffer from opposite problems on the same continuum at all? The only even semi-plausible argument I can think of is that the presence of human rights violations in a state like North Korea or Burma makes it more likely that the state is eventually going to fail--by creating discontent and causing people to rebel against their government. Unfortunately, an oppressive state's willingness to abuse human rights probably makes it less likely to become chaotic or weak. After all, keeping their own people in a state of constant fear is how the North Korean and Burmese regimes have survived for so long.

The appeal of these rankings lies in the comforting sense of empiricism they convey--the sense that you can remove ideology from the equation and just rank countries based on numerical indicators of how bad things look. But that's exactly the problem: Ideology--what principles a state uses to organize itself--matters a great deal. Does it really tell us anything useful about the world to say that a strong totalitarian state that would do its people a great favor by collapsing (Burma) is marginally more "failed" than a weak democracy riven by ethnic tensions that we wish would succeed (Kenya), which is itself marginally more failed than another strong state one hopes might collapse (North Korea), which is itself more failed than another weak democracy that we hope will succeed (Georgia)? Rankings are fun and can be illuminating, but this comparison of apples and oranges doesn't make any analytical sense.

--Richard Just

SHARE YOUR THOUGHTS

Show all 6 comments

You must be a subscriber to post comments. Subscribe today.

6 comments

A big issue is that they don't acocunt for environmental disasters.  Take Yemen, for example (always my example).  It is 18th, but the fact that it is running out of water, and soon, doesn't seem to affect its rankings.  As soon as it does, it will be pushed over the cliff on which it is tetering.  

- boneill

June 30, 2009 at 4:36pm

You must be a subscriber to post comments. Subscribe today.

12 UNWEIGHTED categories generate junk.  Each category has to have a percent importance (eg freedom 23%, reliable police 14%, etc and that all categories sum to 100%).  This is called Multi-attribute Utility Analysis and has been around, but rarely used, for a long time.  The above results are just one more example of its lack of use.  You probably know this, but this is the alternative:

Buying a new car with color and price as the criteria

Car 1:  Color 10 points, Low Price 1 points

Car 2:  Color 1 point, Low Price 9 points.

Only a dummy would choose Car 1, but the numbers lie.

- dashendorf

June 30, 2009 at 5:51pm

You must be a subscriber to post comments. Subscribe today.

Good research, Richard. Rankings are inherently suspect unless they're based on one objective criterion. It's why the US News rankings of colleges and universities are so useless.

- rozenson

June 30, 2009 at 6:22pm

You must be a subscriber to post comments. Subscribe today.

I work in foreign policy, and several colleagues are regularly published in FP, and yet I don't know a single person who takes the ranking of FP's failed-states list any more seriously than they take the rankings on Parade Magazine's annual worst-tyrants list. It's an informative, interesting, deliberately counterintuitive annual report, but it's also magazine fluff.

As to the multiplicity of items scored, that should neither surprise nor trouble, since states have many functions, and may therefore fail in many ways. From simply existing to controlling borders to enforcing laws to providing services to seeing to the state's own continuity to protecting citizens' rights, states have a number of functions. I think a fair analogy for the relative rankings of North Korea and Kenya would be between a fully functional fighter jet that's flying straight toward a mountainside and a passenger jet in level flight on a clear day with an engine that's flamed out, copilots who don't share a language, a broken fuel gauge, attendants who've gone on strike, a blocked toilet, failed hydraulics in the rudder, and rioting passengers who are threatening to open the emergency door. Now take a snapshot of those two planes: Which one is more likely to fall out of the sky in the next five seconds? The Kenyan Airbus, clearly. Yet which one is more likely to land safely? The Kenyan Airbus. Which plane would you therefore prefer to be on? Certainly not the North Korean MiG. While the jet itself is in better shape, you know there's a mountainside somewhere out there ahead, and the North Korean flight will end in a crash.

So it may be perfectly reasonable to conclude that considered as a state, the Kenyan state is closer to being a failed state today or this year, but in the long run, North Korea is more likely to fail catastrophically.

- rhubarbs

July 1, 2009 at 7:42am

You must be a subscriber to post comments. Subscribe today.

Afraid Of The Public Option? This Is What America's Health Care Will Look Like Without It , by Jacob

- Anonymous

July 1, 2009 at 11:01am

You must be a subscriber to post comments. Subscribe today.

"Does it really tell us anything useful about the world to say that a strong totalitarian state that would do its people a great favor by collapsing..."

Five years after Iraq, can anybody still blithely argue that its people were done "a great favor" by the collapse of their strong totalitarian state?  There are an awful lot of headless corpses in unmarked graves who would take issue with Mr. Just's cavalier dismissal of the value of stability, even sans liberty.

It does tell us something useful to say that the people of Kenya are at comparable risk to the people of North Korea, however much we hate tyranny and love democracy.  It tells us that no, ideology *doesn't* matter all that much when we come right down to it, because the dead are just as dead whether they are crushed by tyrants with a monopoly on force or opportunists taking advantage of the government's lack thereof.

- austinexpat

July 1, 2009 at 11:05pm

You must be a subscriber to post comments. Subscribe today.

SHARE HIGHLIGHT

0 CHARACTERS SELECTED

TWEET THIS

POST TO TUMBLR

SHARE ON FACEBOOK

Close