Tuesday, February 14, 2006

Happiness and wellbeing in Australia: some dodgy analysis

There was a bit of a flurry in the media and the Aussie blogosphere yesterday with the publication of the latest "Australian Wellbeing Index", which for the first time compares happiness across Australian federal electorates.

Comparing anything between geographical areas always brings out our latent tribalism, parochialism and schadenfreude.

The (Melbourne based) Age crowed:
VICTORIA is the happiest state in Australia and some of its most contented people live in Melbourne's wealthiest suburbs.
while the Sydney Morning Herald lamented:

IS IT aircraft noise, John Howard's long reign, or being overworked and underpaid that makes the so-called chardonnay socialists of Sydney's inner west the most disgruntled people in Australia?

A new survey that compares the wellbeing of people in all 150 federal electorates reveals the safe Labor seat of Grayndler… tops the national list for all-round unhappiness.

Here’s my guess: it’s not aircraft noise, John Howard or being overworked. In fact, I doubt whether Sydney residents are disgruntled at all – it’s just the way the survey is compiled. Here’s a clue in the SMH article:

The scorecard, known as the Australian Unity Wellbeing Index, is based on how satisfied people feel with seven areas of their life - standard of living, health, relationships, what they are achieving in life, safety, community connection and future security.

It shows the happiest electorates tend to have a lower population density, a high proportion of people over 55, more females, more married people and less income inequality. Importantly, they have a strong sense of "connection to their community".

Hm. An index that rates people’s happiness based partly on ‘community connection’ then finds that happy people tend to have a strong sense of ‘connection to their community’.

If we actually have a look at the report on the index itself, we see why that might be.
Subjective wellbeing is measured by the Personal Wellbeing Index. This comprises seven questions that ask "How satisfied are you with ------>?" The specific items people rate are their standard of living, health, relationships, what they are achieving in life, safety, community connection, and future security. The numerical rating scale offers 11 choices from 0 (completely dissatisfied) to 10 (completely satisfied). The PWI is the average rating across the seven items (domains).

There’s the rub. The index arbitrarily decides the 7 things that makes us happy and weights them equally. So there’s a big danger of circular reasoning: we decide that community connection contributes to 1/7th people’s happiness, so we ask them about how happy they are with their level of community connection. Then we tally up the results and, wow, people who rate their levels of community connection satisfaction low tend to be 1/7th less happy.

Here’s a suggestion. People in the inner city of Sydney don’t care too much about connection with community, at least in the traditional sense of having a chat to your neighbours over the back fence. If they did, they wouldn’t live in a fairly anonymous city of 5 million people. But the study decides that, for everyone in the country, community connection makes up one-seventh of our happiness.

Further, if you look at the results across different geographical areas, they’re actually all reasonably close together. Grayndler, the lowest in the country, scores 69.43 and Wide Bay, the highest, scores 78.55. Here’s another thing: because happiness with health and relationships are fairly randomly distributed across electorates (with some notable exceptions), these don’t contribute much to differences in the index. Which just increases the emphasis in the index on things that do differ, like ‘community connection’ and ‘safety’. Satisfaction with safety contributes another 1/7th to one’s subjective happiness, according to the survey. And, again, perceptions of safety are naturally going to be lower in ‘the big city’.

Now it may very well be that community connection does correlate well with happiness on other measures – but this study can’t tell us that, because it starts out with that assumption embedded in it. So drawing the conclusion from this study - as the Age and SMH do - that connection to community is an 'important' contributor to our happiness is completely invalid. It contributes exactly 1/7th to our happiness - because the survey decides at the outset that it does.

Now, I think surveys like this that move away from our obsession with GDP are a good thing, and I think this survey makes an important contribution to our understanding of happiness and wellbeing. The survey report contains some really interesting analysis of things that are correlated with this particular measurement of happiness.

But you also have to recognise its limitations and the most important one is that when you have a survey that has arbitrarily chosen questions, categories and weightings, you introduce arbitrary biases and you have to be very cautious of how you interpret the results.

No doubt the researchers are aware of this. But the media pundits who want to use it as a geographical scorecard aren’t. So let’s also be very cautious of their analysis.

3 comments:

Anonymous said...

Thanks for setting me straight on this. Here's a trackback: http://imaginingaustralia.blogs.com/imagining/2006/02/glum_research.html

pedaller said...

You have raised a good point regarding community connectedness, except that I suppose if you don't feel connected to you community and it doesn't bother you then your satisfaction would still rate well.
Do we know how the sampling was carried out? Were approximately 180 people sampled per electorate, or were more people surveyed in one electorate compared to another? Was there a geographical bias within each electorate? How were those surveyed chosen?

Veggie Friendly said...

Interesting point about the bias in the survey. After reading your post it also struck me that it probably undervalued the importance of different types of relationships. For example, I'm sure that people could have different relationships with their family, their partner (if they have one), their friends, and for plenty of people each of these three sets of relationships could have a bigger impact on their happiness than their community connectedness. Yet these are each giving a max weighting of 1/21 - assuming that there are no other relationships like work / client thrown in there.

So what do you think would be a better way to test happiness and contributing factors to happiness / unhappiness? Free answers? Different factors to choose from? A combination of both?