Can we trust happiness surveys? Is my 7/10 = your 7/10? Is going from a 2->3 = to 6->7? While researchers assume yes, the evidence is scarce. @HappierLivesIns, we ran a 50-question pilot survey (n=128) (https://www.happierlivesinstit... Researchers: are we asking the right Qs?🧵1/10
Context: in analysing subjective wellbeing (SWB) questions, researchers often assume that self-reports are: 1. comparable: my 6/10 means the same thing as your 6/10 2. linear: that going from 4/10 to 5/10 represents the same size change as going from 8/10 to 9/10
Yet, evidence on these is very thin, and it would be a problem if these assumptions are false, because it would mean we'd interpreted the data wrong. They are the🐘 in the room. Here's what could go wrong with comparability:
And here's how linearity might fail
Another issue, if we want to trade-off quality VS quantity of life - crucial for e.g. healthcare - is: 3. 'neutrality', where on the wellbeing scale is equivalent to non-existence? There's been almost no discussion of this in the literature.
We've tried to think really hard about how to collect data that will test these assumptions. In the report, we set out lots of different methods and over 50 questions (see below). The aim is to get some answers and move the debate forward, so we're not just swapping intuitions.
We didn't have a big enough survey to test comparability. We do find evidence for linearity. We're not sure what to make of the answers re neutrality: people think 5/10 on 0-10 scale means zero life satisfaction but life is 'worth living' unless you're very dissatisfied 🤷♂️
We see these results as indicative only. What we REALLY want now is for SWB - and other - researchers to weigh in and tell us if we're asking the right questions. Once we're happy with the Qs, we'll run a much bigger survey - and hopefully address this 🗝️ scientific issue
Sorry if you hoped for the answers in this🧵. If can give us feedback that helps us get the answers, I would be so grateful! PDF: https://www.happierlivesinstit... You can email me (michael@happierlivesinstitute.org), reply here, or comment on the EA forum (https://forum.effectivealtruis...
Fair warning: these issues are complicated and the report is not light reading! Finally, thanks to my heroic co-authors who, as ever, did all the real work: Conrad Samuelsson, @Karakaai and @caspar_kaiser.
Oh, should said this about the feedback we're after 👇 And that @Karakaai (samuel@happierlivesinstitute.org.) is a better person to send comments to than me!
