So after posting yesterday, I found “Stumbling on happiness” at Barnes & Noble and read about a quarter of it (I didn’t want to buy another copy — I’d already got one for my mom — or I would’ve taken it home and read more); I hope I can get around to finishing it this week. He seems to be a smart guy, and the research he writes about is varied (so far) and fascinating. He does discuss the issue I wrote briefly about yesterday, that self-reported happiness data is likely to contain biases that are hard to disentangle from the desired results. So far there are two main sections on it in the book. In the first, a serious discussion, he uses some intuitively convincing examples to persuade us that often, when we might think different individual’s differing reports of how happy some event made them might be based on differences in the ways they use the language of happiness, they might instead be based in differences in the way past experience frames their present experience (if you’ve never been on a rollercoaster before, a ride will impress you more, or insert better example). I found this persuasive though perhaps not comprehensive. He takes a second crack on it that I don’t think works as well. This is just a snippet:
But if we gave away a million pistols and a million envelopes of money, and if 90 percent of the people who got new money claimed to be happier than 90 percent of the people who got new weapons, the odds that we are being deceived by the idiosyncrasis of verbal descriptions become very small indeed.
I.e., the “law of large numbers” means that if there are small errors in measurement they’ll average out and disappear in large experiments. But this is not true when the errors are patterned in the way I suggested yesterday; it’s only true when the error is distributed independently of the explanatory variables. This is not an insurmountable problem (just as econometrics is not a totally pointless endeavor) but it means, I think, that a lot of the time, results purporting to be about the experience of happiness may well really be about the language of happiness. This is not how they are typically billed.
That’s why controlled experiments simply compare the responses of one group to the responses of another…whatever oddities there are in scale usage or language should be the same for both groups. If something about the treatment variable makes us worry that maybe the two groups are using the scales differently (and this is rarely the case) there are ways to check. E.g., we can use extra-experimentally anchored scales (i.e. scales whose meaning is tied to an objective reference outside of the expermental situation) such as money. Although these issues are real and interesting, they jus arent that important to my mind because it would be VERY hard to interpret all the data on happiness prediction as a result of scaling artifact. SO this really ends up being an intellectual chess game rather than a meaty issue of deep interest…at least to me.
That seems mostly right, but even if the book focuses on lab research, it’s impossible to answer a lot of the interesting questions with controlled experiments: does being richer make you more happy (e.g.)? It seems tough to really capture the psychologically relevant dimensions of being rich in a lab experiment. Instead, you need to draw conclusions based on study of people who are rich versus those who aren’t, and that introduces the potential difficulties I discuss — there are uncontrolled differences, other than happiness, between the groups that could plausibly influence the way they talk about happiness.
Granted, there are lots of interesting questions that can be lab-tested and wherein (as you say) it’s not plausible to suppose scale differences (or similar) account for the results; many of those are discussed in the book (which I’m enjoying). I haven’t finished the book; it may be completely innocent of the error I’m harping on. (In fact, I now know the book focuses on systematic defects in what people expect will make them happy or unhappy, which seems like it may be immune from such troubles.) But I know that accounts of happiness research in the popular press are not so innocent.
Another question that’s not testable in the laboratory is the influence of religion on happiness; in my previous post I sketch how this might cause trouble.
Maybe I’ve misdirected my criticism. Anyway, thanks for the comment, especially but not only if you are who you say you are.
I don’t think I’m important enough to be impersonated. Hope some of your questions are answered when you finish my book. Regards DG
The University of Chicago law faculty is all over this:
http://uchicagolaw.typepad.com/faculty/2006/05/do_we_overcompe.html
(you can see me in the comments)
Shame on you, Dave, for not going to U of C.
Dude, did the author really post on your blog? How did he find it? Is he that bored?
And I’m surprised you liked the book. I thought it would exactly the kind of popular psychology that you hate so much. (cf. Professor Sanderson’s speech at the Senior Assembly.)
*would be