Tuesday, February 14, 2012

The low down on statistics and surveys

As the strip shows, results from such surveys can be interpreted in different ways. The reason is because such surveys (or "studies") only show association rather than cause and effect.
Sometimes, the interpretation fails to consider tertiary conditions or a larger perspective. To give an example bordering on the ridiculous, all murderers pee but it doesn't mean that peeing results in murderous tendencies although if you stand in the way of someone looking urgently for a restroom, you may get different ideas. Likewise, it is also conceivable that a survey may show more smokers eat more ice-cream. This may not mean that smoking causes a craving for dessert or vice-versa (unless it's nicotine flavoured ice-cream. Thank goodness the industry is still above this.) Rather it may just show that both are choice stress relievers for many people.

A recent "study" by SMU Psychology Professor Norman Li shows that Singapore women are more materialistic than their American counterparts while men on both continents went for looks first. (http://sg.news.yahoo.com/blogs/singaporescene/pore-girls-materialistic-smu-study-092109660.html; and http://www.smu.edu.sg/news_room/smu_in_the_news/2011/sources/ST_20110615_2.pdf) Now before women join hands to go up in arms (while others secretly visit surgeons) and some Singapore men toast each other in agreement, a closer look at the study will reveal much. Actually, even a casual look will do. The study surveyed "400 psychology students aged between 19 and 21, studying at the SMU and the Northern Illinois University near Chicago in the United States, on their attitudes towards marriage, children and preferences when it comes to choosing a partner."

So perhaps a more accurate conclusion would be: young SMU female psychology students are more materialistic than than their NIU counterparts while young male psychology students in both varsities are equally superficial. After all, the last time I checked, SMU psychology students aren't representative of our population. And in view of the much recognised fact that self-reporting is one of the most inaccurate methodologies, we may wish to add "self-alleged" somewhere in there. But otherwise, I'd say this study is worrying -- it seems our future psychology workers are either materialistic or superficial. Perhaps the study went through because it largely corroborates conventional results from other surveys. Men tend to filter subjects by looks first and women by power & status (ie security factors) when looking for partners. But again, as far as I know, most of these studies also take the form of self-reporting surveys. And of course, we know that we are always honest when talking about ourselves and we always know ourselves best. That's why some live in mental insitutes and see psychiatrists.

What does this have to do with the library?

Well, how are customer satisfaction surveys done? They are usually done by interviews, a methodology which is just a step above self-reporting surveys -- or not, depending on how trained the interviewers are. Otherwise, we have the bias of the interviewee filtered through the interviewer's lens.
And how representative are the people interviewed? Well, I often received results based on a survey of 3-4 people. In this case, it should really be treated more as feedback than a generalisable finding. And in the past, I had feedback about children programmes given by adults... right.

The other thing we can learn from the "study" is on measuring things. For example, what did the study conclude as materialism? It defined it as looking for social status. The last time I checked, status and material wealth can be closely related but not equivalent. One only needs to look at Mother Therea, Gandhi (later in his life), and many non-Singapore politicians. And conversely, look at the many secret millionaires we don't know about? Who are they you say? Didn't I just say we don't know?

Now how do we measure readership in the library? The primary measure is loans. The two are very closely related but not the same. I would say it is analogous to Newton's and Einstein's laws of physics. It is now widely spread (alright, only among science geeks) that Newton was wrongly thought to be right till Einstein came along. It is obvous that Einstein was smarter because he was uglier and we all know looks + brains = constant. The closer truth is that Newton hit upon a very close approximation of the truth (or what we currently think is right) -- so close that on Earth, it is much simpler and better to use his computations. So unless you are migrating to the milky way or negotiating at light speeds, Newton would pretty much be a worthy guide.

So for us, the question is: when does loans approximate readership? Logically, loans would be consistently lesser than readership (some may browse but not borrow, other may read non-library materials. And assuming humans are creatures of habit this difference should be fairly constant - under normal circumstances). Thus loan trends can mimic readership trends. However, when we throw in loan promotions, this premise can fail -- mainly because we are promoting just loans. Most of us will have encountered people who keep buying McDonald's meals to get the full set of toys - perhaps not because they like the food but because their child likes the toy. If the library equivalent exists, this is when loans deviates from readership and no longer mimics its trends. Part of it now measures greed or obsession.

The upshot is that this also skews our collection statistics. Previously, we may use borrowing trends as a guide to reading preferences. But under such circumstances, it may become more of a guide on what some people can borrow when they don't really want to read. Thus my take is that a poorly conceived promotion is akin to using television to nanny your kids. Effective in the short run, destructive in the larger scheme of things.

The borrowing trends on a poorly developed or organised collection can also exhibit a bit of this anomaly. It may reflect instead what people borrow when they can't find what they want but don't want to leave empty handed.

How can we circumvent this then? I'm taking this to the grave with me.... no seriously, it is simplyto start with a good norm or failing that do constant development and analysis of the statistics to monitor changes based on the development.

But of course, this is also an oversimplification of the situation. Or overthinking. After all, most people end up doing the same thing anyway.

No comments:

Post a Comment