This is a cautionary tale about interpreting people by data alone. I have sometimes told it to clients and they have found it helpful so I’m going to share it here too. It’s also about Bermuda shorts. I’m aware as I get older that some of you whippersnappers born in the 90s (is that legal?) may not be aware of the fashion tragedy that were these hideous neon clashing geometric print garments, so I’m going to make a huge sacrifice of dignity here:
Yup, this is yours truly. In my Bermuda shorts. And in the 80s they were everywhere.
Actually this is me on holiday, in a Caravan park, probably about 1988. Every photo in my album from that year, bar a Christmas day shot and a school play, has me in Bermuda shorts. So, clearly, despite their obvious hideousness I loved them. I mean, they probably had to peel them off me to get me in my costume for school. I bet I had them on underneath. Out of 18 photos of me in this year, only 2 were with no visible Bermuda-ing. Relly x Bermuda 4 EVA.
Except, here’s the thing. I bloody hated Bermuda shorts. Not in retrospect, I mean that I hated them at the time I wore them. I especially hated the pair above. I think my ‘arms pulled down in front of me’ stance was probably in some vain attempt to hide their hideousness. Even the dorky cardigan knitted by my mother seemed a better bet. So, if I hated them so much why am I wearing them so often in my photo album? This is a question of choices and context.
1) I was 8. And 8 year olds in the 80s didn’t choose their own clothes. They got what their mothers bought them, like it or not, and at this time if you wanted shorts you got Bermudas because that’s what there was to buy. And when my mother came to pack the suitcase for our annual holiday I needed shorts.
2) All my pictures were shot on film. My parents were not photographers. They bought around 3 rolls of 24 exposures a year. One for Christmas, one for holiday and one for their kids’ achievements – plays, assemblies, choir trips etc. It should also be noted that my dad was exceptionally gifted at getting his thumb part way in front of the lens and never remembered to turn the flash on when it was dark. This meant that a disproportionate number of shots of me that made it into my book are taken outside, on a sunny day, on holiday. The conditions were optimal for my dad getting a picture of me and optimal for me to be wearing the aforementioned loathed clothing. The reason for taking various photos, ‘I should get some photos of my daughter to remember her when we were 8 and on holiday’ or ‘I should take some photos of my daughter while she is 8 and singing in a choir inside on a wet day’, were essentially that same but the context dictated whether my dad would be successful in getting a photo that you could put in my book.
Why do I bring this up? I have talked about my Bermuda short problem when clients are looking at analytics and data they have taken and are using that to try to work out what their users are doing the most/least/longest/fastest etc. It is easy to see the data form a beautiful line and get excited. I try and encourage people to look at their content and its related data and ask
What choices do your users have? Are they truly engaged or are they doing this because they have to – you have offered them no alternative? Have you packed their suitcase with ten pairs of horrible shorts?
What is the best context for success? How could you create content that works in other contexts? Is one context blurring your requisites for success by being naturally easier to work with? Can they take a picture on a wet indoors day as well as a sunny outdoors day?
Is the way you are capturing your data introducing bias? Is self-selection an issue (ie, with surveys)? Is one topic wildly more popular than another but measured on equal terms? Are you only taking photos three times but measuring a whole year’s outfits by it?
None of this is revolutionary thinking but I have found the Bermuda shorts analogy has helped people understand what I mean by different interpretations of the same data – data without context, and data with. To measure successful content, you need data and context. Otherwise, you are assuming that doing something a lot means that your user likes it.