Measuring up
I managed to get invited to a OHM event at the Saatchi Industry Lab in London on 28th November 2007 where the keynote speaker was Douglas Hubbard, inventor of Applied Information Economics (AIE). Douglas spoke about How to measure anything (which is also the title of his book).
These are my rough notes..
An investment is a cost now with a benefit later.
First of all he defined measurement as an observation that results in reduction of uncertainty about a quantity (that is a measurement has an associated error bar)
Baysenian – each bit of data updates the measurement.
Bookies are great at assessing odds subjectively, Doctors are terrible!
You can easily train people, by calibration, to assess odds (decision psychology)
Computer managers are over confident – use too narrow a range in general – costs were measured more than uncertain benefits, small hard benefits measured more than large soft benefits.
Biggest risk decisions have least quantitative – only measure what they know – no benefits if no one uses it!
Once we determined what to measure we can think of observations that would reduce uncertainty.
Nike method – JUST DO IT – Don’t let imagined difficulties get in the way of starting observations.
Difficult – compared to what?
Ultimate non-random sample – their own experience.
“Its amazing what you can see if you look” Yogi Berra
- Its been measured before.
- You have more data than you think.
- You need less data than you think.
- Getting more data is more economical than you think.
- Probably need different data than you think.
- Your opinion about potential measurement of errors also has a lot of error!
Measure 13 things take 4th largest and 4th smallest you have 90% confidence in the answer.
A very interesting and thought provoking speaker especially with the audience participation in getting people calibrated when ‘guessing’ quantities in bottle etc. He gave a few plugs for the TacAdvisory group who he does work for.
Still not sure how I came to be invited, guess they thought I was some high flying University Don!