I’m beginning to think I might need to change the tag line for this blog. One of my earliest posts was about how we needed to apply some scientific rigour to the process of marketing attribution and therefore ROI. How can marketers be getting away with such unproven and unprovable techniques, spending all this money with so little evidence of success?

But I think I might have changed my mind. We’ve actually had a couple of geniuses at Red Gate looking in to the provability, or otherwise, of marketing ROI, and their conclusion? Nah – you can’t do it. More specifically, for our volumes, with our variation in spend (we’re not a company make a thousand sales a day, all at exactly $10 each), it’s not possible to show statistically that a sample and a control group will differ enough to be discernible at any interesting level of significance. I’ll get more details if they let me (as well as writing about the interesting point raised that – does this mean Google’s whole business model is based on FUD? On us be too scared to kill spend on things like Google Adwords? For another time…).

Anyway, I was thinking about this problem – that completely undermines this blog – and thought that actually there’s more to it than the maths. And that actually, trying to measure return on investment for marketing is very much like trying to measure return on investment for an employee.

There are many similarities between these two investments:

1. They are both investments. I.e. You’re spending money, either on Adwords (or whatever) or a salary. You would only be doing this if you expect to get some sort of return.
2. So far so simple. Next – In theory, both can be measured in terms of Inputs, Outputs and Outcomes. As I outlined in another earlier post, I try to measure campaign success in terms of Inputs (“Am I getting what I wanted? Does the advert look good and say the right thing?”), Outputs (“What’s the immediate KPI? How many webpage visits did I get? How many clicks on the video? How long did they watch it for?”) and Outcomes (“What’s the headline result – was revenue impacted?”). Similarly we can do this for an employee. Take a developer – what’s the quality of his/her code? How do people like working with him/her? (Inputs). How much code does (s)he write? How buggy is it? How often does it get rolled back from production? (Outputs). And – is the end product something people want and buy? (Outcomes).
3. Thirdly – there are trivial examples, I guess, of where this ROI is measurable in both situations. For marketing – you’ve never marketed ever in a particular country, say Micronesia, and never sold a single thing in Micronesia. You run some Google ads in Micronesia, and nothing else, then measure sales in 6 months time. I think you could get a pretty accurate ROI here. Similarly, if you employed someone to act as a consultant on your behalf, paid that person $100,000 per annum and, purely through having that person on your books, was able to bill $200,000 in consultancy work, then perhaps you could calculate an ROI here
4. But, I think these are trivial examples. In most cases, it’s almost impossible to measure the return on investment in hiring someone. Take for example a good product marketing manager, let’s call him Pete. Pete spends a lot of his time researching and thinking about positioning for a set of products. He often has great, innovative ideas and insight (though not always!). Also, he spends a lot of his time with customers understanding their backgrounds, needs, desires and issues. Furthermore, he works fantastically in a team, bringing cohesion and vision to what everyone else is doing and helping to keep those around him motivated. He also helps out with training sessions for others in the company on his areas of expertise, and also helps out with company events both internal and external.
   
   How on Earth do you measure the ROI of Pete? Undoubtedly he’s having a positive impact on his company, but how could you measure it? How could you assess the impact he has on motivating the team around him? Perhaps that stopped someone leaving because “They just love working with people like Pete”. How would you ever know how you saved $10,000 on recruitment fees through that? Or how his work at events helped bring new customers and employees to the company? And more specifically, how would you ever measure the impact of clever product positioning on sales? I’d say impossible!

So really, I’m coming round to the view that, though we might be able to measure Inputs well, both for campaigns and for people (“Is this a great campaign? Did we get it spot on?” and “Is Pete doing well? Is he producing great work, and a great person to have on the team?”) and we can measure some Outputs (“How many page visits did we get?” and “How buggy is Pete’s code? How productive is he?”), Outcomes and therefore accurate ROI are pretty much an impossible requirement.

So what does that leave us with? Well personally, I think it’s an issue both of using intelligent qualitative assessment of work and also some faith that high quality work based on sound judgement will have the desired effect. For example, when thinking of Google Adwords – if we’ve done our research and know that there are lots of customers out there with a particular problem; and we’ve written our search terms to reflect that in an intelligent way; and we’ve worked on the ads so that they really reflect how we know customers are thinking; and of course, we’re checking that the ads are working, and continuously improving them – then, I think it’s reasonable to suppose that this effort (and spend!) will produce some results beneficial to your company. Of course, you have to check the rest of the pipeline – when they’ve discovered you through the ad, can they try your product easily (Validation)? Do they get what it is (Positioning)? Can they buy it in a way that suits them (Pricing and Packaging)? But if these things are also done well, I believe the extra work trying to validate whether or not you’re getting a particular overall ROI from Adwords is a hiding to nothing.