From April 7th – April 10th, I’ll be attending and speaking at the eMetrics Marketing Optimization conference in Toronto. These are my thoughts, feelings, and (hopefully) epiphanies from interacting with the world’s best and brightest in the world of digital measurement.

WEDNESDAY – APRIL 7TH – 9:35 PM

eMetrics day one. First off, the conference director Andrea Hadley did an awesome job putting things together. As with any conference, the true direction of thought leadership happens over dinner (not on stage), and sadly my inbox had to take priority this evening. So at this point, all I can offer is a quick rundown and a few opinions from the first sessions of the day.

Kickoff

After Andrea gave a quick intro, Alex Langshur, President of the Web Analytics Association, talked a bit about the WAA. A good summary highlighting the emphasis on community beyond Web Analytics Wednesday, legislation of privacy information, and education. I’m a big supporter of the WAA’s agenda, so it was a great way to start.

NHL.COM

Theresa Locklear, the director of Web Analytics & Customer Insight talked extensively about the reporting practices of the NHL. I was impressed by their degree of transparency – sharing numbers, talking about real case studies (albeit, mostly wins), and highlighting a few examples of real life optimization success. She also showcased an analyst team bigger than many (five to be exact). Clearly the NHL sees value in data driven decision making.

But at the same time, something kept popping into my head. There were quite a few examples of web optimization in the presentation, and when I asked about testing, they said they did none. In my mind, that means the decisions are largely left up to heuristics. Can’t black box that.

Qualitative vs. Quantitative panel

I’ll be honest – I caught what I could of the panel. But I’m trying to buy a house right now in Toronto, and in this market when the phone rings, you answer. But I did see enough to have a few observations.

A. I’m not sure I fully understand the logic behind having five people with (or formerly with) “Analyst” or “Consultant” in their title debating the merits or uses of qualitative data. I think we lost an important perspective by having no pure qualitative researchers representing their field. Back in my old web days, I used to think anyone could do qualitative research. When you work with PHD’s in the subject, people trained in observing and interpreting human behaviour, you recognize that there’s a difference. A few discussions at the WAA Mixer after the session reinforced my thinking.

B. When did “Quantitative” and “Business” become interchangeable? As the discussion progressed, it felt like a bit of a B-School / D-School discussion – not in terms of one versus the other, but in terms of purpose. Research is used for a lot of reasons, and yes, in some situations one is preferred over the other. Tonight felt rigid, like every situation was an either/or but not a both. Marko Hurst was the exception who I thought represented both disciplines well.

So tonight was largely a technology, mathematics, and alienation free evening. A positive in my eMetrics journey. Looking forward to a big day tomorrow.

WEDNESDAY – APRIL 7TH – 7:54 AM.

Today is my start of the eMetrics Marketing Optimization conference in Toronto, an event I am simultaneously looking forward to and dreading. On the “looking forward to” side, there are many, many awesome speakers presenting. From Avinash Kaushik to Jim Novo (a hero in the space) to my always entertaining former co-conspirator Christopher Berry, it’s an all star lineup.  I’ll also be presenting the numbers behind the Aviva Community Fund, a marquee project for Idea Couture in the social innovation and cause marketing space.

On the “dread” side, I can’t help but feel this conference is going to be the end of my formal association with the digital measurement community.I’ve been a practicing Web Analytics analyst for about 6 years now – an old guy in digital terms. But even in that space, I always felt a bit like an outsider as a user experience strategist (researcher) first and an analytics practitioner second – a divide that is becoming an argument of validation versus prediction.

Meanwhile, many of the dominant conversations are distant and foreign. As Web Analysts, we focus our efforts on what will derivate the largest impact. Usually that means optimizing the user experience around the highest revenue-generating opportunities. Rather than focus our efforts on any one individual, we analyze patterns because it’s a better use of our time. But it seems as though we’ve lost our own way – now focusing in on technologies and methods designed to derive actionable results around strategies that most clients can’t afford to build in the first place.

Over the past six years, I’ve seen an industry migrate technologies, define definitions, formalize education, and find its way onto the book shelves of many of my friends (a big win when you feel you’re in a niche space). Our current direction has me concerned that everything’s about to get effed up.

For the rich few, yes, these new methods and technologies can (and will) create a competitive advantage. But for the poor greater, this only further creates barriers to entry – introducing alienating languages, foreign skills, and high costs. I’ve heard many people in this industry talk about how wonderful Web analytics is because it’s accessible to marketers and (with a little help), it can be understood all the way to the top. Because of this, there is power in our recommendations.

While I can’t be certain yet, I get the sense that digital measurement is going the way of the great black box – a fifedome for a chosen few to understand. It’s also focusing on things most clients simply can’t resource against. While, yes, the decision support may push us towards greater accuracy, we can’t forget the politics behind all of this. I’ve been in enough Marketing Business Intelligence briefings to know (whether right or wrong) who makes the final decisions. In the world of mixed quantitative and qualitative data, whoever tells the best story usually gets the most attention. Black boxes don’t make for good stories.

If you ever visit MIT, you’ll see that their floors look antiquated and cracked. It’s because, in many cases, the hallways of MIT were  where the building materials and construction methods were first developed, and they want every student to now, and remember the origins.

Evolution and revolution are both needed, but that doesn’t mean we should ignore where we came from. I’m hoping as the conference moves forward I see a little humanization in the midst of our attempts to automate human behaviour.

I have reached a point where I am (gasp!) considering making the Montreal Canadians, and not the Leafs, my favourite hockey team. I was born and raised a Leafs fan….so this move is serious business for me, and comes after prolonged deliberation. It is not just the performance of the Leafs– any team can have a bad season or two (and somebody has to be at the bottom of the league), but rather, it is my perception of value associated with the team, which is driving me away. I should also say that I am a ‘casual’ fan, so I welcome any errors in my logic below from those more knowledgeable than I.

In the economics of sport, it is generally in the best interest for any league to have a ‘balanced’ level of competition; that is, the outcome of any game is in question, and any team can be considered a contender for the ultimate goal of championship. This allows each team to promote itself as providing a high level of quality of play, and therefore, a high level of entertainment for consumers. In the case of the Leafs, they have not won the Stanley Cup in my lifetime (not since 1967!) and generally fail to come close. This essentially means that critical elements of the brand promise are simply not being met, nor have been met for a long time. We fans would like to think that our team is a contender- that our community of fans may have the opportunity to watch our team in the playoffs (or, even better, participate in a great street party once the championship has been won!). Sadly, it is been a long time since ‘consumers’ of the Leafs have been able to enjoy this pleasant daydream.

This performance is strangely coupled with a serious imbalance of supply and demand (the Leafs sell out every game), means that it is very difficult for non-seasons ticket holders to get seats. Certainly, one can engage in the ‘valuable’ services of a scalper, but this means taking a family of four to a game should run somewhere in the $500 range for rather poor seats…..and as such, the value of the experience is very much in question. When I think about the vast array of entertainment choices for a night out and the value associated for each, the Leafs are no longer an attractive option, especially if you are part of a large family.

Thinking about all of the factors in my relationship with the Leafs, and the performance of the team in the league, this has an impact on the satisfaction I have with the team in general. For most businesses, we would expect that solid product or service performance leads to satisfaction, and highly satisfied customers tend to be more loyal. Over time, this loyalty translates to more money spent on the organization as customers return for more of the same product or service again and again over a length of time, meaning that an investment in improving the customer experience can ultimately be very profitable. Therefore, it is very much within the interests of the Leafs to perform well, and keep me a happy purchaser of tickets and merchandise.

However, it is a question of both customer satisfaction, and more importantly, loyalty, which forces me to reassess my relationship with my favourite hockey brand. Normally, we would use a number of measures in our research to understand how a customer feels about a product or service experience; overall, I would consider my ‘satisfaction’ with the team is fairly high: I have a lot of history with the organization, and from what I can tell as a casual fan, it looks like the right kind of effort is being made by the individual players. However, the perception of value is quite low….as is my ‘likelihood to continue use’ (the idea of loyalty is very complex, and cannot be captured with just one measure).

To further complicate the matter, multiple studies in a number of industries have shown that customers in the middle range of satisfaction are just as likely to defect as those at the ‘highly dissatisfied’ extreme of the scale. This essentially means that the Leafs (and other service organizations) cannot rely on customers who are merely ‘satisfied’, but rather, needs to push consumers into the realm of ‘delight’ in order to shut the back door of defection. In the case of my relationship with the Leafs, I am pretty far from delighted.

Not sure if you heard, but there was a recent move by RIM (blackberry) founder Jim Balsillie to buy the Phoenix Coyotes and move them to Ontario- possibly Hamilton.  Ultimately, this move was stopped by the NHL, with the league taking the position that ‘the right channels were not followed’.

The issue which was not talked about is that the Leafs are the proverbial golden goose for the NHL – the most valuable team in the entire league. Team value, revenue, operating revenue are all significantly higher than a decade ago…while player costs remain flat. Plus, an average (pre-scalper) ticket price of $97 dollars in Toronto…..versus $55 and $67 for the second and third most valuable teams (New York Rangers and the Montreal Canadians respectively). The Leafs have raised ticket prices by 3.5% this year, and have the highest ticket price in the entire league. To me, this is the equivalent of a Honda Civic having the same price a Rolls Royce.

Hence, I wonder if the move of the Coyotes was really stopped to prevent the legions of dissatisfied/merely satisfied consumers in the GTA a choice of where to spend their hockey dollars, and preserve the artificially high revenue for this team, (revenue that appears to have no relationship with the actual performance of the organization). There are reasons why the league would do this; certainly to appease the voice of such a powerful stakeholder as the owners of the Leafs (primarily the Ontario Teacher’s Pension), but also, the league revenue-sharing program in place means that some of the substantial revenue of the Leafs is shared with teams that are operating in the red. I however, feel that the pain caused by the organization is now greater than the barriers to defection…..so away I go. Vive les Habitants!

At first glance the numbers themselves may seem small, but keep in mind that these are two sites that have tens of millions of users per day. A tiny change in page rendering speed, can have a pretty significant impact over time, especially when your core business is search.

What can we learn from this?

• Speed matters (the obvious one)
• Costs of delay increase overtime and remain persistent (you’ve lost users)
• What you show a user, is more important then it’s actual size

The last point is interesting. What that essentially means is that, it’s important to consider how a page is rendered, and optimize or use techniques, that let you decrease the perceived loading time.

Performance optimizations are important. They are typically neglected, not budgeted for, or done as an after-thought, which can hurt in the long run. However, it is foolish to assume that a particular site can be fast just by doing a couple of tweaks. On the contrary, optimization is a methodical process, which takes small but several aspects into account, which together can bring faster performance and user experience responsiveness.

If design is about critical thinking and problem solving, then there are plenty of other tools. Designers can look to technology, nature or even everyday objects for inspiration. Or should design school start using Lego and Playdough? With just a few basic tools ( just craft materials) people could actually start to think about things from different points of view and be creative. My personal toolkit includes dozens of Playmobil soldiers. I used that for my competitive strategy simulation.

What is a designer? Is it someone who went through applied arts training in Pratt or Arts Center or St. Martin? Is it someone who is trained as an architect and decides to apply his or her skills to problem solving? It is required that a designer’s works are both objective and subjective. The objective requirements are easy to understand, they are technical and typically driven by business requirements that allow for measurement and direct comparison. What is the best way to build this? What is the balance between form, function and performance?

In the last few years we’ve seen everything about the way products are designed, manufactured, and sold be destabilized.  This is the first time in a century that we’ve realize we need to rethink products and sustainability. Great industrial product designs are supposed to be timeless classics, does that mean anything to anyone today? The core virtues of fine industrial design—safety, usability, convenience, serviceability, utility and solid construction.  Name me some classic design these days?

Then there is the subjective, creative side of design that’s difficult to explain and hardest for most people to understand. Some “designers” hide behind the curtain and spin and spin, hoping they will come up with an idea. Others work in a collaborative fashion and bring out the best in others. Others think they are designers but hardly understand the basics of design, thinking that having reasonably good taste equals design. Sorry no.

The aesthetic side of design relates to fashion, human behavior, emotion and cultural influences such as the cultural meaning of symbols. Designers are immersed in the visual (and increasingly audio and other senses) language of their culture and industry specialization. Not only making things look pretty (that’s important too) but also bringing human and cultural values to business problems, humanizing a product or service and turning a corporation into a trustworthy entity.

Image Source: http://farm3.static.flickr.com/2420/2237674772_62e30f699d_o.jpg