Ed Mitchell: Platform neutral

Community ROI Report

Forum One have released another of their very handy online community survey reports, this time about Community ROI. It’s worth a download and a read. I’ve re-published some of the responses here after a bit of personal experience, but recommend the whole report.

Many of us are wrestling with emergent community metrics and their meaning in this time of increasingly distributed community activity. Since doing my MSc IT thesis in this subject in 2002/3 (’Are organisations who are investing significant money into websites measuring the activity of those websites, and are they using the data those websites produce to assist their business decision making?’), it seems not much has changed; to boot: they aren’t (although editors and IT bods were very very keen on web metrics, the business types hadn’t got to grips with them).

I’m hearing a lot about The ROI of communities and it reminds me of the baying for the ROI of ‘Knowledge Management’ from the commercial members of organisations as the KM-ers tried to introduce KM stuff some years ago. Was it possible? Did the KM-ers convince the money folks?

Here’s one question I’d like to explore: ROI for whom?

So the hosts put money into something emotional and human and weird and unpredictable (like a community) and demand a financial return on investment. Apart from that being very hard, it’s a bit one-sided isn’t it? In my years of community facilitation, the members put a lot of ‘I’ into the communities, but this is never considered by the hosts’ commercial teams: the communities wouldn’t exist without the members’ investment.

Have we considered the members’ ROI? But I’ll get to that in another post.

Either way, one of the elements of the forthcoming blended facilitation work I’m doing for The Media Sandbox will include this (I’m particularly excited about this work and will write about it very soon) so it’s, again, at the forefront of my mind. As well as facilitating and then freely disseminating as much community-generated knowledge using free software as possible, we also want to look at how to measure/understand the reach/range/engagement of people with that knowledge.

Steve Bridger and I explored this for our work with Amnesty, and we’ve been musing on an online community dashboard thingameejig since meeting some years ago which Steve is also taking forward as part of his ongoing ‘buzz director‘ idea. It’s a two part gig: 1: capture data, 2: report effectively into useful management structures.

The communities team discussed it a lot around the CILIP membership communities, and identified a clear need to (a) collect the right data based on a the proper research question, and (b) build into practical business reporting procedures. The problem was getting to the question to start with (you may not be surprised to hear).

So we’re seeing some commonalities from work done so far:

• Need a clear strategy first that balances the ROI for the sponsor with the ROI for the punters
• Need to define some measurables to help you keep track of that strategy
• Need to assess how the data your ‘thing’ generates and how to map that to your measurables
• Need to build a regular, easy reporting schedule into your organisation’s reporting schedule
• Need to have time to do this too - it’s not just a bolt on

And onto some of the highlights of the OCR ROI report:

Question 6:
What kinds of online community activities does your organization engage in?

Summary:
Not surprisingly, discussion groups were the most popular online communityactivity, followed closely by blogs, support forums and social networks. Video sharing, wikis, and social bookmarking were deployed less frequently by survey respondents.

Question 7:
In general, what is your current organization’s predisposition towards online community investment?

Summary:
Overall, the survey results indicate a fairly high tolerance for investing in online community activities without clear “hard numbers” ROI. As we will see later in the survey results, dimensions of value other than fiduciary are being accepted as “return” on community investment and involvement. However, the majority of respondents did say they were expected to communicate clear return in the future. Creating a clear ROI model for most organizations is clearly a priority, even those not under immediate
pressure to communicate value.

Question 8:
What is your current attitude towards your organization’s online community initiatives?

Summary:
A relatively small number of respondents were confident in their ability to tie community initiatives back to their corporate goals, and to clearly communicate ROI. The majority of respondents feel their initiatives are adding value, but can’t provide a complete ROI model. A small percentage of respondents feel their initiatives are disconnected from corporate goals, and they currently don’t report on value. This again speaks to the need for most organizations to create an ROI model, and one that includes more than direct financial value.

Question 11:
Which of the following dimensions of value do you regularly report back to management and stakeholders?

Summary:
Not surprisingly, page views, unique visitors and new memberships topped the list of value points reported back, as they are the most accessible metrics. A fair number of organizations are reporting on engagement and loyalty, which speaks to longer-term relationships and the perceived value of customer advocacy via the community. It’s surprising that call avoidance and cost reduction ranked the lowest on the list, as these were points of value that have been articulated for years with regard to support
discussion groups. Direct revenue per member was also surprisingly low.

Question 12:
What other dimensions of value do you regularly report back to management?

Summary:
The write-in answers concerning other dimensions of value proved to be a rich source of data. Highlights are summarized below. Please note that answers that duplicated selections from the question 11 were not included.

• • Demographic data: particularly member profile data is an under-analyzed and underreported source of value, especially when you consider the cost of obtaining this data by other means, like consumer research.
  • Member vs. Visitor Activity: Monitoring and reporting on member activity vs. nonmember activity can help benchmark and track the relative value of community members.
  • Comments / Ratings: User comments and content creation are valuable in several respects, including steering editorial direction, getting product feedback, and engaging the community ecosystem (offsite) in a broader conversation.
  • Lead generation: Lead generation is an established metric for most marketing activities. This is an easy metric to compare cost to other lead generation activities, like trade shows and print advertising.
  • Member Content Creation: Member-generated content value can usually be compared to and valued by the cost of internally produced content.

Question 13:
How often do you report back to management on ROI?

Summary:
The majority of respondents indicated they were reporting back to management monthly given that the majority of respondents also indicated they were reporting back on relatively few points of value. There were also a surprising number of respondents reporting back weekly, especially given the overhead of manually gathering and compiling ROI data.

Question 14:
From your experience, what are examples of effective strategies and tactics for making the case to management for investment in online community activities?

Summary:
Key strategies and tactics for making the case to management for community investment were:

• • Report Direct Revenue: Report on direct revenue from Community Members.
  • Establish Comparative Costs: Compare cost of the same result or activity in other media, like blog post comments vs. the cost of hosting a focus group.
  • Communicate Cost Savings: Cost savings by hosting community, including support call deflection. Word of mouth vs. traditional marketing vehicles.
  • Compare value of Member vs. Non-Member: Track and compare activity of subset of community member vs. non-member activity, including activity outside of community.
  • Highlight the Direct Connection: Communicate the disintermediation opportunity for businesses, especially if your business is not directly selling to customers. Their online community creates a connection with the customer during and after the sale being handled by channel.
  • The 24/7 Focus Group: A subset of your community will be giving real-time feedback on initiatives, products and company activities. “Human voice compliments the numbers.”
  • Evangelist Creation and Mentoring: Communities attract your biggest fans and give them a place to connect with others, in the process creating new evangelists.
  • Communicate the Cost of Not Being Engaged: Including increased marketing spend, and the threat of competitors engaging prospects and taking existing customers.
  • “Value for all Departments” Communicate the broad value back to the business, including:

  • • “HR - Recruiting talent through fostering relationships with engineering interns and other prospects
    • Customer Service - Tie in to the KnowledgeBase and help users helpthemselves and others
    • Sales - pre/post sales, online
    • EBC (executive briefing center), lead gen
    • Marcom/Marketing - brand awareness and loyalty

And here are some brilliant detailed responses to the strategy question:

Have both quantitative and qualitative data to show off both numbers and community member interviews. Tie feedback to critical company initiatives. Compare community activity to traditional corporate activities: example blog readers/comments vs. product web page or direct mail responses.

In our case community is one of the products we offer. The page views it drives are key to maintaining our leadership position. Showing how rapidly communities grow, the value of content they create, the level of satisfaction among participants and how evangelical they are about our site has helped. We do this with traditional metrics and regular surveying of the audience. We’ve also had success showing how insights from community members drive growth throughout the rest of our business. It’s a 24/7 focus
group. You just need the right folks listening…

Question 15:
What advice would you have for a colleague that needs to quantify and report on the return on investment of their online communities?

Summary:
Common threads ran through responses to several of the survey questions, and these themes emerge again in this question about advice for a colleague. Of particular note were suggestions to:

• Establish an ROI Model: Establish an ROI model, ideally by understanding costs, revenue, indirect revenue and value unique to the community experience. If you can’t put together the “big picture”, establish some dollar value to the quantitative metrics you do have, like the value of page views on your community vs. the same page views on your main site (if applicable).

• Additional metric suggestions include:
  • Value of Member to organization “measured in number of posts, number of friends, or some other contribution made”
  • Value of member-to-member activity, based on overall content and satisfaction ratings.

• Evangelize Your Community Efforts: Provide metrics, clear examples of ways community helped business goals, engage executives with active links to examples of community at its best, circulate industry reports and best practices, use community tools (don’t just talk about them).

• Know the Metrics you Need: Know what community metrics you want, based on business goals. Understand how you will use those metrics to communicate value.

• Ensure You Can Get Your Metrics: Ensure that you have the proper infrastructure to get the metrics you need. Community platforms and service providers rarely have the exact set of metrics “out of the box” your business requires. Cookie-ing all visitors to the community will help establish value of member vs. non-member.

• Additional Advice includes:
  • Check membership data against key wins and big customer data (tie in)
  • Cost reduction, especially with regard to research.

Thanks again to The Online Community Report team for sharing their findings and helping us all knock these ideas around and share our experiences.

Here’s the link to the report again: Online Community Report: ROI, April 2007