Flickr Photo: Andrew_NWe had a staff meeting at NTEN a few weeks ago.  In addition to seeing a bald eagle while kayaking, we brainstormed our ideas for 2009, and started translating those ideas into goals for the year.  I've been chewing on those goals as we finalize our plans here.  I have this unsettling feeling that the metrics we're setting for ourselves are adequate, but not outstanding.  In short, I wonder if we're measuring what matters.

At the same time, I spent the last two days with some ridiculously smart, funny, and inquisitive folks talking about social media.  These were sophisticated nonprofits, and as we discussed what social media might mean for their organizations, they zeroed in on one of the most interesting topics in the space: how do we measure this stuff?   It seems that measuring what matters is in the water.

When most of us think about measurement, we try to draw tidy lines between cause and effect, and we make one basic assumption: more is better. If you have a dashboard at your organization, it's probably a lot like the one I use.  It tells you things like:

• Total unqiue site visits
• Dollars raised/earned
• E-newsletter open rate
• Number of people in your community
• Number of event registrations

When these numbers go up, we feel good.  When they go down, we work to get them back up.  Bigger, faster, more: that's what we strive for.  But those aren't the metrics that really tell us if we're engaging our audiences, or creating social change.  Measuring impact is more complicated than that.

I'm going to give you one example Daniel Sinker shared during our chat yesterday.  Think what you will of the Drudge Report, it's got a huge sphere of influence.  The reason it's influential?  It's not the site traffic (because it doesn't get the most traffic overall).  It's the fact that Drudge's visitors are RABID consumers of his content.  The don't just read his content, they devour it.  They come to his site A LOT.  And they STAY on site.   And when they're done, it's a good bet they pass it on. 

If you looked at your website in the same way, as a way to ENGAGE visitors, how would you stack up?   Here at NTEN:

• We're having record traffic overall, with 21,000 visits in the last month.
• 69% of our visitors in the last month made just one visit to our site.
• Average time on site for nten.org visitors in the last month was 2:17.

Looking at just site traffic, you'd think we're doing really well.  Looking at how we're engaging those visitors, you can see we have some work to do. Let me give you another example from NTEN, this time focused on program.  This year,

• 2,899 members have registered for an event (NTC, Webinars, Ask the Expert).
• 4,535 individuals receive member benefits from NTEN.
• That's an average of .6 events per person.  

Our mission is to help nonprofits use technology to change the world.  We think that if you attend an event, you'll learn something to help you do just that.  It's exciting to us that more event registrations are happening overall, but it's clear that we've got to work on getting ALL our members engaged.

So we're going to strive to reach some deeper goals in 2009, instead of just tracking the easy stuff.  Then next year at this time, I can ask myself: Did focusing on these metrics mean that we helped more people change the world?  (Does anyone know the calculation I would use for that? :)

Here are a few more links on the topic:

• A McKinsey article that got me thinking about this a couple of years ago.
• The Tyranny of Metrics from Tactical Philanthropy
• The Quest for Metrics from Michael Gilbert

P.S. I have to admit to feeling a wee bit queasy about just coming out and sharing some of our numbers, even though I'm totally pro-sharing.  But I hope it will inspire you to do the same.  We should share more numbers!