Home > Web Content Managers Forum > Conference Call Archive > Meeting Minutes

Thursday, October 18, 2007, Meeting Minutes

More than 70 Forum members in attendance

Topic: New Tools to Identify and Measure Your Customers' Top Tasks

1) News

• Several members of the Forum recently attended the Usability Professionals Association (UPA) 1-day conference in Washington, DC. Key takeaways (from Nicole Burton, GSA):
  
  
  • Keynote speaker Jared Spool said big website redesigns are dead. This is based on substantial research he's done with customers like NetFlix, Amazon, and Southwest Airlines. Instead of large-scale redesigns, these major companies are using A/B Testing to make incremental improvements. With A/B Testing, you put up slightly different versions of a webpage, where some customers see version A, some see version B. You look at the metrics, see which page produces the most success for customers accomplishing their task, and serve up that page.
  • Next semester, Web Manager University will address A/B Testing as part of its Advanced Usability course.
  • Jared Spool also talked about the importance of every web manager doing some usability—it's everyone's job, especially plain language. Everybody can do some testing, and everyone should be practicing plain language.
    
• Several Forum members also attended the annual e-Metrics Summit in Washington, DC, with about 500-600 people in attendance. Conference highlights (from Joe Pagano, Library of Congress and Tim Evans, SSA):
• Government agencies need to develop more meaningful metrics—beyond just collecting data. We need more emphasis on INTERPRETING the data so we know what our customers are trying to do and how successful they are in accomplishing their tasks.
• Lots of discussion on how to measure the impact of Web 2.0, such as blogs. Everyone is struggling with this and there aren't any specific answers this point. Presenters also talked about the value of using A/B testing to test small areas of your site before investing in larger-scale improvements.
• Agencies should use metrics to focus on the future goals of their website—instead of past performance. Use metrics to get management's attention on the future—what you intend to do to improve your website.
• Keynote speaker Jim Sterne talked about the urgency of improving our websites—they're “burning our money.” The most important task is to get executives to focus on the Web. Look for short-term victories, which may give you the support you need to implement a metrics program across the board.
• Anyone can join the Metrics Task Group. Their next meeting is in early November. For more info, go to Webcontent.gov.

2) New Tool: Quick and Easy Customer Profile

As we've laid out in our Strategic Plan, October is "Metrics Month." In this spirit, we shared two exciting tools we developed to help our community use metrics to identify and measure your customers' top tasks:

1. Quick and Easy Customer Profile
2. Web Metrics Task worksheet

About the Quick and Easy Customer Profile

Presenter: Nicole Burton, usability Specialist, USA.gov, and co-chair, Usability Task Group

This tool helps you identify your top customers, top tasks, and top metrics. We've piloted the tool with several offices within GSA and with various agencies, including the FEMA example that we sent to the Forum listserv. It's meant to serve as a guide—please feel free to improve it for your use.

How to Use the Tool:

• Conduct the Q&E Profile exercise before the metrics inventory (which is a more extensive exercise).
• Look at the FEMA example. They identified very specific audiences (not just “the general public.” The more specific you are, the better you can identify top tasks.
• Find a partner to complete the tool: someone who works on the site with you, a usability specialist if you have one. Look at your metrics ahead of time and fill in the profile as you go. In about 2 hours, you'll have an executive summary of your site.
• After you finish, review, edit, and list your audiences in order of importance. Continue to brainstorm about where you come in contact with top customers.
• Finally, validate your findings. Share it with your web team and call center staff. Make concrete plans based on what you uncovered. Repeat this process. It's a way to get started.

Q&A about the Quick and Easy Customer Profile:

• A few callers mentioned that they plan to post their customer profiles on their intranet. This is very valuable. Your employees see who they're serving. The tool is not proprietary, so agencies should feel free to post it and share openly.
• We should have a central repository for agencies to post their customer profiles, so we can see what similar agencies have discovered. This is a great idea, and we can do that on Webcontent.gov. If you've completed your profile, please email it to Nicole Burton so we can compile an inventory.
• Other agencies have created “State of the Web” reports, such as HUD and CDC. This is also a great best practice.
• We should share top tasks across agencies. USA.gov should be based on top tasks that come directly from agencies. USA.gov does have a page, “What's on Americans' Minds” that lists the most popular pages on USA.gov and top search terms. But we may want to supplement this with data that comes directly from agencies.
• We'll post the Quick and Easy Customer Profile to Webcontent.gov shortly. If you have any questions or have feedback, email Nicole Burton.

3) Metrics Worksheet for Identifying and Prioritizing Your Customers' Top Tasks

Presenter: Joe Pagano, Library of Congress, and co-chair, Metrics Task Group

About the Metrics Worksheet:

• This tool complements the Q&E Customer Profile. The difference is that the Metrics Worksheet requires a little more detail and provides a more extensive analysis.
• It's very much in draft form, so please give us your feedback. We encourage you to take it and change it in ways useful to your agency. The worksheet provides a way to take information from a variety of sources and make assessments.

How to Use the Metrics Worksheet:

• Metrics are categorized into five major areas:

• Agency
• Direct Customer Feedback (email, call centers, conversations, etc.)
• Website data
• Social media
• Market research

Q&A about the Metrics Worksheet:

• In addition to using the worksheet to come up with an inventory of top tasks, you can use it to decide on performance measures for each task. This is going to become very important in government, just asi it's important in the private sector.
• After identifying performance goals, how will you measure this goal? You need to measure, test, measure, and then test again.
• Qualitative data is also important. Some agencies are using the ACSI online survey to get specific data about customers' top tasks. At SSA for example, they have a question on their survey, “Why did you come to the site today? Did you succeed in your task?” We also ask that question on Webcontent.gov, and have significantly reduced the number of overall questions.
• The current restriction on using permanent cookies hinders our ability to collect more meaningful metrics. This is an issue that the Web Managers Advisory Council may take up when it reconvenes beginning in November.
• It's hard to measure “number of people affected.” Yes, this can be difficult to track. Segment your audience as much as possible. In your agency, talk to program officers who are most familiar with that task and ask them how many people are trying to get information. It's okay to provide estimates. The goal is to come up with an inventory of the most important tasks, so having exact figures isn't critical. Knowing relative importance of one task over another is what's important.
• Many callers said the tools are easy to understand and will be very useful.
• For those working on state websites, it's very helpful to see what the other 49 are doing. Benchmarking with other agencies can be helpful—it doesn't always have to be competition. For example, we should identify best practices for applying for a driver's license and apply that to all states.
• This tool will help us move from being reactive to proactive. It will help us work with our managers to put web activities into better alignment with the agency's mission.
• The key will be to continue to use these tools to evaluate our websites, year after year. We should step back and look at the data regularly.
• Email any feedback about the Metrics Worksheet to Joe Pagano.

4) Wrap Up

• We shared great ideas on the call. Please use these tools, and when you've completed them, send them to us so we can share with the larger community. Doing so will better help us collaborate, see the content that other agencies are delivering, and hopefully reduce duplication across agencies.

• Rachel Flagg and Sheila Campbell, co-chairs of the Web Managers Forum, reminded everyone to take a look at the “Roadmap” on Webcontent.gov. The Roadmap offers different activities to help you transform your website around your customers' top tasks. You can work on any piece at any time. You don't have to do the steps in order.

• Candi Harrison's webinar on Leading Change is coming up on October 30.

• Interested in joining our task groups? Contact the task group leaders.

• Next Forum call will November 15, 2007 from 11AM Eastern time.