POST MORTEMS: THE ANATOMY OF MARKET RESEARCH PROCESS IMPROVEMENT

Did you learn everything you possibly could from your last project—not just from the final results, but by examining the process itself? Conducting a detailed review, a post mortem so to speak, can help pinpoint exactly what worked—and what needs work.

For example, let’s say a recent project used in-person focus groups. Did you and your team discuss how the process went?  What worked well about recruiting and what didn’t? Did the facilities and the moderator meet your expectations? Did the discussion guide support the intended goals?

All great questions to reflect upon.

So, how do you conduct a project post mortem? By objectively evaluating the process, step-by-step. But remember, the goal is not to place blame—it’s to find opportunities for improvement. So hone your scalpels and focus your microscopes; come time for your next project, you’ll be glad you did.

Let’s use the case of a quantitative project and break it down, stage by stage. For each stage, I suggest a few key questions to get you started.

Stage 1: PROJECT DESIGN

Did the specification of project scope and objectives go smoothly? Was it challenging to get all team members to agree on a reasonable, finite set of project goals? If an outside agency was engaged, how did the process of writing an RFP and selecting and hiring the agency go?

Stage 2: QUESTIONNAIRE DESIGN

How many iterations were anticipated and how many were made?  How did the programming and pretest go?  Were things discovered in the pretest results that needed significant reworking?  If there was a lot of reworking late in the process, could it have been avoided by preliminary qualitative research or by clearer thinking about project hypotheses?

Stage 3: SAMPLE SELECTION AND SOURCING

Did the sample sources you planned on using work out?  Were you satisfied with response and incidence rates?  What percent of records from a sample source did you have to toss because they were suspicious?

Stage 4: DATA ANALYSIS

Was a clear data analysis plan in place so that everybody knew exactly what was going to be done and how?  Were there data analysis errors?  Was there a lot of back and forth asking for more cuts of data, which could have been avoided with better planning?

Stage 5: DELIVERABLES

Were deliverables created on time and error free? Were reports professional looking and well received?  Were there other types of deliverables that internal clients or colleagues would have preferred?

And perhaps the most important measure of success: did people embrace the results, leading to important conversations, decisions or strategies?

WRAP IT UP: THE REPORT CARD

Boil down takeaways into a single page summary, and keep it factual. Avoid sneak attacks. Nobody should dread report card time.

You’re simply identifying improvements for next time around. This is your post mortem deliverable, an at-a-glance reference for next time.

Here are some options:

Self-assessment Report Card. In this case, the project team grades the project by phase. A weighted scorecard approach can be used by considering grades for items such as budget adherence, timeline adherence, usefulness of final results, and so on.

Client feedback. Collect client feedback (from internal or external clients). Ask for feedback on overall satisfaction and on specific projects’ aspects such as communications, schedule, and deliverables. How well did the project results and deliverables meet expectations?

  Did the results sufficiently support intended decisions and strategies? If not, why not? Giving clients a chance to grade the project can uncover some surprising areas for improvement; some clients won’t complain along the way, but sure do have feedback to give when asked.  Get it promptly, and address it, so that they won’t turn into market research skeptics.

Whatever approach you take, formal or informal, self-assessed or client feedback-based, you will certainly find opportunities for improvement. The key? Capture post mortem information before rigor mortis sets in, within a week of the project’s conclusion or sooner.  Projects over three months long may call for a midway mini-review.

[If you haven’t done so already, click HERE to sign up for Research Rockstar’s Market Research Newsletter. ]