The Importance of Contextual Design in Reducing Project Costs and Increasing Customer Satisfaction

The Importance of Contextual Design in Reducing Project Costs and Increasing Customer Satisfaction

By Angella Derington, OCI Software Engineer 

March 2008

What is Contextual Design?

Contextual design (C.D.) is a scientific method for determining how software should be developed based on the users' actions and not their words. C.D. utilizes three distinct steps to help determine and prioritize software requirements.

  1. Contextual Inquiry. Gather data from observing multiple different users doing their work
  2. Work Modeling. Use standard modeling methods to model the individual user's work
  3. Consolidation. Consolidate all of the individual work models to create a complete view of users' work practices

After consolidation, the C.D. team can make recommendations on software requirements and prioritizations based purely on how users will actually use the software.

Unlike traditional methods of requirements gathering, C.D. has the ability to objectively determine and prioritize requirements, even with the influence of those "token" requirements being pushed by influential customer and/or project team members. And I would bet that most of us have been on projects where the sum of the most resource-consuming parts of the project were these "token" requirements.

Are you convinced yet?

Maybe you're thinking:

Let's do a little exercise to illustrate why customers might not be able to accurately describe their work, and therefore might have holes in the requirements or inefficient prioritization.

Contextual Design Exercise: Car Driving Software

Let's say that we are going to write software that does the work of driving a car, something along the lines of a precursor to the cars in "Demolition Man."

Your exercise is to think up the requirements for that software and its priorities. Do you have them ready?

So your top requirements might go something like this:

You might even have thought about requirements like:

Now, just to make it realistic, let's throw in some "token" requirements from those influential team members.

Now we're ready to develop great software that every driver in the world will want in his or her next new car, right?

Our software developers have what they need to design software that provides the necessary functionality in a way that users will find easy to use and understand, right?

Developers should be able to design this software easily; after all, we all drive cars and know how they should work.

But ... our project manager has been under the gun because our team keeps having problems with scope creep, release slipping, and increased costs. And to top it off, when our software finally gets into the hands of our end-users, they are not happy with the software, and our customer service expenses keep going up.

Someone on our team has heard of this new design methodology, Contextual Design, that is supposed to help with this process. It is supposed to help our team figure out what features are really important to our users and how our users will actually use those features, which should hopefully reduce project costs and customer service calls. Our project manager decides to take a risk on this new methodology to see if it will help. After all, her understanding is that C.D. should take only 4-5 weeks, and our project is being estimated at $5 million dollars and a 2-year timeline.

Step O: Get prepared for Contextual Design

The first step in C.D. is Contextual Inquiry (C.I.), a fancy term for observing and interviewing people while they do the work with which the software is going to help.

Before we can start our C.I. step, we need to line up our potential interviewees. So, we need to answer a few key questions:

  1. Who are our target users?
  2. How many people do we need to interview to get a good cross-section of our target users?
  3. How many team members are good candidates for C.I. work?
  4. How many of those team members can we devote to the C.D. process?
  5. What type of work do we want to observe?

Then we'll set up the interviews and get started!

1. Who are our target users?

In order for C.D. to be effective, our interviewees should be good representatives of our target users; otherwise we will not get valuable data from our analysis.

For this study, let's assume that our marketing department thinks that our driving software will be most appealing to drivers on road trips and long commutes, both with and without children in the vehicle. So, we obviously need some parents who travel with their children and some commuters.

Is that all?

It might be good to also ask marketing who else they think might be interested in this software in the future, which turns out to be technophiles and the elderly.

So, now we know the sub-groups of people that we want to interview.

2. How many people do we need to interview?

We need to figure out how many interviewees we should find for each subgroup in order to achieve a good cross-section of our target users.

In practice, interviewing at least four different people from each of our primary groups (in this case, families and commuters) will generally provide adequate data coverage. Interviewing one to two additional interviewees from each of our "future user" groups (technophiles and the elderly) will provide adequate outlier coverage.

However, as the analysis progresses, it may become necessary to add more interviewees to answer any significant questions that may arise.

3. How many team members are good candidates for C.I. work?

The next step can often be the most daunting for a development team, especially one that does not have prior experience with C.D. analysis. The team leaders need to ask, "who on our team would even be good at doing this, let alone want to give up "real" development for 4-5 weeks?"

For C.I., it is preferable to have one facilitator and one observer for every interview.

C.I. takes patience and strong observation skills, and not all developers are a good fit for C.I. work, especially those that like to work in a silo and those that strongly prefer coding over designing. Most developers, however, become strong believers in C.D. after participating in the process.

The most important trait in selecting team members for C.I. work is their attitude toward designing software. A C.I. team member needs to believe that software should be designed to best fit the users' needs, over what is "cool" technologically, what has been done before, or what is easiest to develop.

C.I. is all about users and user needs, so team members need to be able to see the world from that perspective.

4. How many of those team members can we devote to the C.D. phase?

Many development teams may have only a few people who are good candidates for C.I. work, and some of those people may be your most important developers. This is when the project or development manager needs to decide how important the C.D. analysis is to the project in relation to the tasks that these developers are working on now.

Contextual Design Timeline

Each phase of C.D. takes a certain amount of time and manpower, so in order to figure out how many people to devote, we need to figure out how much time we need to complete our analysis.

  1. Contextual Inquiry. Approximately 2 hours and 2 people per interview
  2. Work Modeling. Approximately 2 hours per interview; should be conducted in the same day as the C.I., if possible
  3. Consolidation. 1-2 weeks, depending on variations in data collected during the C.I. and Modeling stages
    • It is beneficial to have as many team members as possible participate in this step. This step can help developers better understand how users will use the software, which generally leads to better design decisions during the design and implementation stages of the project.

The C.D. has decided that the following interviews are necessary:

If the team performs only one interview per day, we could complete the C.I. and modeling phases in about three weeks (given days off, travel time, etc.). That seems doable for our team: three weeks and only two team members off active development. The main concern, however, is that we now have only two people doing the modeling. Is that going to give us good results?

It is preferred to conduct C.I. with two teams working in tandem, with the teams conducting separate interviews in the morning and then performing modeling work together in the afternoon. This allows for deeper discussions around what the interviewee really meant and potential holes in understanding the users' actions.

For this exercise, however, we will assume our manager is only approving one two-person team, since we are new at C.D., and she is still not convinced that this is the best use of the team's resources.

5. What type of work do we want to observe?

Before the team goes into an interview, we need to figure out what kind of work we want to observe.

In our case, we want to observe our users driving a car. It would also be helpful to see (or at least hear about) the actions leading up to and after driving a car. These pre and post actions can give us important tidbits of information on how we can add value to our application.

Time to Set up interviews and get started!

Now that we have a team, a timeline, and the work we want to observe, we need to set up interviews with users in our desired groups. Let's assume that we have people in our organization who can get us names and numbers of interested users that fit into our subgroups.

When setting up interview times, one of the most important pieces of information we need to transfer to our users is that we want to watch them actually performing the work we want to observe. This can be a difficult concept for users to comprehend, since they probably have never had an experience where an interview is based on actions and not on words.

For our example, we will want to encourage our users to have a natural reason to drive their cars during our interview with them. So, we want to observe a parent driving a car with children and a commuter going to work in the morning.

Now we're ready to get started!!

Step 1: Contextual Inquiry - Gather the data

There are three primary parts of Contextual Inquiry:

  1. Warm up. Reiterate with the user what to expect during the interview
  2. Observe the work. Observe and clarify the user's actions
  3. Wrap up. Clarify the high-level actions observed and ask follow-up questions

1. WARM UP. REITERATE WITH THE USER WHAT TO EXPECT DURING THE INTERVIEW

After our initial "meets-and-greets" with the user, we will want to reiterate why we are here and what work we would like to observe him or her performing.

The following points are the most important to present to the user:

After the facilitator has presented our key expectations to the user, the facilitator should make sure that the user understands these expectations, and then we are ready to observe some driving!

2. Observe the work. Observe and clarify the user's actions

Now we are in the meat of C.I., actually observing the user perform the work.

Facilitator Responsibilities

The flow of this part of C.I. is mostly determined by the user, but it is the facilitator's responsibility to keep the user on track.

Some things the facilitator needs to be cognizant of are:

The most important thing for facilitators to keep in mind is that it is better to feel like they are asking too many questions than to get to the modeling step and not understand why the user did something.

In C.D., the whys behind actions can be the most important information in determining requirements and priorities.

Observer's Responsibilities

It may seem that the observer does not have a very important role during C.I., but in practice, the observer's role can be the most important.

During the interview, it is the observer's responsibility to watch and listen closely to the user and the facilitator and record everything the observer sees and hears. The smallest of details can be critical to creating a complete picture of the user's actions and intentions during the modeling and consolidation steps.

Often, the facilitator will not pick up on every hole in his or her understanding of the user's actions, and the little details that the observer records can give clues to those answers.

For example, the user may flash his or her lights, but the facilitator may not know why. The observer could have noticed that a car on the other side of the road did not have its lights on, and our user was trying to inform the other driver of the problem.

3. Wrap up. Clarify the high-level actions observed and ask follow-up questions

Once the user feels that he or she has completed the tasks we are there to observe, we move into the wrap up.

During wrap up, the facilitator has a few key tasks:

After wrapping up what was observed during the interview, the facilitator should give users an opportunity to ask questions of the facilitator and observer, as well as to add any additional information that they think is important for the team to know.

Some useful information that can be gathered at this point is:

Before leaving users, thank them again for allowing you to disrupt their day and reiterate how important the information that you've gathered is.

Step 2: Work Modeling - Model the data gathered

Work modeling allows us to organize the raw information that we gathered during C.I. in a methodical, scientific way, and is crucial to the outcome of C.D.

The work modeling process is as follows:

  1. Both members of the interview team simultaneously read through their notes for the modeling team.
  2. After each individual observation, the modeling team decides in which model the observation belongs.
    • During this exercise, it is the other team members' responsibility to make sure they fully understand the sequence of events.
    • By creating the work models on the same day as the interview, the interview team may be able to remember more details about the interview because of the other team members' probing.

C.D. utilizes several model types:

  1. Flow model. Tracks the flow of information between different people
  2. Sequence model. Tracks the user's intentions and actions consecutively
  3. Physical model. Graphical model of the user's work environment
  4. Culture model. Records any important influences, rules, etc. placed on the user because of the culture of the work environment
  5. Observations. Any other important information presented by the user that does not fit into one of the other models

Contextual Design Recording Standards

C.D. uses three colors to highlight different types of data. The purpose of using these different colors is to help the C.D. team better analyze the models during consolidation.

  1. Red. User breakdowns
    • Events or actions that caused the user to be less than successful or caused inefficiencies in the user's work
  2. Green. Non-observed data
    • Events or actions only described by the user, but that were not actually observed during the interview
    • This data should not be given as much weight during consolidation, because users' memories of events are not as complete as observations.
  3. Black/Blue. All other data
    • All events or actions that were observed during the interview

1. Flow Model

The purpose of the flow model is to record the flow of information and artifacts (paperwork, containers, etc.) between people during the execution of the user's work.

C.I. data that should go into a flow model for our driving software could include:

Figure 1. Example flow model (reference: groups.sims.berkeley.edu/waypoint)

Figure 1: Example flow model

2. Sequence Model

The purpose of the sequence model is to record the user's intentions and the actions taken to address that intent in a consecutive manner.

In practice, this model is the most important; it clearly displays the user's high-level goals and the steps necessary in order to address those goals.

Sequence model is also the most complex of the models, so for this article, I will be giving only a high-level overview of how to create a sequence model. If you are interested in learning how to create a sequence model in practice, please check out the additional resources listed at the end of this article.

Creating a Sequence Model

  1. Write down sequence actions in the order they occurred
    • Sometimes actions will be recorded out of order during C.I. This can occur either because the user is describing events that took place in the past or because the note taker wrote down actions out of order for some reason.
    • The importance of the sequence model is to understand the order of the steps the user needs to take to address the task
  2. Determine what the user's intent was for performing the actions
    • This is the most important aspect of the sequence model. In order to accurately determine requirements and priorities, it is imperative that the C.D. team understand why the user did what what was done.
    • Every series of actions should have a corresponding intent by the completion of the sequence model.
      • Intentions should be nested as the user's intentions become more detailed.
      • The intentions will become the outline for the sequence model.

Example Sequence Model

  1. Intent: User needs to go to work
    1. Intent: User needs to get settled into the car
      1. User sets coffee cup in the cup holder
      2. User plugs his cell phone into the car adapter
      3. Intent: User needs to re-adjust car settings, because wife was last driver
        1. User adjusts seat
        2. User adjusts side and rear view mirrors
        3. User adjusts steering wheel
      4. User buckles seat belt
    2. Intent: User needs to back-out of the garage
      1. ...
      2. ...
      3. ...

3. Physical Model

The purpose of the physical model is to record any important information regarding the user's physical work environment.

This model is generally recorded as a rough drawing of the interesting aspects of the user's environment. In practice, a physical model is only done if interesting data was observed during the interview that is attributed to the user's physical environment. The physical model can identify inefficiencies or breakdowns in the user's work. The physical model can also identify potential limitations on the software.

C.I. data that could indicate a need for a physical model includes:

Figure 2. Example physical model (reference: supernovel.wordpress.com)

Figure 2. Example physical model

4. Culture Model

The purpose of the culture model is to record any influences on the user's actions that are attributed to politics, work culture, rules of conduct, etc. In practice, a culture model is rarely created, because there are rarely enough different pieces of information to make it useful; interesting culture observations are instead recorded in the observation.

An example of C.I. data that could go into a culture model might be that "the husband speeds, but only when his wife isn't looking."

5. Observations

Observations are all of the other data that was recorded during C.I. that could not be attributed to one of the above models.

Observations are generally recorded as a bulleted list. Observations can also include any design ideas that the team thought of and any questions that arose during the modeling session.

Step 3: Consolidation - Make a complete view of the data

Once all of the interviews have been conducted and modeled, it is time to create the consolidated models from the individual work models.

At this point, it is beneficial to the project to have other members of the development team participate. In practice, it has been found that project team members retain more of the knowledge gained from C.D. by being involved in the analysis phase, even if they do not spend 100% of their time with the C.D. team. This knowledge in turn benefits the project by having more people on the team understanding the use cases and customer needs.

We will be creating consolidated models of each of the model types we created for the individual interviews, plus two additional models:

  1. Affinity diagram. Hierarchical model of relationships between the data gathered in the observations
  2. Artifact model(s). Model(s) that demonstrate the important aspects of related artifacts

In practice, it is most useful to consolidate the models with the least data first. Often, those models include the physical, culture, artifact, and flow models. Sometimes individual models are not easily consolidated, such as the physical model, and in those cases, it is acceptable to keep the individual models separate for the analysis phase.

The most important thing to remember during consolidation is that the goal is to organize the data in such a way that it can assist the team in clarifying the requirements and priorities needed for our software project.

Consolidation: Sequence Model

Once again, I'll provide a brief overview of creating a consolidated sequence model; if you would like to learn how to create a sequence model in practice, please reference the resources at the end of this article.

To consolidate the sequence model, we will start by:

  1. Identifying similar intents in each of the individual sequence models
  2. Identifying the sequences of events that match up to similar intentions
  3. Add intents with decreasing similarity until all of the items in the individual sequence models are in the consolidated sequence model

The consolidated sequence model will start to look and feel a lot like pseudo-code. The consolidated model will most likely have if/else statements and variations to the intentions.

Consolidation: Affinity Diagram

The creation of the affinity diagram is an ideal activity to involve the larger project team. The purpose of the affinity diagram is to organize all of the observations by their relationships to each other.

Creating an affinity diagram is an organic process that involves immersing the team in the observations left out of the other models.

  1. Place each individual observation on its own small piece of paper
  2. Team members start organizing the observations by placing each one near other observations that seem related (no right or wrong answers)
  3. Team members can move observations around even after a relationship is first identified
  4. Relationship groups will start to form; each group should have six or fewer items in each group
  5. After all of the observations have been placed in relationship groups, each team member attempts to create a description of the groups
  6. The team decides on one description for each group; the team can decide to use one of the members' descriptions or create a new description
  7. The relationship groups are now placed near other groups that seem related
  8. Second-level relationship groups will start to form. Each second-level group should contain six or fewer first-level groups
  9. Descriptions are written for each second-level group, just like for the first-level groups
  10. Steps 7-9 are repeated until we have five or fewer high-level relationship groupings

At any time during this process, observations and relationship groups can be moved around to make better relationship groups.

Figure 3. Example affinity diagram (reference: www.sapdesignguild.org)

Figure 3: Example affinity diagram

The affinity diagram will be a strong indication of requirements and priorities needed by the application. It will also quickly show additional value-added features that can be added to application in the future.

Requirements and Priorities Analysis

At this point the C.D. team has all of the C.I. data modeled in an objective way and can begin identifying the requirements revealed by the models and their appropriate priorities.

For ease of understanding, let's prioritize our requirements into three buckets:

Now that we have our three buckets filled with the requirements ascertained from our C.D. analysis, our marketing department or customer team can decide what requirements should be included in the first release of our application. And the coding can begin!!

Other Benefits of Contextual Design

Beyond knowing what requirements are really important for users, there are many more benefits to conducting a C.D. analysis. These include:

References

Software Engineering Tech Trends (SETT) is a regular publication featuring emerging trends in software engineering.

More Popular Articles

Planning for the Unpredictable: Project Estimation Simplified with a Middle-Out Approach

Machine Learning in Practice: Using Artificial Intelligence to Read Analog Gauges

Improving Project Quality with PMD

From Test Driven Development to Behavioral Driven Design