Into the Mind of a Subject Matter Expert: Uncovering Key Information
Katy Montgomery
I generally have a strong rapport with subject matter experts or SMEs. Learning from them is often a highlight of the instructional design projects I've worked on. In my experience, an ability to listen as well as a naturally inquisitive mind go a long way. When an instructional designer is comfortable asking about what they don't know, is dedicated to fully understanding something, and interacts with the SME with curiosity and interest, it fosters open channels of communication.
Beyond these practices, it's also helpful to understand the mind of a subject matter expert more deeply.
Knowing without knowing why
Please indulge me in a short exercise to illustrate a point about knowledge. Which verb choice would be correct in each of these examples and why?
1. She sometimes drinks / is drinking coffee in the morning.
2. She makes / is making coffee now.
3. She wants / is wanting more coffee now.
If English is your first language, you will likely make the correct choices of “drinks,” “is making,” and “wants”. Unless you have explicitly studied this, the trickier part of the question is explaining why. It might puzzle you. Why is it that both the second and third sentences are in the time frame of "now" but the second uses the present continuous form of the verb (the form with -ing) and the third does not?!
As this illustrates, you very expertly and instinctively know what is correct in your first language without necessarily knowing what is behind those choices. (On the other hand, if you studied English as an additional language and learned the rules explicitly, you would have an easier time explaining that “want” is a non-action verb—just like “love,” “need,” and “see” to name a few others—and, generally, non-action verbs do not take the present continuous tense regardless of the time frame that they reference.)
The mind of a subject matter expert
While working with a subject matter expert is recommended, it seems that sometimes experts do what they do so skillfully because they have internalized so much of the requisite knowledge—similar to how we follow the rules for our first language instinctively.
Therefore, because it is so automatic and internalized in their minds, simply asking subject matter experts to explain how they accomplish a task may not elicit all the information that a novice would need to carry it out. As Ruth Clark explains, “Your expert is not trying to be obtuse. It’s just that he or she has a lot of tacit knowledge—knowledge that cannot be readily articulated. Experts literally can’t tell you what they know” (Clark, 2013, p. 157).
Fortunately, there are some proven strategies that can help unearth important information from subject matter experts. These strategies can help reveal information about actions, decisions, cues, rationale, and monitoring, which can all be important to include in a learning product (Clark, 2013).
Concurrent verbalization
With this technique, experts verbalize what is going on in their minds while solving a real problem or going through a realistic simulation (Clark, 2013). The instructional designer records this for later analysis; it is also important for the instructional designer to remind the expert to speak whenever they are silent for more than five seconds.
The critical decision method
Another technique, called the critical decision method, involves an expert recounting a previous experience (Crandall et al., 2006, as cited in Clark, 2013). The instructional designer should approach this interview in five phases. First, the expert identifies a previous experience that is relevant to the learning target. The next focus involves the expert giving an overview of how the issue and how they handled it. After that, the expert creates a timeline that outlines the sequence for solving the issue. The instructional designer next reviews what has been explained and investigates further by asking specific questions to get additional information. The final stage focuses on “what ifs” regarding the situation.
Expert panels
Using a panel of three to six experts is another technique that involves recounts of previous experiences (Van Gog et al., 2010, as cited in Clark, 2013). First, the experts individually record three experiences related to the learning target; one should be easy to resolve, the second should be of moderate difficulty, and the third example should relate to a challenging issue. When the panel convenes, the experts should first make sure they agree about the classification of each example in terms of difficulty. Then, as a group, they discuss the key components involved in solving each issue.
Benefits of this approach include creating a collection of examples classified by difficulty, which can inform or be directly used in the learning product, as well as ensuring that the information gathered reflects a collective mindset rather than a particular approach of just one individual (Clark, 2013).
Reflecting on recorded behaviors
Because information discussed after the fact may be distorted by memory, recording an experience and asking an expert to discuss their thoughts while viewing it may lead to more accurate information (Clark, 2013). The recorded information can be in the form of video, audio, or even eye-tracking data as appropriate for the situation. While reviewing the recording, the instructional designer should ask the expert investigative questions as to why they took particular actions or made certain decisions.
Final thoughts
The technique that is best depends on the needs of the instructional project as well as the available resources. When it is possible, using one of these techniques or a combination of them can be very helpful in getting beyond the natural ease in which experts go about their work and discovering information that novices need to learn.
References
Clark, R. C. (2013). Scenario-based e-learning: Evidence-based guidelines for online workforce learning. San Francisco, CA: John Wiley & Sons.