How Do We Know Who to Include in Collaborative Research? Toward a Method for the Identification of Experts

Abstract

Collaborative research, defined as research involving actors participating in the problem situation under study, has an important role in operational research, strategic management and systems thinking. In a recent study, we found that a strong organizational focus incorporated into many soft operational research (OR) approaches is inadequate for studying societal problem situations, which are fragmented and have no clear boundary. Specifically, we failed to find a process of identifying individuals that is capable of representing the perspectives of actors and sufficient for research into societal problem situations. We found no clear terminology accounting for ontological differences between actors, individuals representing them and conceptual representations of acting entities. In response to this gap in the literature, we propose terminology that differentiates among actors (individuals or collective entities in the real world), experts (individuals capable of representing the perspective of an actor) and agents (ideal-typical representations of actors). Based on this terminology, we propose an iterative method to guide the assembly of an expert group to undertake collaborative research into societal problem situations. To demonstrate the application of our method, we present selected insights from our study in an electronic supplement.

Reprinted from Publication European Journal of Operational Research, 216/2, Matthias Otto Müller, Stefan N. Groesser, Silvia Ulli-Beer, How do we know who to include in collaborative research? Toward a method for the identification of experts / pages 495–502. Copyright (2013), with permission from Elsevier.

Appendix

1.1 An Illustrative Case Study: Identifying the Actors Driving the Diffusion of Energy-Efficient Buildings

1.1.1 Application

To illustrate our method, we present its application in a study that we conducted on the diffusion dynamics of energy-efficient buildings in Switzerland. The study was carried out between 2005 and 2009 by an interdisciplinary team of researchers collaborating with an expert group representing the important actors. Methodologically, the study combined soft OR methods, such as causal mapping (Bryson et al. 2004), with System Dynamics (Forrester 1961; Sterman 2000) and survey methods. The collaborative research approach was chosen to overcome the difficulties associated with fragmentation and because we wanted to synthesize objective and subjective elements. The study implemented a case study design in a medium sized city in Switzerland.

At the beginning of the study, our understanding of the housing and construction sector was not very elaborate. However, before empirical investigations could begin, a theoretical framework needed to be defined. After some deliberation, we chose to combine Porter’s (1998) value chain approach with the agent-in-environment framework (Kaufmann et al. 2001; Kaufmann-Hayoz 2006; Ulli-Beer 2006). The agent-in-environment framework conceptualizes action as a co-evolution of acting entities and their environment by means of perception-action cycles. We relied on this, as it was compatible with the systemic perspective that the study wanted to achieve and because it focuses attention on the way actors shape societal structures and how societal structures feed back and influence action. The value chain approach seemed appropriate, as the construction of buildings can be represented as a sequence of different steps. Figure 4.2 shows the theoretical framework, including the actors we positioned within it after several iterations. Guided by this theoretical framework, we began to look at websites, searched and read scientific and non-scientific literature, and undertook discussions with other researchers. In addition, we conducted face-to-face interviews with persons involved in the administration of the city where we carried out our case study. As a result, our understanding of the societal problem situation began to evolve from a layperson’s piecemeal understanding of the construction sector toward a theoretically saturated and empirically grounded perspective (phase 1). We expect that any collaborative research project profits from consciously undergoing an exploratory stage because exploratory research allows an understanding of the issue under study to develop quickly. As our study progressed and we deemed our understanding of the societal problem situation to be sufficient, we moved on to phase 2. We began to evaluate potential actors by asking the following questions adapted from stakeholder theory: Who holds the power to significantly influence the societal problem situation? Who can claim to have a legitimate interest in the state or the evolution of the situation, and what reasons or justifications are used? For whom does the present or any potential state of the situation suggest urgent consequences? (adapted from Mitchell et al. 1997) Inspired by the power-interest grid approach (Eden and Ackermann 1998: 122), we first developed hypotheses about the power actors have to accelerate or block the diffusion of energy-efficient buildings and what interest such actors might have in favor of or against energy efficiency in buildings. We think that asking these kinds of questions would be helpful in any research project to develop the understanding of important actors. The next task we needed to tackle was the identification of individuals capable of and willing to represent an actor’s perspective by participating in our expert group (phase 3). We approached this task by relying on reference buildings and investigating individuals involved with the construction of these buildings. By researching recent building permits, we learned about recent construction projects. This information and the data obtained from phone books and the Yellow Pages allowed us to contact the persons applying for the building permits, who were mostly architects. In addition to inviting the architects, we asked them to provide contact details of other important actors involved in the project, whom we subsequently contacted. When contacting the persons involved with the reference buildings, we asked them to evaluate who has an interest in energy-efficient buildings or the power to affect the diffusion process. To achieve a broad sample of reference buildings, we asked architects to contribute recent projects that met specific criteria. On several occasions, we went to look at construction sites and obtained listings of the construction companies involved with that particular construction site from signs. To identify actors outside the value chain, we contacted government agencies dealing with energy in buildings on the federal, cantonal and communal levels. We asked them to participate and provide us with the names of private or semi-private organizations that they thought should be included. These were subsequently contacted and asked to help us identify further actors and individuals capable of representing them. The procedure of selecting‚“a reference cases” as a starting point to identify experts capable of representing important actors is expected to be of value for other researchers investigating societal problem situations.

Fig. 4.2

A representation of main actors in the societal problem situation

When no new actor emerged as important and phase 3 was completed for the time being, we conducted the first workshop with our experts to discuss our typology of important system agents (phase 4). During the first part of the workshop, we evaluated whether the important actors of the societal problem situation were represented. As it turned out, we had missed producers and importers of advanced technology. Consequently, after the workshop, we briefly moved back to phase 2 to adapt our understanding of important actors. Then, we moved to phase 3 again and identified and contacted individuals who would be willing to serve as experts representing advanced technology producers and importers. After this iteration, we were fairly certain that in the future workshops, we would be collaborating with individuals representing all the important perspectives. We found that although we had asked the experts to tell us which actors had an interest in or power to affect the diffusion process of energy-efficient buildings, the workshop setting triggered further actors. Hence, we think that fellow researchers performing collaborative research would profit from holding a workshop in addition to interviews. The second part of the first workshop was reserved for investigating behavioral aspects of actors. To reduce the complexity associated with actors’ behavior in the real world, we approached this task by representing actors as agents in a modified version of the power-interest grid. Researchers and the members of the expert group jointly debated and developed hypotheses concerning the power that actors have to accelerate or block the diffusion of energy-efficient buildings and what interest such actors might have in favor of or against energy efficiency in buildings. Figure 4.3 depicts a refined version of the power-interest diagram developed during the first workshop. Developing this power-interest diagram with the expert group allowed us to gain deeper insights into the behavior and rationales of actors. This illustrates how the use of problem structuring methods with an expert group representing the important actors can be applied to model actors of the societal problem situation as agents (phase 4), and this work increased our understanding of the societal problem situation (phase 1).

Fig. 4.3

Agents of the societal problem situation

In the weeks following the first workshop, we conducted individual interviews with the experts using cognitive mapping techniques described by Bryson et al. (2004) to research the experts’ cognitive map of the feedback structure of the societal problem situation. A second workshop was used to discuss causal-loop diagrams representing the merged cognitive maps. Subsequent workshops discussed the System Dynamics simulation model representing the experts’ and researchers’ joint perspectives. This work can be interpreted as a series of iterations between phase 4 and phase 1. These iterations contribute to testing because members of the expert group would sometimes reject the researchers’ perspectives, leading to further refinement of the System Dynamics model.

1.1.2 Evaluation

In retrospect, we found that applying our method in the context of our research project had several benefits. First, by applying the method, we were quickly immersed in the problem situation and were able to make the transition from a layperson’s perspective to a deeper understanding of the problem situation rather quickly. Second, our method led to the formation of a dedicated group of experts. Consequently, we gained a solid epistemic source from within the problem situation. In particular, we learned a great deal about the specific situation of each actor or category of actors by collaborating with the experts. This, in turn, provided important guidance for the subsequent development of a formal System Dynamics model. A possible limitation to our approach is the fact that it is comparatively resource intensive and may take several weeks to implement. Depending on the specifics of the situation, a determined effort may be necessary to find experts willing to participate. In our research project, it took several weeks to complete the process and prepare the first workshop. Therefore, we think our method will primarily be useful for researchers conducting larger investigations into societal problem situations characterized by high degrees of fragmentation and that lack a gatekeeper who can guarantee a balanced selection of experts. Secondarily, however, we think that our approach also provides terminological precision and theoretical grounding for practitioners who work under more constrained time frames.