Coherence and Transparency: Some Advice For Qualitative Researchers
Coherence and Transparency: Some Advice For Qualitative Researchers
Coherence and Transparency: Some Advice For Qualitative Researchers
qualitative researchers
Crispin Coombsa*
Loughborough University, Loughborough, Leicestershire, United Kingdom of Great Britain and Northern Ireland
a
*c.r.coombs@lboro.ac.uk
Abstract
There is relatively little advice in the Engineering domain for undertaking qualitative studies. Researchers have to rely on
generic guidance that may result in imprecise application of qualitative methods. A related discipline to Engineering is
Information Systems (IS) and the experiences of the IS domain may provide some useful insights for undertaking qualitative
studies. This paper synthesizes the guidance from the IS community for crafting high quality qualitative studies and
manuscripts. It reports on five themes: i) Establishing philosophical underpinnings; ii) Clarifying theoretical aims; iii) Selecting
qualitative methods; iv) Demonstrating rigour in qualitative data analysis; and v) Grappling with generalisation. The review
stresses the importance of coherence and transparency for crafting qualitative research manuscripts and provides a list of
reflective questions for qualitative research design.
Keywords
Qualitative studies. Guidance. Data analysis. Case study. Generalisibility. Information systems. Engineering.
How to cite this article: Coombs, C. (2017). Coherence and transparency: some advice for qualitative researchers.
Production, 27, e20170068. http://dx.doi.org/10.1590/0103-6513.006817
1. Introduction
Engineering research has traditionally adopted positivist research approaches using quantitative methods
(Borrego et al., 2009; Gawlik, 2016; Koro-Ljungberg & Douglas, 2008; Zhou, 2012). However, the dominance
of this research paradigm and method is starting to change in some Engineering disciplines. For example,
(Daly et al., 2013) observe that numerous studies have used qualitative methods to study Engineering Design.
However, because of the dominance of positivist traditions much work is still judged by positivist criteria and
expectations, which may be limiting the publication opportunities for qualitative studies. One way to address
this challenge is for researchers to ensure that they are adopting appropriate research paradigms, suitable
methodological techniques and crafting convincing and persuasive arguments in their manuscripts. Unfortunately,
there is relatively little advice in the Engineering domain for undertaking qualitative studies. Consequently,
researchers have to rely on generic guidance that may result in imprecise application of qualitative methods.
A related discipline to engineering that has experienced similar positivist dominance is Information Systems
(IS) (Davison & Martinsons, 2011). Like the Engineering disciplines, IS is distinct from the approaches of the
natural sciences that are concerned with describing or explaining what exists or what has existed. By contrast,
the Engineering and IS disciplines aim to describe and explain how to create what does now not exist or has not
yet existed (Lee, 2014). IS has seen steady growth over the past 20 years in the number of qualitative research
studies published in high quality international IS journals and conferences. Therefore, the experiences of the IS
domain may provide some useful insights for Engineering researchers wishing to undertake qualitative studies.
There has been considerable advice provided by the IS research community regarding the design and
application of qualitative research. However, much of this advice is scattered across journals either as editorial
The paper then presents a discussion of two overarching themes, transparency and coherence, that permeate
throughout the review and concludes with a series of reflective questions and associated resources to help
researchers focus on key considerations for crafting qualitative studies.
3. Method
In order to collate the relevant literature to inform this study the Association of Information Systems (AIS)
basket of eight IS journals were searched between 1991 and 2017. These journals are considered the top journals
in the IS field and include European Journal of Information Systems; Information Systems Journal; Information
Systems Research; Journal of AIS; Journal of Information Technology; Journal of MIS; Journal of Strategic
Information Systems; and MIS Quarterly. Each journal was searched using the term ‘qualitative’. The search
results were then manually reviewed by scanning the title and abstracts of the identified articles and included
if they provided guidance on qualitative study design. Articles on action research and design science were not
included as they include aspects that are not typically core to qualitative studies (Sarker et al., 2013). This filtering
process provided a sample of research essays, editorials, viewpoint articles, commentaries and literature reviews.
Reviews of IS case study research (Keutel et al., 2013) and qualitative research (Sarker et al., 2013) reveals
a number of common characteristics of qualitative studies. The theoretical aim of the majority of interpretive
studies was to develop theory to explain whereas positivist studies tended to test constructs and hypotheses
i.e. theory to explain and predict. For positivist case study research there was a slight preference for multiple
cases, but a significant number of positivist case studies adopted a single case approach. By contrast, for
interpretivist case study research the majority of studies adopted a single case approach (Keutel et al., 2013).
Sarker et al. (2013) found a similar pattern with over half of their sample of qualitative studies adopting a single
case study design. Sarker et al. add that that the majority of papers in their sample provided a rationale for
their sampling logic and explained why certain cases were chosen. Both Keutel et al. (2013) and Sarker et al.
(2013) found interviews to be the dominant mode of data collection in positivist and interpretivist studies,
often supported by document and observation data. In Sarker et al.’s sample the number of interviews ranged
from 175 to 6 with an overall average of 40 interviews with the majority (64%) stating that the interviews had
been recorded and transcribed. Many of the studies in Sarker et al.’s sample used a combination of document,
observation or field notes to support interview data.
However, the reviews also revealed several weaknesses in reporting qualitative case study research. Keutel et al.
(2013) observed the majority of researchers did not state their philosophical paradigm that underpinned the
study. A significant number of studies (44%) did not include rationale for their sampling logic or explained
why certain cases were chosen (Sarker et al., 2013). The majority of studies did not state the unit of analysis
adopted for the study (Keutel et al., 2013), provide details of the interview schedule or topic list used to guide
interviews, or explain and illustrate the role of supporting materials (e.g. document, observation or field notes)
in their analysis.
It is hoped that this review provides a resource for doctoral researchers, early career researchers, established
researchers and evaluators to guide the development and evaluation of qualitative research studies without
being prescriptive. There are many detailed text books (e.g. Corbin & Strauss, 2014; Miles et al., 2013) and
established web based resources (e.g. the AIS living scholarship site to support qualitative methods in IS research
(Myers, 1997)) that already exist. This review is not intended to replace these and researchers are encouraged
to refer to these sources for precise details regarding design and method.
Qualitative studies provide a valuable contribution to research knowledge complementing and enhancing
findings from more commonly applied research methods. They provide an opportunity to explain and understand
phenomena in a deeper way than quantitative studies. However, in order for the contribution of qualitative
studies to be heard, they have to be able to demonstrate the same levels of coherence, transparency and rigour
as we have grown to expect from positivist quantitative studies. Hopefully, this paper will be a useful resource
to help researchers achieve this goal.
References
Abubakre, M. A., Ravishankar, M. N., & Coombs, C. R. (2015). The role of formal controls in facilitating information system diffusion.
Information & Management, 52(5), 599-609. http://dx.doi.org/10.1016/j.im.2015.04.005.
Avison, D., & Malaurent, J. (2014). Is theory king?: questioning the theory fetish in information systems. Journal of Information
Technology, 29(4), 1-10. http://dx.doi.org/10.1057/jit.2014.8.
Benbasat, I., Goldstein, D. K., & Mead, M. (1987). The case research strategy in studies of information systems. Management Information
Systems Quarterly, 11(3), 369-386. http://dx.doi.org/10.2307/248684.
Borrego, M., Douglas, E. P., & Amelink, C. T. (2009). Quantitative, qualitative, and mixed research methods in engineering education.
Journal of Engineering Education, 98(1), 53-66. http://dx.doi.org/10.1002/j.2168-9830.2009.tb01005.x.
Cheng, Z., Dimoka, A., & Pavlou, P. A. (2016). Context may be King, but generalizability is the Emperor! Journal of Information
Technology, 31(3), 257-264. http://dx.doi.org/10.1057/s41265-016-0005-7.
Conboy, K., Fitzgerald, G., & Mathiassen, L. (2012). Qualitative methods research in information systems: motivations, themes, and
contributions. European Journal of Information Systems, 21(2), 113-118. http://dx.doi.org/10.1057/ejis.2011.57.
Corbin, J. M., & Strauss, A. L. (2014). Basics of qualitative research : techniques and procedures for developing grounded theory.
London: Sage. Retrieved in 2017, August 17, from https://books.google.co.uk/books?hl=en&lr=&id=Dc45DQAAQBAJ&oi=fnd&pg
=PP1&dq=strauss+and+corbin&ots=M2EI3QjYur&sig=EW4ndA40fHGiFZxlUWrfcgQcLBo#v=onepage&q=strauss and corbin&f=false
Daly, S., McGowan, A., & Papalambros, P. (August, 2013). Using qualitative research methods in engineering design research. In
Proceedings of the 19th International Conference on Engineering Design (pp. 203-212). Seoul: ICED. Retrieved in 2017, August 17,
from https://www.engineeringvillage.com/share/document.url?mid=cpx_M34eddd49145902ed14eM711f10178163125&database=cpx
Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: a comparison of two theoretical models.
Management Science, 35(8), 982-1003. http://dx.doi.org/10.1287/mnsc.35.8.982.