Exploring the Qualitative Quagmire for Information Systems Research Clare Tagg Open University Business School, The Open University, Walton Hall, Milton Keynes, MK7 6AA Tel: 01462 896600 Fax: 01462 895777 Email: c.tagg@open.ac.uk Abstract The discipline of information systems straddles the scientific and engineering research traditions of computer science and the more pluralist research approaches of the social sciences. As a consequence while research methods have a higher profile than in computer science, they have not yet reached the established maturity found in the social science disciplines. So, although research method is discussed in the information systems literature, there is a paucity of texts on research method or of detailed explanations and justifications for research method in the literature. This is particularly true of research that uses qualitative approaches. For researchers in information systems, particularly those with a computer science background or based in computer science departments, this poses particular problems. These problems are particularly acute for research students because of their limited timescales and relative inexperience. This paper explores this issue by considering one studentÕs search for an appropriate research paradigm for a PhD on the human aspects of software development. Presented at first UKAIS conference, Cranfield UK, 10-12 April 1996 Commended as most innovative research paper Introduction The interdisciplinary nature of information systems means that the need for a plurality of methods is recognised (Fitzgerald et al 1985). Hirschheim (1985) identifies the need to draw on the social sciences for an epistemology while Antill (1985) sees information systems as a hybrid activity involving technical, personal, organisational, and philosophical aspects and thus drawing on research methods from different disciplines. However, as Avison and Myers (1995) point out, the choice of research method often depends on more practical factors such as the predilections of the funding body or academic department of the researchers. Despite this plurality, in most cases the underlying paradigm is positivist. Jenkins (1985) describes 13 different research methodologies (based on his experience as editor and running PhD seminars) but he exposes his scientific assumptions when he orders his methods on the strength of hypothesis testing and states Ôvariables are, of course, central to all researchÕ (p112). Iivari (1991) in analysing the epistemology of information systems development found it to be almost entirely positivist. In many cases this outlook is not even questioned (Orlikowski and Baroudi 1991). While Vitalari (1985) describes a post-positivist research philosophy the approach he advocates is largely quantitative and he has no suggestions for how qualitative data may be treated (although he does recommend including some open-ended questions in surveys). One problem highlighted by Klein and Welke (1982) is that information systems can be regarded as a scientific discipline only because of a community of shared beliefs. Farhoomand (1992) reports that the discipline has not made much scientific progress because of the lack of theory of its own. Nevertheless, most information systems research is either theoretically grounded (implicitly or explicitly) or descriptive (generally case-study based) (Orlikowski and Baroudi 1991). Jenkins (1985) describes a Ôfundamental research processÕ (p103) which is presented as a linear process with feedback loops beginning with library research. More recently, the research approaches featuring on the front cover of Galliers (1992), show via the use of case study or action research some use of data in theory building. The practical emphasis in the discipline (Avison and Myers 1995) and an understanding of the uniqueness of each system (Vitalari 1985) probably accounts for the extensive use of case studies and action research in information systems research. Multiview (Avison and Wood-Harper 1990) and Soft Systems Methodology (Checkland and Scholes 1990) are both well-known examples of the outcomes of action research. However, concerns with the usefulness of single case or action research identified by Cotterman and Senn (1992) may arise from a lack of detail (dare one say rigour) in the way in which the cases are used in theory-building. Despite the fact that Antill in 1985 identified the following key issues in information systems research: repeatability of results vs uniqueness of an information system, impartiality of the observer vs experience of the designer, analysis of data vs hypothesis testing; Orlikowski and Baroudi in 1991 only found 5 journal articles (out of 155 surveyed) to be interpretative in outlook. Nevertheless qualitative approaches are beginning to emerge as illustrated by the ethnographic work of Low (Low and Woolgar 1993), the critical perspectives used by Murray and Wilmott (1993) and the grounded theory approach used in the first phase of Land, Le Quesne and WijegunaratneÕs (1992) study. The problem for any information systems researcher is that while the need for qualitative approaches may be apparent, there is a dearth of published material on research method in information systems. For example, Davis (1992) reported from a review of journal articles that there is little academic research in systems analysis and design and Cotterman and Senn (1992) conclude of the conference ÔSystems Analysis and Design: A Research StrategyÕ that Ôthe question of appropriate research tools and strategies generates more heat than light.Õ (p459) The problem is particularly acute for those coming from a computer science background. In computer science, the scientific approach is endemic (Fitzgerald et al 1985) and research method rarely discussed. This is evident from doctoral students attending the annual PhD consortium of Information Systems (Galliers 1993) and is exemplified by comparing the research methods courses provided for computer science doctoral students (eg Ross and Staples 1994) with that provided for information systems students (eg Wood-Harper, Miles and Booth 1993). On the one hand the courses include such things as communication skills, writing, and library searches; whereas the doctoral program in information systems at the University of Salford includes seminar-based sessions on the foundations of inquiry. In order to explore the implications of these problems for PhD students in information systems, this paper traces how I arrived at a research method over the three years of my PhD research. The next section explains the background in terms of my outlook and experience, the nature of the research question and the academic institutions where I work and study. This is followed by an explanation of the difficulties I encountered in pursuing a research approach in which data is informed by theory. Subsequent sections describe how I arrived a data driven approach and what the consequences were both in terms of research paradigm and in delivering an acceptable thesis. The analysis of the story of my research reveals the many difficulties for a PhD student in undertaking qualitative research and suggests ways in which the academic information systems community could work towards lessening these. Background The overwhelming influence at the start of my research was positivist; my first degree is in Mathematics and second in Computer Science. The dissertation for my MSc project (Tagg 1979) is very much in the tradition of computer science research. I come to software development with both long-term practical experience and a theoretical background. I worked for ten years for the Bank of England based first in a user department and then in the professional systems development department developing systems to support the BankÕs internal financial functions. Since then I have been a consultant developing software and helping other people to develop software. I lecture in computer science at Hatfield Polytechnic (now the University of Hertfordshire) and have supervised very many computer science student projects at undergraduate and postgraduate level. My research topic grew out of this combination of practical and theoretical experience. There seemed to be a mismatch between the issues that seemed important in my consultancy work and the subjects we were teaching in computer science. While at the University I would hotly debate the virtues of structured methods over object-oriented or the benefits of C++ over Ada, the consultancy work seemed to centre around people. For example, in one case the actual approaches used in development depended upon the expertise and ability of the development staff and the success of the project depended upon convincing the manager of the users that the system was not ÔstupidÕ. This view that people are important in the development process is supported at least in part by BrookesÕ (1987) well-known article and in looking for his silver bullet I thought we might do well to approach the problem of software development tangentially and focus on human issues. The first title of my research was consequently ÔSoftware development is more than methodÕ and I sought to demonstrate that even if you get the method and tools right, there are other important considerations. For my PhD, I registered at the Open University Business School in order to give me a different perspective on the problems of software development. In keeping with the research traditions of the disciplines, research method was more of an issue at the Business School than in the Division of Computer Science where I was lecturing but in both departments, a positivist outlook prevails. Theory informed by data In the first year of research, my concerns were mainly with theory and the difficulty of handling diverse amounts of theory. I seemed to suffer from Gummesson's chinese box syndrome: every time I opened a box I found myself in a bigger box (1991 p18). I had decided that I would like to undertake a research programme and write a thesis which would be acceptable from both a management and a computing perspective despite the differences between PhDs in the disciplines (ARBC 1993). Initially I saw research method as unimportant, uninteresting or irrelevant and although I was reading many of the sources I subsequently cited in my thesis, I did not make sense of what I was reading. I have to confess that I found the arguments about research philosophy a distracting side issue and agreed with Hughes (1990) that there is Ôno necessary reason why the solution of philosophical problems should be a precondition for social scientific researchÕ (p160). However, I did accept that as I was undertaking a management PhD, I would need to demonstrate an understanding of research methods (Gill and Johnson 1991). I was unconvinced by the rather simplistic approaches which are depicted as a series of steps moving in an orderly progression through data collection and analysis (eg Gill and Johnson 1991, Evans 1984). The naivety of these approaches reminded me of the waterfall approach to software development. I was much more convinced by the erratic and iterative nature of research portrayed by Feyerbend (1975) and Frost and Stablein (1992) who describe research as an Ôimprecise, creative craftÕ that is Ôsubject to serendipity as well as to plansÕ (p290). Within the first week of the research, I recognised the importance of following particular cases through time. My own experience indicated that the impact of decisions and factors influencing systems development was often only apparent after the system had been implemented (maybe years later). Despite the fact that Ômany of the important issues in information systems analysis and design can only be studied by longitudinal research extending over a number of yearsÕ (Davis 1992 p 11) longitudinal studies are seldom used. A survey conducted by Orlikowski and Baroudi (1991) of 155 information systems research articles published from 1983 to 1988 showed that only 7 (4.5%) were longitudinal. I searched for a research method based on longitudinal studies but there is little literature about studies of information systems development and literature from the social sciences seemed irrelevant because of its emphasis on individuals rather than organisations (Young, Savola and Phelps 1991). From reading the literature on science and scientific method, I adopted a research strategy combining both theoretic approaches and hypothesis testing within a feedback loop (Sparkes 1981). In this approach data and theory are in one feedback loop so that Ôlooking back from anywhere in the loop reveals the precedence of the other component in it!Õ (p157) and scientific knowledge is refined in the light of experience. This was attractive because it parallels experiential learning theory (Kolb 1984) which I use in my teaching. This approach was also in line with the ideas expressed in the 1984 IFIP WG 8.2 colloquium on research methods in information systems where Vitalari (1985) talked of needing a research method which permits learning and feedback into the method and Wood-Harper (1985) stressed the feedback elements in action research. Using this approach, the research outcome was to be a framework of the influences on software development. The framework was to be derived from the literature and informed by a few rich longitudinal studies to provide examples and enable hypotheses to be tested within a feedback loop. Taking this feedback approach, I began to read widely from both the management, information systems and computer science literatures. It was soon apparent that I was going to be swamped both by the volume and diversity of the material. To assist with the process of building the framework, I turned to mind-mapping (Buzan 1989, Svantessan 1989) to facilitate the innovative structuring of information that is part of research. Drawing upon my computing background, I built a prototype Hypercard stack for classifying theory based on the mind map concept. The resulting tool for academic writing enables the researcher to build a knowledge base consisting of material from the literature search - a hypercard stack of references. These references can be cited in a stack representing a mind map of the theory. Notes and quotes can be included in this mind map stack and the entire stack can be output as a Word outline complete with cited references (Tagg 1993). During this period, I was beginning to think about the longitudinal cases that were to inform the theory in the feedback loop. The first case was C, the consultancy project which had provided the impetus for the research. My consultancy was mainly concerned with supporting the project manager who had no experience of developing software systems. This case provided examples to illustrate the theory I was developing for human influences during the development and implementation of a software system but did not cover the early part of development when a decision is made to build a particular piece of software. To address the issue of why a piece of software is built and to pilot the approach to be taken in the longitudinal cases, I became involved with T, a small manufacturing business which makes high quality bespoke furniture and specialist joinery. They did not use a computer but were considering buying a laptop and agreed to become a case study for my research provided I gave them advice about which computer system to buy. The action-oriented nature of the case study element of my research is still very evident in my handling of T. I was interested in proposing that ÔwhyÕ should be considered along with the ÔwhatÕ and the ÔhowÕ of traditional structured analysis (DeMarco 1979). I argued that while neither structured nor object oriented methods address the reason (the ÔwhyÕ) for building a software system, this issue is covered by approaches based on strategy, soft systems and prototyping. My original intention with T was to use a number of these approaches to help the partners assess the potential for a computer within T. In practice, I completed an analysis of T based on Process Quality Management (PQM) (Ward 1990) which provided useful input to TÕs plans for computerisation (Tagg 1994). In undertaking the T case study, I concentrated largely on doing the PQM analysis and wrote up the work in a traditional passive style. In discussing the case it was noticeable that my verbal descriptions were much richer than my written notes and that important contextual information had been lost in the writing. I also realised that I was losing a lot of the flavour of the case by using my own words rather than those of the partners. Discovering that context can be as important as content is definitely out of character for scientific research (Sparkes 1981). My third case, was completely different. G is a subsidiary of a large insurance company. They were in the process of having an internationally distributed database built for them by the IT function at head office. At the start of the study, the first phase had just gone live. I decided from the outset to tape the conversations I had at G. I found that I had to take a much more passive role than in the previous two cases because of the size and complexity of the system and the different form of access that had been agreed. Conducting open-ended conversations with the main players from both the development team, users and managers resulted in a mass of useful and interesting information about the system, the development process and the attitudes of the players. This pattern of conducting a series of open-ended conversations which were taped was repeated with the remaining two cases. Data informed by theory After six months of serious data collection with the three new cases fully underway, I began to become overwhelmed with the data. At this point, chance played a part. As part of my continuing development of a Hypercard writing tool, I had been recommended to read about Hypersoft. Although it was no longer pertinent when I finally acquired a copy of Dey (1993), I started to read it because of its easy style. What I found was a structured, organised approach to the analysis of qualitative data. This was extremely attractive because it appeared to retain the kind of rigour needed for ÔgoodÕ science (Strauss and Corbin 1990 p57) whilst coping with the quantity of data I had already amassed. This was the beginning of a year spent trying to make sense of the literature on qualitative research. It soon became clear that qualitative research means more than the analysis of non-numeric data although this was my starting point. Moreover, qualitative research means such different things to different people that it is hard to codify (Tesch 1990). At the simplest level there is an awareness of the characteristics of qualitative descriptions which includes context which is often non-verbal, intentions so that meaning is negotiable and process both in terms of action and consequence (Dey 1993). Considering the process of qualitative research rather more generally, the same kind of ideas emerge. Van Maanen in the introduction to Van Maanen, Dabbs and Faulkner (1982) whilst stressing the individuality of qualitative research, identifies five procedural principles: detailed observation as primary data source, observation of actual events, study of ordinary behaviour, consideration of human meanings and intentions, and, a descriptive focus. Miles and Huberman (1994) include these principles in their list of core recurring features (pp5-7) but have a greater focus on the data analysis. Thus they include the aim of gaining a holistic overview, the possibility of many interpretations of the data and the role of the researcher as research instrument. Moving away from these concerns of data collection and analysis to more general issues of ontology and epistemology, the literature becomes confused particularly for someone from a positivist tradition as mirrored in GardinerÕs (1993) presentation and in ongoing discussions on the QUALRS_L electronic list. One of the underlying causes is the influence of positivism and the desire of sociology to create a science of human behaviour (Hughes 1990). So, when the qualitative movement started to grow in the 1970s, the positivist stance was often rejected out of hand (Tesch 1990) leading to stereotyping and polarisation in the literature typified by: ÔIn the red corner is phenomenology; in the blue corner is positivism.Õ (Easterby-Smith, Thorpe and Low 1991 p22). Harre (1981) identifies the subsequent confusion, difficulty and profusion of the literature. A variety of terms such as naturalistic, interpretivist, phemenological are used to refer to non-positivist research but in some contexts each of these are taken to imply specific world-views or methods. This complexity of terminology with writers bent on describing what they are not (posivitist) rather than what they are, meant that the concept of discovery essential in qualitative research was not obvious to me. Another facet of this confusion arises from the limited boundaries of much literature, so, for example, a text coming from a background in educational research will place a different interpretation on terms than one coming from sociology (Avison and Myers 1995). Given the research traditions in the contributing disciplines of computer science, information systems and management and my background it is hardly surprising that my research was begun within a positivist tradition that was implicit rather than explicit. During the journey of discovery described above, three important interrelated strands emerged which have shaped the final research approach. These may be loosely characterised as the ontological, epistemological and methodological dimensions which define the inquiry paradigm (Guba and Lincoln 1994). Within the methodological strand, the concern for method or how the research is conducted is gradually overlaid by an awareness that this can not be separated from underlying ontological and epistemological beliefs. So, for example, when using case studies in research, the way in which the data is to be used to generate or support theory affects the choice of case study and the way in which it is conducted. This is also affected by the stance that is being taken on objectivity. The ontological strand is characterised by considerations of what is worth knowing. The initial hypothesis: Ôthe process of software development depends on more than the methods, tools and techniques which are the traditional concerns of computer scientistsÕ, was replaced by the much more open-ended research question: Ôif software development is viewed as a human activity, what emerges?Õ This change arose from concerns over the paucity and narrowness of hypothesis-based research when compared with more holistic-based approaches together with an increasing awareness of the complexity of reality. Intertwined with these concerns are epistemological issues of appropriate ways of addressing the research question. Initially, as reflected by the action-research approach, there is an emphasis on identifying causal relationships. This was replaced by an acceptance of the value of studying the world as it is which is reflected in the change in emphasis between theory and data. Developing a research paradigm The holistic world view indicated by the previous section in which theory is generated from data collection indicated that a qualitative approach should be adopted towards the research (Cassell and Symon 1994). Additional indicators were the difficulty of justifying the existence of an objective reality when palpably users and software people see things differently and the absence of a well-understood core theory on which the research could be based. Having decided that a qualitative approach is appropriate for this research, the obvious (scientific?) solution is to adopt a well-defined, existing qualitative approach that fits the nature of the research question. Unfortunately, the existence of separate well-defined research approaches is illusory. Although there appears to be some consensus over ontological and epistemological concerns, there is a separate non-overlapping consensus over methodological issues. Moreover, as Tesch (1990) illustrates, the same term can mean different things within different disciplines. Within the disciplines contributing to this research there is very little literature that establishes coherent, complete research paradigms in the qualitative tradition. It is much more common for people to choose a blend of research ideas according to a mixture of personal and political preferences together with the aims and context of the research (Easterby-Smith, Thorpe and Low 1991, Orlikowski and Baroudi 1991). In consequence, and justified by TeschÕs (1990) principle that each qualitative analyst has to find their own way, the research stance was described by giving my response to the three questions which define the Ôbasic belief system or world view that guides the investigatorÕ (Guba and Lincoln 1994 p105). The ontological question is covered by a discussion of views of reality. According to the original positivist definitions, reality is what is available to the senses (Hughes 1990). Post-positivists agree that people may have different, possibly conflicting beliefs about reality and that there may exist completing theories to explain reality, but there is nonetheless at most one view that is right (but which view may not be known) (Phillips 1990). In contrast, relativists believe Ôthat there is no single tangible, fragmentable realityÕ (Lincoln 1990 p77) because reality is a social construction. The development of information systems are embedded in social context (Orlikowski and Baroudi 1991, Vitalari 1985) so the relativist approach would seem to apply. Within software development, developers and users certainly seem to have different perceptions and accepting these two equally valid realities changes the nature of the research endeavour. Instead of trying to arrive at a definition of correctness that all parties can agree to and is well enough defined to be useful, we can separately define what is meant by correctness in the two realities and explore the connections (Orlikowski and Baroudi 1991) and contradictions within the definitions. In taking this approach, I departed from the idea that knowledge is individualistic (eg Schwandt 1994) and have assumed the existence of shared, publicly understood realities. This seems to me to be a perfectly plausible consequence of knowledge as a social construction. It fits the data in that people do seem to be able to discuss the world as if they shared a reality even when they may not share a common language. This was apparent during my conversations with programmers in one of the cases working on network drivers. This is outside my experience so that at times they could have been using a foreign language and yet, because of our shared experience as programmers, I could usefully discuss their work. This contrasts with LowÕs experience (Low and Woolgar 1993) where she had difficulty penetrating the technical discussions perhaps because of her background as a sociologist. In the research, I focused on two identifiable realities: the technical and non-technical as typified by computer science and management. I consciously decided to ignore other realities such as psychological and sociological. Having described my ontological position, there are implications for epistemology since there is no longer the necessity to maintain the objectivity of the researcher implied by a positivist view of a single ÔrealÕ world (Guba and Lincoln 1994). One of the problems with understanding software development is the difference between the public, rational view of development and what actually happens. For example, software developers know about the value of good design but often do not undertake design activities. To investigate this difference it is necessary to get close to the development which is not consistent with objectivity (although of course distance does not ensure objectivity (Dey 1993)). The third element required to define the inquiry paradigm is methodology. By this Guba and Lincoln (1994) do not just mean methods but how in general the researcher goes about finding out what can be known. This is again constrained by the ontological and epistemological stance adopted so that my view on reality and research roles implies that my methodology involves the construction of realities through discussion. As Morse (1994) so clearly illustrates, there is a close relationship between the overall strategy or strategies adopted within this methodological framework and such issues as research question and outcome. The methodology which developed over several months, drew on aspects of grounded theory and to a lesser extent ethnography. During this period, the research question was further refined to be: ÔWhat are the key themes of software development?Õ A detailed method of analysis and synthesis supported by a software package emerged from this period. A significant factor in the methodological considerations was the issue of quality. The criteria for judging quality research had to be established based upon arguments presented in the literature combined with considerations of the research audiences. One consequence of the methodological development was that only one of the cases, G, was analysed in detail. Delivering the results With the research paradigm determined and the research method established, I thought my problems were over but there were (and are still) a number of problems of delivery to be resolved. The main issues surround the production of the thesis: its structure, content, language and length. One of the problems is the absence of exemplars and although we are not constrained in Britain, as US PhD students often are, the thesis still has to Ôlook rightÕ to the audience. I have partially solved this by adopting a fairly traditional structure - thankfully because there are no core theories, I did not have the problem of what to do about a literature review. Moreover, I have carefully limited my use of the personal in my writing style to those aspects of the thesis where my role as researcher is key. Content is a more complex question: to what extent do I explain and justify my research paradigm and how do I present the data so as to achieve quality. I have devoted a large proportion of the thesis (about 25%) to issues of research which is justifiable given the contribution to knowledge in the method. However, in doing this I have not taken the normal approach within qualitative research, of explaining how I arrived at my paradigm. Nor, because of length restrictions, have I explained issues in qualitative research, assuming, perhaps unreasonably, that my readers would be familiar with this material. The presentation of the case study data is a harder problem to solve. My criteria for research quality (drawing upon Lincoln and Guba 1985) rely for validity on the reader believing the analysis and being able to generalise from their own experience. This is only possible if the reader can see more than snippets of conversations. I have addressed this by constructing a chronological story of the development in G by blending together the words of the participants. This enables the reader to get close to case but unfortunately it is still overlong for inclusion in the thesis. Analysis Having retold my story, I am no longer amazed that I have become daunted by the entire process, I only wonder that I have reached the point where I can see my way out of the quagmire. The main issues arising from this story are: ¥ Qualitative research is important in information systems research because of the rich insights it provides into problems which are not well understood. The key issues in software development emerging from my research are accepted as critically important by practitioners and yet are not addressed in the academic literature in any coherent manner. ¥ Doing qualitative research well is more than adopting a different research method; it involves a reappraisal of the paradigm of research which has significant impacts on all aspects of the PhD process. This includes the nature of the research question, the way in which data is collected and analysed and how the results are presented. ¥ Although a PhD is supposed to be a training in research, allowing students to wallow in a qualitative quagmire is not going to progress the subject. Qualitative approaches can be applied with rigour and can lead to the theory development that the subject needs but without some established frameworks this rigour would be beyond most PhD students. ¥ Relying on work in other disciplines is dangerous because of the different interpretations and because of the special concerns of the information systems discipline. However, at present this is necessary because of the shortage of detailed literature on research method in information systems. ¥ For qualitative PhDs to become commonplace in information systems, there is a need for a more in-depth understanding of the issues amongst supervisors and examiners. A number of recent events have served to raise consciousness of alternative paradigms but without detailed, practical experience is it possible to guide a PhD student through the quagmire? ¥ Even if qualitative research approaches were commonplace, it could be argued that it would still be more difficult for a PhD student to adopt a qualitative approach because the method can not be defined up front but must emerge as part of the research. This problem can be alleviated by good research methods courses so students have had an opportunity to try methods before embarking on their main research. This would provide sound foundations for emergent methods. Conclusions This paper has highlighted the dichotomy between the established recognition of the importance of qualitative research in information systems and the difficulties for a PhD student undertaking such research. The case described is not typical; I had far greater experience of my subject and research than many PhD students. By being an extreme it highlights rather more clearly the issues. The main issues centre around the shortage of literature and widespread experience of qualitative research in information systems. An awareness of the importance of qualitative research has been established; there is now a need to follow this by detailed treatments of doing qualitative research. This need can be realised through published papers making their qualitative approaches explicit, by new texts on the practical procedures of undertaking qualitative research, and through in-depth courses for faculty and students. In thus providing some solid ground in the quagmire, it is important not to lose sight of the issue of research paradigm. Establishing the credibility and practicality of qualitative research in information systems is critical both in terms of the usefulness of information systems research and in establishing information systems as a respectable academic discipline. Bibliography ABRC (1993) Nature of the PhD: Discussion Document, Advisory Board for the Research Councils Antill, L. (1985) 'Selection of a Research Method', in: Mumford, E., Hirschheim, R., Fitzgerald, G. and Wood-Harper, T. (eds), Research Methods in Information Systems, North-Holland Avison, D.E. and Myers, M.D. (1995) The Interdisciplinary Nature of Information Systems and the Case for Anthropology, University of Southampton Discussion Papers in Accounting and Management Science, Number 95-101 Avison, D.E. and Wood-Harper, A.T. (1990) Multiview: An exploration in Information Systems Development, Blackwell scientific Publications Brooks, F.P. (1987) 'No Silver Bullet', Computer, April pp10-19 Buzan, T. (1989) Use Your Head, BBC Books Cassell, C. and Symon, G. (1994) 'Qualitative Research in Work Contexts' in: Cassell, C. and Symon, G. (eds) Qualitative Methods in Organizational Research: A Practical Guide, London: Sage, pp1-13 Checkland, P. and Scholes, J. (1990) Soft Systems Methodology in Action, John Wiley & Sons Cotterman, W.W. and Senn, J.A. (ed) (1992) Challenges and Strategies for Research in Systems Development, Chichester: John Wiley & Sons Cresswell, J.W. (1994) Research Design: Qualitative and Quantitative Approaches, Thousand Oaks, California: Sage Publications Davis, G.B. (1992) 'Systems Analysis and Design', in: Cotterman, W.W. and Senn, J.A. (ed) (1992) Challenges and Strategies for Research in Systems Development, Chichester: John Wiley & Sons, pp9-21 DeMarco, T. (1978) Structured Analysis and System Specification, Yourdon inc Dey, I. (1993) Qualitative Data Analysis: A User-Friendly Guide for Social Scientists, Routledge Easterby-Smith, M., Thorpe, R. and Lowe, A. (1991) Management Research: An introduction, Sage Publications Evans, K.M. (1968) Planning Small Scale Research, Nfer-Nelson Farhoomand, A.F. (1992) 'Scientific Progress of Management Information Systems', Information Systems Research: Issues, methods and practical guidelines, Galliers, R. (ed), Blackwell Scientific Publications, Chap. 5 pp93-111 Feyerabend, P. (1975) Against Method, Verso Fitzgerald, G., Hirschheim, R., Mumford, E. and Wood-Harper, T. (1985) 'Information Systems Research Methodology: An Introduction to the Debate' in: Mumford, E., Hirschheim, R., Fitzgerald, G. and Wood-Harper, T. (eds) Research Methods in Information Systems, North Holland Frost, P. and Stablein, R. (eds) (1992) Doing Exemplary Research, Sage Galliers, R. (ed) (1992) Information Systems Research: Issues, methods and practical guidelines, Blackwell Scientific Publications Galliers, R.D. (1993) 'Doctoral information systems research in Britain: a report on the UK Information Systems Doctoral Consortiums, 1991-1993', Journal of Information Technology, 8 pp118-120 Gardiner, P.D. (1993) 'Researching Conflict in Project Management: A Methodological Case Study' presented at British Academy of Management Conference, Milton Keynes 20-22 September 1993 Gill, J. and Johnson, P. (1991) Research methods for Managers, Paul Chapman Publishing Guba, E.G. and Lincoln, Y.S. (1994) 'Competing Paradigms in Qualitative Research', in: Denzin, N.K. and Lincoln, Y.S. (eds) Handbook of Qualitative Research, Thousand Oaks, California: Sage publications, pp105-117 Gummesson, E. (1991) Qualitative Methods in Management Research, Newbury Park, California: Sage Publications Harre, R. (1981) 'The positivist-empiricist approach and its alternative', in: Reason, P. and Rowan, J. (eds) Human Inquiry: A sourcebook of new paradigm research, John Wiley & Sons, pp3-17 Hirschheim, R. (1985) 'Information Systems Epistemology: An Historical Perspective', in: Mumford, E., Hirschheim, R., Fitzgerald, G. and Wood-Harper, T. (eds) Research Methods in Information Systems, North Holland Hughes, J. (1990) The Philosophy of Social Research, Longman Iivari J. (1991) 'A paradigmatic analysis of contemporary schools of IS development', European Journal of Information Systems Jenkins, A.M. (1985) 'Research Methodologies and MIS Research', in: Mumford, E., Hirschheim, R., Fitzgerald, G. and Wood-Harper, T. (eds), Research Methods in Information Systems, North-Holland, pp103-117 Klein, H.K. and Welke, R.J. (1982) 'Information Systems as a Scientific Discipline', Proceedings, ASAC, University of Ottawa, May 1-3, pp106-116 Kolb, D.A. (1984) Experiential Learning: Experience as the Source of Learning and Developnment, Prentice-Hall Land, F.F., Le Quesne, P.N. and Wijegunaratne, I. (1992) 'Technology Transfer: Organizational Factors Affecting Implementation - Some Preliminary Findings', in: Cotterman, W.W. and Senn, J.A. (ed) Challenges and Strategies for Research in Systems Development, Chichester: John Wiley & Sons, pp65-81 Lincoln, Y.S. (1990) 'The Making of a Constructivist: A Rememberance of Transformations Past', in: Guba, E.G. (ed) The Paradigm Dialog, Newbury Park, California: Sage Publications, pp67-87 Lincoln, Y.S. and Guba, E.G. (1985) Naturalistic Inquiry, Beverly Hills: Sage Publications Low, J. and Woolgar, S. (1993) 'Managing the Social-Technical Divide: Some aspects of the discursive structure of information systems development' in: Quintas, P. (ed) Social Dimensions of Systems Engineering: people, processes, policies and software development, Ellis Horwood, pp34-59 Meloy, J.M. (1994) Writing the Qualitative Dissertation: Understanding by Doing, Lawrence Erlbaum Associates Miles, M.B. and Huberman, A.M. (1994) Qualitative Data Analysis, 2nd edn, Thousand Oaks, California: Sage Publications Morse, J.M. (1994) 'Designing Funded Qualitative Research', in: Denzin, N.K. and Lincoln, Y.S. (eds) Handbook of Qualitative Research, Thousand Oaks, California: Sage publications, pp220-235 Murray, F. and Willmott, H. (1993) 'The Communications Problem in Information Systems Development: Towards a relational approach' in: Quintas, P. (ed) Social Dimensions of Systems Engineering: people, processes, policies and software development, Ellis Horwood, pp165-178 Orlikowski, W.J. and Baroudi, J.J. (1991) 'Studying Information Technology in Organizations: Research Approaches and Assumptions', Information Systems Research, 2(1) pp1-28 Phillips, D.C. (1990) 'Postpositivistic Science: Myths and Realities', in: Guba, E.G. (ed) The Paradigm Dialog, Newbury Park, California: Sage Publications, pp31-45 Ross, M. and Staples, G. (1994) 'Computing Research', in: Information Systems Teaching Improving the Practice, Suningdale, England, March 1994, University of Hertfordshire, pp123-125 Schwandt, T.A. (1994) 'Constructivist, Interpretivist Approaches to Human Inquiry', in: Denzin, N.K. and Lincoln, Y.S. (eds) Handbook of Qualitative Research, Thousand Oaks, California: Sage publications, pp118-137 Sparkes, J. (1981) 'What is this thing called Science?', New Scientist, 15 January, pp156-158 Strauss, A. and Corbin, J. (1990) Basics of Qualitative Research: grounded theory procedures and techniques, California: Sage Svantessan, I. (1989) Mind Mapping and Memory, Kogan Page Tagg, C. (1979) LEGOL,Hatfield Polytechnic MSc Dissertation Tagg, C. (1993) 'A Hypercard Tool for Academic Writing', presented at British Academy of Management Conference, Milton Keynes, September Tagg, C. (1994) 'Process Quality Management: A case study of its use in determining the IT strategy for a small business' in: Ross, M., Brebbia, C.A., Staples, G. and Stapleton, J. (eds) Software Quality Management II Vol 1: Managing Quality Systems, Southampton: Computational Mechanics Publications Tesch, R. (1990) Qualitative Research : Analysis Types and Software Tools, Falmer Press Van Maanen, J., Dabbs, J.M. and Faulkner, R.R. (1982) Varieties of Qualitative Research, Beverly Hills, California: Sage Publications Vitalari, N.P. (1985) 'The Need for Longitudinal Designs in the Study of Computing Environments' in: Mumford, E., Hirschheim, R., Fitzgerald, G. and Wood-Harper, T. (eds), Research Methods in Information Systems, North-Holland, pp243-263 Ward, B.K. (1990) 'Planning for Profit' in Lincoln, T (ed) Managing Information Systems for Profit, John Wiley & Sons pp 103-146 Wood-Harper, T. (1985) 'Research Methods in Information Systems: Using Action Research', in: Mumford, E., Hirschheim, R., Fitzgerald, G. and Wood-Harper, T. (eds), Research Methods in Information Systems, North-Holland, pp169-191 Wood-Harper, A.T., Miles, R.K. and Booth, P.A. (1993) 'Designing Research Education in Information Systems: Towards a Global View', in: Khosrowpour, M. and Loch, K.D. (eds) Global Information Technology Education - Issues and Trends, Harrisburg: Idea Group Publishing, pp453-487 Young, C.H., Savola, K.L. and Phelps, E. (1991) Inventory of Longitudinal Studies in the Social Sciences, Sage