Discipline, Community, and Standards for Educational Researach:

The Importance of New Technologies

in Promoting Collaborative Educational Research[1]

Nicholas C. Burbules

Bert Lambeir

Appearing in Beyond Empiricism: On Criteria for Educational Research," Paul Smeyers and Marc Depaepe, eds. (Leuven, Belgium: University Press of Leuven, forthcoming).

Educational research, like research generally, is becoming more collaborative. Individual researchers still operate quite happily on their own, of course; but there is an increased awareness that a broad-scale understanding of educational issues requires the perspectives and skills of different researchers approaching a problem from multiple sides, especially if the research is expected to have significant policy implications. Research of all varieties is tackling problems of increasing scope, complexity, and - therefore - interdisciplinarity. In a globalized world, more interdependencies are coming to the fore, and more people have a stake in the costs and consequences of large-scale research.

Educational research, in particular, must confront the skepticism that it has often been of relatively poor quality and has had very little effect on changing or improving educational practices. One reason frequently cited for this failure is the tendency of many educational researchers to treat educational problems as site-specific phenomena, abstracted from a larger set of social processes: focusing on a specific classroom or a particular teacher-student relation. Alternatively, focusing on wider contextual influences inevitably raises multidimensional, extremely complex dynamics not amenable to site-specific research categories or methods. On a policy level, this means that interventions to be implemented across a range of social institutions and interactions require research that does not fall within simple disciplinary categories, and that does not frame hypotheses around simple, linear, cause-effect dynamics. Rather, such research would need to be sensitive to the highly complex and interactive ways in which significant social phenomena are overdetermined. Thus, the decision about which categories, methods, and frameworks will guide a research project needs to be made in an interdisciplinary context. It is not simply a matter of saying "the methods must fit the problem," because even defining what the problem is already assumes certain constraints on how it will be studied. Deciding what factors are relevant to consider affects the range of possible explanations that can be derived.

In scientific contexts, there is a growing interest in issues of a fundamental, even profound, nature. Scientists across a host of domains have the sense now that they are working with very basic questions about the deep structural workings of phenomena: astronomers hypothesizing about the formation of the universe in its first moments following the "Big Bang"; particle physicists theorizing about the component elements and forces that make up everything in the universe; or the Human Genome Project, which seeks to unravel the underlying code of all human life. Government and institutional funders for such research unquestioningly accept the assumption that research on this scale must be enormously expensive; must be focused and continuous over a long term of effort; and must be collaborative. "Collaboration" in recent years has taken on the force of a mantra in education, government, and business; and in part this is because of the scope, complexity, and interdisciplinarity of the problems being pursued.

Our purpose here is not to argue that social research, and educational research specifically, ought to strive for "scientific" status, or that it can adopt the specific patterns of collaboration typical of scientific research like the Human Genome Project (though it would certainly appreciate such a research budget!). But educational research does share with these other fields of inquiry an intrinsic interest in interactions across many categories of phenomena; and, we are suggesting, especially where educational policy matters are concerned, research that does not operate across a complex range of factors will be inadequate.

To study education is to study a complex reality: every educational situation varies from every other since it includes teachers, students, their relations to each other and to other teachers and students, interaction patterns, different class cultures, shared and divergent experiences, and so on. Hence any specific issue of educational policy or practice almost certainly will entail questions of learning or cognition, of pedagogy, of social institutions, of communication, of cultural dynamics. (Imagine a large-scale study on how to reduce teen pregnancies in school, for example, or reducing dropouts.) It is quite odd then, when educational research wants to address educational reality by trying to draw simple causal inferences about instruction. The requirement to generalize and the overall search for causal explanations lose their meaning when the dynamic nature of educational processes is taken into account. In this respect it becomes doubtful whether educational research can directly cause the improvement of education in a means/ends sense. In ignoring the versatility of education, researchers tend to lose their grip on the reality of the school ground. Furthermore, in trying to objectify educational reality, this approach to educational research (and policy) precludes the opportunity of doing something really new.

Collaboration in the educational domain is important, then, not only because of the scope and interdisciplinarity of the problems under consideration, but because collaborative approaches better reflect the complexity of the situation; they encourage more subtle problem definitions and recognize simple causal assertions as too reductionistic. Moreover, because the people and educational contexts being studied are very diverse, the representativeness of the body of researchers involved with studying a problem (their race, gender, and so forth) is one of the few safeguards that various kinds of blind spots will not fundamentally damage the credibility of the research.

A key factor shaping attitudes toward collaboration in research is the increased incorporation of new information and communication technologies into the processes by which research data are gathered, represented, analyzed, written up, reviewed, shared, criticized, and published. This is our central concern in this essay. In important and sometimes unprecedented ways, these new technologies are not just facilitating collaboration among people, along lines of well-established practice; they are changing people's understandings of what collaboration is. Discussions of these new technologies invoke the idea of collaborative spaces and collaborative tools as sites of inquiry and creation, and not simply as facilitators or media of collaboration. These ways of rethinking collaboration, we will argue, raise new issues for disciplinary methods and standards of research; for the contexts in which research gets evaluated, and against what aims; for the populations of researchers and the boundaries of who is central, and who is peripheral, to the research community; and for expectations about knowledge and the uses and effects of knowledge upon policies and practices.

II.

The first area we will discuss concerns the processes of writing, representing, and publishing research online, and the new ways in which these support new understandings and practices of collaboration.

(1) Normally, researchers think of collaboration in writing as the process of drafting text, sharing these drafts, reviewing and revising one another's work, and so on, until a consensual version is completed that includes the ideas of many contributors and speaks with the voice of everyone and of no one in particular. Collaboration is viewed as the process that yields a product that belongs to them all; the text displays the fruits of their collaboration. New technologies for generating written text, however, invert this relation: the writing space becomes the site of collaboration, and the text that is being produced facilitates and instantiates the collaboration.

One example of this new kind of writing space is an "interactive paper" technology developed by Jim Levin and Jim Buell at the University of Illinois (http://cternt1.ed.uiuc.edu/ipp/about.cfm). In this relatively simple form, each author has access to the same textual space online, and each written statement can be the subject of comment, criticism, or elaboration by other writers. The text may begin with something written by one individual; but the additional commentary and amplifications of others may dwarf the original or leave the rudimentary text far behind as the revisions, comments, responses to comments, and responses to responses all grow out of the original text, each in turn potentially generating its own branches and lines of further discussion (of course many online newsgroups or discussion forums have a similar threaded or branching structure).

There are many questions that arise from this basic idea: What kind of "text" is this, and to whom does it belong? Is the result "one" text or many? Is a text generated in this way online significantly different from a conventional text with copious annotations and marginalia? How is it different from a sequential set of texts, each written in response to the one before, but constituting some kind of serial conversation over time? Can a text like this ever be "finished," and would that necessarily be a good thing?

Some online journals have experimented with incorporating something like an interactive paper format, blurring the lines between circulating a draft for reviewer comments and publishing a final version for readerly response. In this context, a journal is not only a delivery medium for disseminating final works, but a forum that uses the works as an occasion for discussion and reformulation of the ideas - here again the textual space is not simply the product of collaboration, but the facilitator of it.

(2) A second area of new collaborations in writing concerns the fundamental hypertextuality of online text. Early on, this capability was described as "electronic footnotes." But footnotes are still clearly subordinate to a "main" text, and typically operate as optional content included only to elucidate or reference textual content that is regarded as primary. Hypertext goes beyond citational practices to relate multiple texts to each other in many ways; one can link not only to a reference, but to partial or complete texts that become in a sense part of the original text; and this linking relation can be reciprocal and co-equal. Hence, each link, and hence each text, is embedded in the others, in ways that do not necessarily direct a linear sequence of order or priority in which these embedded text segments must be read.

Here, again, the written artifact instantiates a process of collaboration; in this case among authors who may be quite unaware of one another, joined now by a text that, through hyperlinks, creates a new set of relations among ideas by juxtaposing and reassembling text segments written at other times and even for other purposes. Indeed, the new text might contain nothing but such assembled segments! This writing form, which one of our colleagues calls "patchwording" (like a quilt), creates a collaboration, either through the efforts of a single compiler/editor (is this activity properly termed a kind of authorship?); or through the efforts of the authors of these text segments, each building links to the others, perhaps each for no purpose but his or her own, but in the process creating a complex interwoven set of interlinked texts because each is linking to the others, possibly without knowing that those others are also linking to each other, and back to that author as well (see for example "web rings" as one form this can take). The resulting questions echo those of the previous example, in terms of challenges to the idea of "authorship," of what it means to "finish" a text, or in terms of primacy of ownership over this new collaborative text - or if in fact "ownership" (in the sense of copyright, etc.) should even remain an important determination in such cases.

(3) A third aspect of online writing as a collaborative medium concerns the new possibilities of multimedia composition; in one instance simply as a compilation of text, music, images, and so on, that brings together authors/creators in different media to make something together that none could have made alone. But this would simply leave the question at the level of collaboratively producing a multimedia text. Beyond this level, as we have said, one should look at the possibilities of multimedia as creating a collaborative environment or occasion - where a creative format that allows such interrelating of sound, image, and text already defines a collaborative possibility, one that might change views about how these different media sources relate to each other: for example, overturning the traditional academic privileging of text as the medium through which proper research needs to be published by, say, collaboratively producing a documentary video as a form of serious scholarly research.

A step even further than this considers the ways in which multimedia and hyperlinked forms of representation create a context in which different voices, cultural value systems, or researcher perspectives can find simultaneous and co-equal modes of expression. One of the points being stressed in social research, including educational research, today is a suspicion about unquestioned authorial neutrality: Whose voice, whose perspective, is driving the study? Multimedia research can more easily include, for instance, primary qualitative data (say, unedited video or audio tapes from interviews), relatively unfiltered by analysis, along with the author's edited selections and interpretations. This would allow those researched a chance to "speak for themselves." This can also be seen as a kind of "test" of authorial reliability, by allowing readers to check the author(s)' interpretations against their own readings of the primary data (in this sense, they become collaborators too!).

As we have been emphasizing, these new forms of writing, representing, and publishing research are not just new tools in the researcher's toolkit; they change the relations among multiple researchers, those researched, and the audience. They decenter a single authorial voice (even a single "collaborative" one), in favor of multiplicity, dynamism, and interpretive openness. Together, they constitute a productive space in which collaboration can happen, but a collaboration shaped by the forms in which it expresses itself - not simply a collaboration directed toward a specific product. Indeed, as we have suggested, certain kinds of collaboration can only happen under these conditions, and certain potential collaborators can only be involved in these ways. Because new forms of writing, representing, and publishing research break open the boundaries of who qualifies to participate in scholarly production, the decision of who will be involved is not even entirely in the hands of the original authors - scholars sometimes have collaborative partners whether they want them or not.

III.

A second dimension of this process concerns new tools of research and data-gathering and the ways in which they also support forms of collaboration that decenter notions of where, how, and among whom research happens.

One example of this change includes new forms of teleconferencing or videoconferencing, and the ways in which they provide opportunities for synchronous, or simultaneous, interaction at a distance. In part, of course, these technologies are simply recreating the conditions in which this same collaboration might have occurred around a single table, face-to-face. But now such interactions can involve potential collaborators who would never be part of immediate, face-to-face collaboration around a single table - people from outlying corners of the globe, disabled people, people without academic credentials, people who cannot afford travel, and so on. This point is not trivial when you consider that one way of changing our views about research collaboration is not only in the how of doing it, but in who should be included in it. Nor, as we move toward virtual interactive environments that are multisensory and increasingly naturalistic, where participants interact as active personae with each other and with a data environment, it becomes even more clear that the virtual space is the site of collaboration, and that it shapes the intentions and actions of the participants.

But this example only scratches the surface of the changes we are discussing here; new information and communication technologies are not just media for talking back and forth. They also make possible gathering data (through remote instruments, like cameras on another planet, or sensors buried deep in a volcano, or images from fiber optic cables inside a medical patient's body) that could never be observed directly. In educational research, we talk about "observation," of course, but normally this is in unavoidably intrusive ways, in particular sites to which researchers can gain personal access; distant video technologies can support other kinds of observation and data-gathering (some of which, however, may raise problems of "surveillance" in the Foucauldian panoptic sense).

Another area of research concerns very complex simulations running on supercomputers that model large-scale processes so that they can be parsed and analyzed in a way that actual phenomena never could; this makes the simulation or model not only an aid to research, but a research environment itself, one that can generate new data which, in turn, can be studied in productive ways. The Human Genome Project, for example, would have been simply unthinkable without enormously fast and powerful computers, a speedy connection among them, and complex modeling technologies. In these sorts of examples the technology establishes an occasion for collaboration around data that are created with the technologies, not just collected by them. Because these models involve massive amounts of information there is no way they could be studied by single individuals, or even by a simple collaborative group - they require many disciplinary tools. Building and maintaining such data environments can itself be a complex collaborative research task.

For example, to take a case closer to our own field of study, witness the huge Wittgenstein archive at the University of Bergen (http://www.hit.uib.no/wab/). Gathering all of his textual production in one site, and linking it together in a complex hypertext, is both a substantive scholarly endeavor itself, and a space for further scholarly collaboration. It is not just archival or documentary work in the simple sense. In the examples we are describing here, therefore, the very existence of data of a certain scale, of a certain difficulty, of a certain complexity, creates an informational space through which researchers can and must work together to create it and then to make sense of it. The data environment, if you will, compels collaboration; it would not exist without it.

The examples from the natural sciences may seem an inappropriate model for social science research, specifically educational research. We don't have large-scale "models" or "simulations" of learners or learning environments. But even here large data sets can be formed by melding multiple smaller data sets; as with quantitative meta-analyses, sometimes a clearer picture emerges from aggregated particulars. This technique is often termed "data mining," and can be developed in such a way that visual representations of the data literally reveal emerging patterns. Data mining can work with information spaces that dwarf ordinary conceptions of statistical analysis or qualitative interpretation. For example, the Shoah project of the Virtual History Foundation involves recording and coding video interviews with every living Holocaust survivor. When those tens of thousands of hours of interviews have been recorded and digitized, how will researchers use them? Are there ways of bridging conventional dichotomies of quantitative and qualitative methods to create tools - by necessity involving some statistical or numerical analyses - that also respect the qualitative nature of the material being studied? Here again, such very large data sets are open-ended exploratory environments that compel collaborative research approaches; and this is an underused approach in educational studies, partly because of the tendencies toward particularism noted at the beginning of this essay.

We have tried to highlight how these kinds of changes in research environments pose challenges to some traditional categories and distinctions of research. We have just shown how the quantitative/qualitative distinction might be questioned. Another is the distinction between gathering data, in a purportedly open-minded empirical fashion, and then only later analyzing and interpreting it. In the kinds of research domains being described here, the design and development of information environments is already a research endeavor, structured by multifaceted efforts not only in developing technologies as tools, but in building into them assumptions about knowledge and the phenomena to be studied that are themselves complex theoretical endeavors. In this sense even the data themselves are, to use a familiar phrase, "theory-laden," but here because the very design of an information space requires collaboration, planning, and original insight; there is no way to collect and archive useable data on this scale without building into the design a set of assumptions about how it will be used, for what purposes, and by whom.

IV.

A third dimension of these information and communication technologies that affects collaborative research is the way that they exist within, and reinforce, an increasingly global context of interaction.

It is clear enough that a global medium allows disparate and far-flung scholars to work together; but there are further effects of doing this. When disparate and far-flung scholars are brought together, other things change; disciplinary standards will need to be re-examined and negotiated; research assumptions and aims may work at cross-purposes; even the choice about the language in which the collaboration will take place has implications for what can be talked about, and by whom. In such instances, the technologies that facilitate collaboration, from one standpoint, also shape and determine the collaborations that can and cannot occur.

These technologies also promote a certain kind of publicity for research that has rarely existed in the past. Data and preliminary results on the Internet can be studied (and criticized) by anyone with access to them; in some cases the data might be used by other scholars for other research purposes; issues of ownership and confidentiality may be at stake. One can argue that it is in the researcher's self-interest to encourage access to, and commentary on, his or her research and results in order to help improve it. Yet other imperatives, obviously, press researchers to keep their data and results private until they are ready to "publish" the work officially (and even after the final publication of their work they may still feel proprietary toward "their" data). Either choice manifests assumptions about the scale of collaboration and who one's collaborators are (as we asked earlier, can strangers who reanalyze one's data be seen as collaborators?). Of course, such choices have to be made even in traditional research environments too; but new information and communication technologies heighten the issue.
But there is another side to this global context: When decisions about supporting collaborative research are made by policy makers on the national level, what is their underlying rationale? The tendency of global collaboration in research seems to be another attempt to satisfy the consumer (the policy community) with "useful" information that can support "cost-effective" reforms. Is this not a market-based discourse, characterized by a fundamentally economic rationale? Is the effect of globalizing research, ironically, another way to reduce the scope of what is counted as valuable research? How far will we have to go in "bridging the gap between theory and practice," and what is the price to pay? We have highlighted the benefits of sharing ideas, exchanging data, co-generating theories and explanations, and so on. Clearly, information and communication technologies can play a major role in these activities. But at the same time this might drive a process toward standardization, for example, the standardization of language: Which language will predominate on the network? Moreover, there is a benefit to much social, educational and historical research because of their particularity. More general (or "global") social or educational theories may not respect the heterogeneity of cases, especially across wide cultural differences. Hence, the global context of research can be productive and counterproductive simultaneously; technology enables some things and reduces other possibilities. When the latter is at stake, perhaps educational researchers should withdraw from the apparent "problem" at hand, from time to time, to think and eventually think again, about their methods and aims of research.

And here is the overarching import of these observations: these changes in the forms, purposes, and media of collaboration coincide with larger contemporary doubts about the nature of research and research knowledge. Standard assumptions about the objectivity and finality of "facts," about the neutrality and universality of methods, about generalizability, or about the unquestioned good of modernist, predominantly Western, approaches to intellectual production being spread to the rest of the world all come under challenge in this new context.

An exploration and reconsideration of the nature of collaboration brings questions about the methods and aims of research into contact with questions about the contexts of research, and how these contexts act to include and exclude prospective participants. When we view the problems and processes of research as themselves structuring the forms of collaboration that can arise in response to them, we avoid the simple instrumentalism of thinking that the form of collaboration can simply be shaped to the question at hand. Sometimes the very definition of the question (and who gets to define it) is the problem.

When we realize that new information and communication technologies are vastly opening up the possibilities of what collaboration can look like, and who can be part of it, certain value questions come to the fore. For fields like educational research, which by nature invite interdisciplinary and open-ended questions, the forms of collaboration also express, inevitably, conflicting assumptions about human nature, society, knowledge, and value. What is the relation between the development of interdisciplinary collaborative teams and the standards or criteria of educational research?

Interdisciplinary collaboration in the full sense involves scholars from different disciplinary, theoretical, and methodological perspectives who define common questions and investigate them using mixed methods, and who are continually in dialogue with one another about how their different perspectives relate to one another. But interdisciplinary research, if it is to be successful, must overcome certain kinds of barriers. One is an issue of vocabulary: different disciplinary, theoretical, and methodological perspectives often use different terminologies to describe the features of a complex phenomenon, and this is not a simple or trivial problem because these differences in language often reflect deep differences in how the world is conceived. Interdisciplinary collaboration often requires the negotiation, or invention, of a new, shared vocabulary, but this reintroduces the potential problem of language standardization. Interdisciplinary collaboration also requires a reexamination of methods of inquiry, and often involves scholars working with tools that are not professionally their own - what benefits are to be gained, for example, when a non-historian helps sift through archival data, or when an anthropologist helps conceptualize an experimental research design?

Finally, and perhaps most challengingly, interdisciplinary collaboration requires some negotiation around the criteria of evidence and truth that pertain to shared investigations. This is not as simple as, say, delimiting quantitative versus qualitative inquiry, because there are gulfs almost as deep within those categories as there are between them, where issues of evidence and truth are concerned. But interdisciplinary collaborations must engage such questions because the very nature of the collaboration requires that two or more persons will look at the same information and come to some agreement about what it means, what it proves, or what it stands for. Turning these differences from a problem into an intellectual resource is a primary challenge here - and needless to say can be a deeply contested one.

But there are other impediments to interdisciplinary collaboration that are also worth noting here. One is the tendency, already discussed, to want to "own" one's data and preserve it for one's own analysis and interpretation. Sharing it may entail that others "get credit," or even more problematically perhaps, others may interpret it in ways that are antithetical to one's own view, or even show it to be mistaken. As we have discussed, the capacities of new information technologies to make possible the "publication" - the making public - of one's data, along with one's analysis or interpretation of it, creates a huge new possibility in the way we view intellectual production, collaboration, and accountability within a community of inquiry. And yet, for reasons that do not need to be belabored, this potential continues to be resisted by many researchers.

The fact that scholars in humanities disciplines especially tend to be more individualistic also creates a significant impediment to collaboration. There are deep issues here about the intellectual and moral significance of formulating and advocating for a personal perspective in these disciplines (think of Nietzsche in philosophy for instance; could he have been part of a collaborative team?). We also need to acknowledge the institutional implications and reward systems that often drive scholars into this self-interested mode: the greater weight given to sole-authored publications in many disciplines, for example.

All of this returns us to the question of standards and criteria for educational research, and how these shape or delimit collaborative possibilities. If one accepts that mutual criticism and cross-checking are part of how a community of inquiry advances its knowledge base, then the question arises of how this can happen in interdisciplinary collaborations, where the methods of investigation and of confirmation are often quite diverse. One path is to make these methods of investigation and confirmation an element of reflective discussion and negotiation within the group. But a group may feel, with justification, that it is spending so much of its time hashing through those meta-questions that it never gets on with the substantive issues at hand.

Moreover, the benefits of mutual criticism and cross-checking within a community of inquiry need to be supplemented by the benefits of criticism from outside that community of inquiry. This may come from those who explicitly do not share the disciplinary, theoretical, or methodological perspectives of those working within it; or it may come from distinct standpoints of political or cultural difference. In either case, it may be the very resistance of such perspectives to the setting of common standards of investigation and of confirmation that makes their criticisms so important; by definition such criticisms must come from the outside.