Interplay Between Theory and Research

Theory and research are equally important to the process of accumulating knowledge through the scientific method (Bringle, 2003; Bringle & Hatcher, 2000). The process can begin at different points on the diagram (Figure 1). It may start with a preliminary theory that, through the deductive process, generates testable hypotheses that are evaluated through research, the results of which produce decisions about the theory (e.g., supported, refuted, need to revise). Alternatively, specific observations may be used to generalize principles that are conceptually developed into a theory that then guides subsequent research that evaluates research questions and deduced hypotheses. The presumption is that, in every case, there is a symbiotic relationship between theory and research, such that theory guides the research process, and research results arbitrate an evaluation of the appropriateness of the theory (e.g., supported, needs modification, refuted). This is true whether the research is quantitative or qualitative in nature. Figure 1 illustrates the importance of two types of connections between research and theory, namely that the relationship involves a cycle of both inductive and deductive processes.

Thus, theories are comprised of statements about the nature of constructs, their manifestations, and the relationships between constructs. Constructs are abstract or hypothetical entities. Critical thinking is a construct. No one can see critical thinking. Theorists can map the conceptual domain and identify attributes that are presumed to be indicative of good or poor critical thinking, but the construct itself does not exist in a tangible way. The manifestations of the construct (e.g., verbal or behavioral manifestations of critical thinking) may be apparent, help differentiate among individuals, and allow one to rank persons on some attribute associated with critical thinking, but the construct itself is not directly accessible. Variables, or the phenomena of interest in a research study, are the concrete manifestations of constructs that are either (a) quantitative, in that they vary in intensity or degree, or (b) qualitative, in that they differ in kind.

Operationalization refers to a statement about the specific way in which a quantitative or qualitative variable is measured, observed, documented, or manipulated in research. The progression from construct to variable to operationalization, thus, is a deductive process that goes from more abstract to more concrete. For example, reflection is assumed to be an integral component of designing a successful service-learning class. Reflection is defined as the "intentional consideration of an experience in light of particular learning objectives" (Hatcher & Bringle, 1997, p. 53). As such, reflection is a construct. There are many ways in which reflection can occur, including journals, directed writings, critical incident papers, group discussions, and portfolios. There are also dimensions on which these methods can vary (e.g., structured vs. unstructured). The implementation of these forms of reflection, their operationalization, could be a quantitative variable (e.g., some students are asked to write ten pages of journal entry whereas other students are asked to write 100 pages). Or, the operationalization of reflection could be a qualitative variable in that the activities differ in kind (e.g., some students are asked to write 20 pages of journal entry whereas other students engage in a series of group discussions). Thus, not only are there multiple variables associated with a construct, but there are also many ways to operationalize any one of the variables. In quantitative research, the key is to operationalize the construct in such a way as to be able to evaluate a hypothesis, which is a tentative statement about the expected result. Classically, qualitative research is characterized by a discovery-type approach in which no prior constraints are made on the observation methods or study results. In practice, most qualitative researchers do at least make an initial outline of what type of instruments and procedures they will use, and what types of questions they are seeking to answer.

In order to evaluate a hypothesis using the quantitative approach, the researcher must structure the data collection in such a way that inferences can be made. This requires adequate design and implementation of the research procedures (internal validity), utilization of reliable and valid measurements or observations, conducting appropriate analysis of the data (statistical validity), and making appropriate inductive inferences about the pattern of results to be able to draw conclusions about its practical and theoretical implications (external validity).

The remainder of this primer will outline the processes and procedures that are useful in conducting research on service-learning. Chapter 2 describes the process of designing servicelearning research, including the research cycle, qualities of good research, and differentiating between research and evaluation. The chapter will also provide a description of how to define research variables and give a list of sample variables that might be investigated in servicelearning research. Next, the chapter focuses on research designs, common problems in servicelearning research and how to address them, and ethical issues.

Chapter 3 focuses on measurement issues in service-learning research. First, a description of the most common types of assessment tools used in service-learning research, such as surveys, focus groups, and content analysis, is provided. Following this is a discussion of the characteristics of good measurement instruments (reliability, validity, and practical concerns). Then the chapter presents the pros and cons of an important decision point, whether to use existing tools, to adapt from those that have already been developed, or to create new tools to fit a specific purpose. Chapter 3 concludes with a list of resources for conducting online surveys.

Data analysis and interpretation is the focus of Chapter 4. The chapter starts with an introduction that includes a list of common pitfalls in analyzing and interpreting data, and moves to the most commonly used forms of quantitative analysis, descriptive and inferential statistics, are briefly discussed along with a special focus on the procedures more frequently used in service-learning research. This section is intended to be an introduction to the topic; for more indepth and specific information the reader is urged to consult a statistics manual and/or another researcher who has experience or expertise in this area. Following this is a discussion of generalizability and the inductive process of research interpretation, drawing conclusions, and making recommendations.

Chapter 5 begins with a list of potential avenues for disseminating research, and then we discuss qualities of good quantitative research articles. Following this is an annotated list of publication outlets for research on service-learning. Finally, the appendices provide an annotated list of service-learning research resources on the internet, online resources on research methodology and statistics, and a listing of potential funders for service-learning research projects.