Tugas Kelompok 5 Terjemahan (Patta Bundu)

Quantitative Research Design

As indicated above, quantitative research focuses on analysis of numeric data. The approach often follows particular scientific methods (e.g., design, sampling, measurement). Quantitative research can be classified into three types shown in Table 4 (Trochim, 2006).

Table 4. Quantitative Research DesignsResearch Design            Non-Experiment           Quasi-Experiment         Experiment

Random Assignment of Subjects to Group        No       No       Yes

Control Group or Multiple Waves of Measurement       No       Yes      Yes

Non-experimental designs

Non-experimental designs do not involve random assignment of subjects to groups, nor is there a control or comparison group. Non-experimental designs also do not involve multiple waves of measurement. This type of design is very useful for descriptive research questions such as:

What percentage of students is involved in community service?

Do male students have different attitudes than females about the need for social service agencies?

How many faculty members have taught a service-learning course in the past three years?

The simplest, very common form of non-experiment is a one-shot survey. For example, a researcher might conduct a survey of opinions about community activism. In a variation on this, a researcher might measure attitudes at the end of a semester in a service-learning course. This design (called the post-test only, single group design, Campbell & Stanley, 1963) lacks a comparison, and therefore the ability to conclude that the outcome was the result of the servicelearning experience.

Correlational research designs evaluate the nature and degree of association between two naturally occurring variables. The correlation coefficient is a statistical summary of the nature of the inferred association between two constructs that have been operationalized as variables. The correlation coefficient contains two pieces of information (a) a number, which summarizes the degree to which the two variables are linearly associated; and (b) a sign, which summarizes the nature or direction of the relationship. The numeric value of a correlation coefficient can range from +1.0 to -1.0. Larger absolute values indicate greater linear association; numbers close to zero indicate no linear relationships. A positive sign indicates that higher values on one variable are associated with higher values on the other variable; a negative sign indicates an inverse relationship between the variables such that higher values on one variable are associated with lower values on the other variable.

Causal inferences are very difficult to make from a single correlation because the correlation does not assist in determining the direction of causality. For example, a positive correlation between volunteering and self-esteem indicates that more volunteering is associated with higher self-esteem. However, the correlation does not differentiate among at least three possibilities, (a) that volunteering affects self-esteem; (b) that self-esteem promotes volunteering; or (c) that a third variable (e.g., self-efficacy) is responsible for the correlation between selfesteem and volunteering.

Experimental designs

In contrast to correlational methods that assess the patterns between naturally occurring variables, experiments manipulate a variable, the independent variable, and see what consequence that manipulation has on another variable, the dependent variable. Not all experimental designs are equally good at allowing the researcher to make causal inferences. An outline of experimental designs is presented below. Note that this section is only intended to be an introduction to the topic. For more specific information on experimental research design, the reader should consult a research methodology text (e.g., Campbell & Stanley, 1966; Cook & Campbell, 1979; Cozby, 2009; Kerlinger, 1986). Consultation with experienced research colleagues is also helpful. Some online resources on design are listed in the appendices of this document.

The strongest research design in terms of drawing cause-and-effect conclusions (internal validity) is the randomized or true experiment. In this "gold standard" of quantitative research designs, subjects are randomly assigned to different groups or treatments in the study. Traditionally these groups of subjects are referred to as the experimental or treatment group(s) (e.g., students in a service-learning course) and the comparison or control group(s) (e.g., students in a traditional course). Note that random assignment of subjects to a group in an experiment is different from the random selection of subjects to be involved in the study. Random assignment makes it unlikely that the treatment and control groups differ significantly at the beginning of a study on any relevant variable, and increases the likelihood that differences on the dependent variable result from differences on the independent variable (treated group vs. control group). Random assignment controls for self-selection and pre-existing differences between groups; random selection or sampling is relevant to the generalizability or external validity of the research.

There are a variety of designs that utilize random assignment of subjects, but true experimental studies are relatively rare in service-learning research, as in most educational research. This is because it is usually difficult, especially in higher education settings, to randomly assign students to service-learning versus traditional courses or to different levels of a variable in the instruction. Nevertheless, the U.S. Department of Education has proposed that all research use random assignment so that education practice can be based on research with internal validity (www2.ed.gov/rschstat/eval/resources/randomqa.html). A close approximation of random assignment occurs when students are not aware that some sections of a course will be service-learning and some will not be service-learning when they register for courses (Markus, Howard, & King, 1993; Osborne et al., 1998). Also, there may be opportunities to randomly assign students to different conditions in service-learning classes (e.g., students are randomly assigned to (a) written reflection or (b) reflection through group discussion).

Quasi-experimental designs

Like experimental designs, quasi-experimental designs involve the manipulation of an independent variable to examine the consequence of that variable on another (dependent) variable. The key difference between experimental and quasi-experimental designs is that the latter do not involve random assignment of subjects to groups. A large portion of past quantitative research on service-learning involves quasi-experimental design. We do not intend to comprehensively cover all quasi-experimental designs in this primer; instead we will discuss some designs commonly seen in service-learning research. For more advanced information, or for information on other designs not discussed here, we recommend that the reader consult a graduate-level research methodology text (e.g., Campbell & Stanley, 1966; Cook & Campbell, 1979; Cozby, 2009; Kerlinger, 1986). Consultation with experienced research colleagues is also helpful. In addition, some online resources are listed in the appendices of this document.

One aspect of designing a study relates to temporal arrangements. Some researchers are interested in the developmental aspects of service-learning, or in the effects of service-learning over time. For example, they may be interested in the question of whether involvement in volunteer service during high school leads to increased involvement in service during and after college. There are two approaches to designing research to answer these types of questions. In a cross-sectional design the researcher gathers data from several different groups of subjects at approximately the same point in time. For example, a researcher might choose to conduct interviews with groups of college freshmen, juniors, graduating seniors, and alumni. Longitudinal studies (sometimes also called time series designs) involve gathering information about one group of people at several different points in time. Astin, Sax, and Avalos (1999), for example, collected survey data from entering freshmen in 1985, then surveyed the same group of students four years later in 1989, and again to the now-alumni in 1994-95. Longitudinal studies are extremely valuable sources of information for studying long-term consequences of servicelearning, but they are rare in service-learning research because of the practical, technical, and financial difficulties in following a group of people over time.

Other researchers focus their interest on questions that do not relate to developmental issues or impact over a long period of time. In fact, many if not most service-learning studies are limited to one semester or sometimes one year in length. A common strategy is to give an attitude measure to students in a service-learning course at the beginning and end of a semester. This pre-test, post-test single group design examines the difference between pre- and post-test scores for one group of students. Unfortunately, there is no assurance that the difference in pretest and post-test scores is due to what took place in the service-learning class. The difference in attitudes could be attributable to other events in the students' lives (history), natural growth changes (maturation), dropout of the least motivated students during the course (mortality), or carryover effects from the pre-test to the post-test (testing).

Another experimental design is the post-test only, static groups design1, which compares the outcomes of a pre-existing treated group to the outcomes of a pre-existing untreated group. Using this design, an instructor could give an attitude scale at the end of the semester to a service-learning section of the course and also to a section that did not contain service-learning. This design suffers from the limitation that it is not possible to conclude that the difference on the dependent variable, attitudes, is due to the difference in instruction because it is not known if the two groups were equivalent in their attitudes at the beginning of the semester.

An alternative arrangement, the nonequivalent (or untreated) control group design with pre- and post-test, is to give a pre-test and a post-test to both a service-learning section of a course and to a traditional section that does not include a service component. In this design the researcher can evaluate whether or not the two groups were equivalent at the beginning of the semester, but only on the measured variables. A second step is to examine the pattern of changes between the two groups across the semester.

The biggest problem with the nonequivalent groups design is self-selection bias, described above in the section "Common Problems in Service-Learning Research." Frequently in higher education, and sometimes in high school settings, service-learning courses are optional for graduation, and/or service is an optional component of a particular course. That is, students must select or opt to be in the class and to participate in service. The result is that students are nonrandomly assigned to the treatment group (service-learning course) and thus there is non-random assignment of students to groups. There are likely to be many differences between students who choose to be involved in service-learning classes and those who do not (Eyler& Giles, 1999). Even with a pre-test to compare equivalence of groups at the beginning of the study, a researcher could never completely eliminate the possibility that there are differences on other, unpre-tested variables, or that post-test differences are due to inherent differences in the groups, rather than differences in the educational intervention. Sometimes researchers use multiple measures preand post-treatment to help assess whether groups are equivalent on several relevant variables; statistical procedures (i.e., analysis of covariance) also can help control for differences between treatment and non-treatment groups, but only for measures that are obtained prior to the educational intervention. Of course, the best solution is random assignment of students to groups, which makes this an experimental design, rather than a quasi-experimental one.

A common variation of the nonequivalent groups design occurs when students in two sections (one including a service component and one not) of a course are being compared, but the two sections are taught by different instructors. This creates a problem in interpretation because one cannot infer that post-test differences in scores are due to the style of pedagogy (service-learning) rather than other differences between instructors. Another variation is to compare two sections of the same course, one involving service and one not, but taught in different semesters. In this case it is possible that differences in post-test scores are due to events extraneous to the study, which happened during one semester but not the other. In sum, it is important for the researcher to be aware of potential pitfalls of any research design and to take these into account when drawing conclusions from the study.