Experiment 4 Math Program Evaluation – Optional 2017
This assignment involves the evaluation of an experimental pedagogical approach to teaching math at the elementary and middle school levels in a particular local school district. Briefly, simple random samples of 84 third graders and 84 eighth graders were selected because these are particularly critical years for standardized texting procedures. One-half of each of these two groups of students were randomly assigned to the experimental program and the others were randomly assigned to the corresponding control groups, who were taught math using the traditional approach for the duration of the semester. In addition, one half of the 42 students in each of these four groups (as defined by grade and the experimental vs. control variable) were randomly selected to take a pretest in math at the outset of the semester, with all 168 students participating in the experiment taking the same posttest at the end of the semester.
The data from this experiment are contained in an SPSS file named “Experiment4Math.sav” which contains the following variables:
|Grade||3 = third grade, 8 = eighth grade|
|Gender||1 = male, 0 = female|
|Gradstat||1 = at grade level, 0 = below grade level in academic standing at start of semester|
|Program||1 = treatment group – experimental, 0 = control group – traditional approach|
|Pretesting||1 = Yes – pretested, 0 = No – not pretested|
|Pretest||pretest score on a scale ranging from 0 to 100|
|Posttest||posttest score on a scale ranging from 0 to 100|
|Testing||1 = pretested, 0 = not pretested|
|Groups||Group as defined by the program and testing variable|
|GainScore||Gain score between pretest and posttest; Posttest – Pretest|
Reporting and Analysis
- What is this analysis about, and what is the central research hypothesis to be tested in this evaluation? Identify all the variables that are included in the data set you developed in this study and indicate why each might be relevant.
- What kind of research design are you using here, and why? Be specific in your description of how this evaluation was designed, and draw a schematic to represent it as well, showing the number of cases in each group structured into the design.
Experiment 4 Math Program Evaluation continued.
- Describe briefly what kind of a sampling strategy you used in this study, and provide a brief rationale for doing it that way.
- Run appropriate descriptive statistics to get a sense of the parameters of interest in this evaluation and check on the extent to which baseline data (those measures that describe cases at the outset of the evaluation) meet the assumptions underlying the research design.
- Run an appropriate means procedure — and as always use the pivoting trays in SPSS to format the resulting output when needed– to observe the joint effects of the Program and Pretesting variables on Posttest scores. Re-render the resulting table in clear format in the body of your report and provide a complete interpretation.
- Develop a simple regression model to gain an initial sense of whether, and the extent to which, the experimental program outperformed the traditional approach.
- Obtain an appropriate multiple regression model to examine the effect of the treatment variable while controlling for grade, gender, grade status (at or below grade level), and testing. This model should be run on al 168 cases in the aggregate.
- In addition, you should also run a multiple regression model on only the subset of students who were pretested to analyze the effect of the experimental program on posttest scores controlling for pretest scores and other relevant variables.
- State your conclusion regarding the effectiveness of the experimental approach to teaching math at elementary and middle schools and its implications for local public school systems.
Limitations of the Research
- Discuss any limitations of this research in terms of research design, sampling, and the measurement model. What suggestions, if any, do you have regarding future research in this area?