Third Party Independent Research
Research Based Study Demonstrates the Positive Effects of the Camelot Mathematics Intervention Program on Student Learning
Dr. Ronald Thomas, Towson University
A third-party review of Camelot Learning’s research study substantiated the meaningful increases in the mathematics understandings and skills demonstrated by students completing the Camelot Learning Math Intervention Program. This research design and data review was completed by Dr. Ronald S. Thomas, a past winner of the Judith A. Ruchkin Research Award from the Maryland Association for Supervision and Curriculum Development. As the Assistant Superintendent for Educational Accountability for the Baltimore County Public Schools from 1996 to 2001, Dr. Thomas supervised the assessment and research/evaluation offices of this large urban-suburban school district. He is currently the Associate Director of the Center for Leadership in Education at Towson University.
Prior to the research study, pre- and post-tests were developed that were aligned with the standards of the National Council of Teachers of Mathematics (NCTM) and the objectives assessed on Level 15 of the Terra Nova standardized achievement test. Instrumentation was field tested with fifth graders at two elementary school sites. Data were gathered about the alignment of items with the NCTM standards, the grade-level appropriateness of the questions, and the time it took for students to complete the assessment.
A matching pretest-posttest control group design was then used in the spring of 2003 to evaluate the Camelot Learning Mathematics Program:
This design is often used when random assignment is impossible — that is, when intact groups must be used (Franekel and Wallen, 1993)
This type of research design controls for the following internal threats to validity that may have affected the results:
- History: Other instruction during the school day
- Maturation: Developmental changes that occurred in the students during the study
- Testing: Student learning from the pretest
- Instrumentation: Possible inconsistency of the measuring instruments or the way the data were collected
- Selection: Initial differences in the mathematics achievement of groups prior to the study
- Mortality: Differences in achievement between students who left school during the study and students who remained (Franekel and Wallen, 1993).
Five sites were selected to participate in the Camelot Learning Math Intervention Program research study. Two of the schools were Title I elementary schools, in urban areas, where fifth graders participated. Two of the sites were suburban middle schools, where sixth graders participated. One site was a rural elementary school including fourth, fifth, and sixth graders.
In each school, principals identified students to participate in the experimental group (N= 52) who were, according to data available at the school, deficient in fifth grade mathematics skills, as defined by the NCTM standards and commonly assessed on the Level 15 Terra Nova. Principals matched this group with a control group of students (N= 63) with skill levels that school data showed to be comparable to those of students in the experimental group but who were unable to attend an after-school program.
The experimental group attended the Camelot Learning’s math intervention program for 15 sessions. Four of the five schools offered the program twice a week, while one school offered the program three times a week. Teachers were trained in the program prior to starting instruction, and Camelot staff visited each site twice during the study to ensure consistent implementation. On-line technical assistance by Camelot staff was available to teachers as needed.
Results of the Study
Results of comparisons from pre- to post-testing showed an overall improvement rate that was three times higher for the Camelot Learning group than for the control group.
|N||Average Percent Test Score Change Pre- to Post-Test||Range of Percent Test Scores Change Pre- to Post-Test for Schools|
|52||+16.21||+15.9 to +16.49|
|Control group||63||+5.64||+10.16 to -1.77|
Students in the Camelot Learning Math Intervention Program showed an average 16.21% improvement in their scores from pre- to post-testing, whereas students in the control group improved on average only 5.64% from pre- to post-testing. Improvement for the five Camelot school groups ranged from +16.49% to +15.9%. Average performance improved in four of the five control groups, but at a substantially lower rate, from +10.16% to –1.77%. The difference between the improvement of Camelot students and the improvement of the control groups ranged from a 5.8% difference at one school to an 18.3% difference at another school.
These results indicate that completion of the Camelot Learning Math Intervention Program made a substantial difference in student performance on a test assessing the mathematics skills taught in the 15 session program.
Fraenkel, Jack R., and Norman E. Wallen. How to Design and Evaluate Research in Education. Second edition.
New York: McGraw-Hill, Inc., 1993.