Saxon Research 1
Saxon Research 1
Table of Contents
Foundational Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Theoretical Framework for Saxon Math . . . . . . . . . . . . . . . . . 4
Incremental Instruction Distributed Across the Level . . . . . . . 5
Continual Practice Distributed Across the Level . . . . . . . . . . . 5
Cumulative Assessment Distributed Across the Level . . . . . . 6
Efficacy Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Historical Effectiveness of Saxon Math:
Elementary & Middle School . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Experimental Studies of Saxon Math:
Elementary & Middle School . . . . . . . . . . . . . . . . . . . 7
Elementary School . . . . . . . . . . . . . . . . . . . . . . 7
Middle School . . . . . . . . . . . . . . . . . . . . . . . . 10
Eighth Grade . . . . . . . . . . . . . . . . . . . . . . . . . 11
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Foundational Research
A well-articulated curriculum challenges students to learn increasingly more
sophisticated mathematical ideas as they continue their studies. John Saxon,
founder of Saxon Publishers, had a similar philosophy in mind when in the
early 1980s he developed his theory-based distributed approach to
mathematics instruction, practice, and assessment. Saxon’s approach has
evolved to include a K–12 textbook series with a comprehensive approach
Effective concept to mathematics.
development involves Because smaller pieces of information are easier to teach and easier to learn,
the Saxon Math series was developed by breaking down complex concepts
incremental skill instruction, into related increments. The instruction, practice, and assessment of those
increments were systematically distributed across each grade level. Practice
distributed throughout a
is continual, and assessment is cumulative. The Saxon approach differs from
school year. most programs in that it distributes instruction, practice, and assessment
throughout the lessons and school year instead of massing these elements.
In a massed approach, instruction, practice, and assessment of a skill or
concept occur within a short period of time and are clustered within a single
chapter or unit. In the Saxon Math program, as students encounter new
increments of instruction, they also continually review previously introduced
math concepts. Frequent assessments of new and old concepts are
encountered throughout the lessons, ensuring that students truly integrate
and retain critical math skills.
Figure 1.
Growth in TAAS Texas Learning Index by group (and statewide)
Efficacy
85
Saxon
Studies 1
83.8 84.0
84
Control
83.6 Historical Effectiveness of
Statewide
Saxon Math: Elementary
Texas Learning Index
82.7
83
and Middle School
81.6 82.4
Since 2005, Harcourt Achieve has
82 81.5
contracted with PRES Associates—an
80.8
81
external, independent educational
81.2
research firm—to conduct analyses
80.7
80 using longitudinal state assessment
data to document the effectiveness of
79 Saxon’s elementary and middle school
Pre-Saxon 6th Grade 7th Grade 8th Grade
math programs over time in several
states, including South Carolina
(Resendez, Sridharan, & Azin, 2007),
California (Resendez & Azin, 2007), Georgia (Resendez & Manley, 2005),
and Texas (Resendez, Fahmy, & Manley, 2005; Resendez, Sridharan, & Azin,
2006). Analyses were conducted during specified years on school and
student-level achievement data that compared users of Saxon Math to those
who used other math curricula during the same years.
1
For further information on the evaluation reports cited in this section, please visit the Saxon Web
site at www.SaxonMath.com
The South Carolina study (Resendez et al., 2007) found that among Saxon
schools in South Carolina there is significant growth in achievement on the
statewide math assessment (Palmetto Achievement Challenge Test [PACT])
within both elementary and middle school grade levels (see Figures 2 and 3).
Students using Saxon Math in South Carolina from
2002 to 2006 have shown growth year after year in
Figure 2.
math achievement. Additional findings from South Elementary Saxon Students’ PACT Match Performance
Carolina suggest there are increasing trends in math (Growth from 3rd–5th grade)
performance among all subgroups in Saxon elementary
650
and middle schools, especially among limited English
600
proficient (LEP) students. Analyses found that LEP
550 502.9
students showed accelerated rates of math
PACT Scale Score
500
performance over time; specifically, a closing of the
450 405.5
achievement gap between Saxon LEP and non-LEP
400
students was shown to exist between these groups. In
350 306.6
addition, preliminary analyses, examining aggregated
300
PACT scores only, showed that the average
250
performance of both elementary and middle school
200
Saxon students was significantly higher than the 3rd Grade 4th Grade 5th Grade
Experimental Studies of
Saxon Math: Elementary and
Middle School Figure 3.
A number of experimental and quasi-experimental Middle School Saxon Students’ PACT Match Performance
evaluations of the Saxon Math program (K–8) have (Growth from 6th–8th grade)
also been conducted through independent research
850
organizations, including universities and school-district 782.0
800
evaluation departments.
750
684.2
PACT Scale Score
Results indicated that, overall and for each grade level, Saxon students made
The Saxon approach significant gains on all three SAT 9 math achievement measures (overall
math achievement, math problem solving, and math procedures) over the
differs from most course of the school year (see Figure 4). Students in subgroups that are
programs in that typically regarded as academically, economically, or culturally disadvantaged
who were in Saxon schools made significant gains on all three SAT 9
it distributes instruction, achievement measures. Specifically, significant gains in achievement were
seen over time for English language learners, students who qualified for free
practice, and assessment… or reduced-priced lunch, students who were classified as special education,
throughout the lessons and minority group members.
Figure 4.
Average SAT 9 Math Achievement Growth for Saxon Students, Grades K–3
576.00
Scaled Scores
580
560 553.35
539.31
540
520
500
480
Overall Math Achievement Math Problem Solving Math Procedures
Note. Math Problem Solving and Math Procedures, Grades 2 and 3 only.
All differences significant at the 95% confidence interval.
The ITBS scores from the 1993–1994 school year were collected and used
to evaluate growth in mathematic skill over
the implementation time for both groups.
52
50.6
differences between groups on the Math
50
Concepts, Science, and Social Studies
48
tests were significant (p < .05). These
46
results indicate that, collapsed across
grade levels, students who used Saxon 44
Scott Foresman program (see Figure 5). All differences significant at the 95% confidence interval.
Figure 6. Figure 7.
Saxon Math students’ TerraNova Math Computation postest scores of Saxon Math
performance at pre- and post testing and control students
660 660
652 652
650 650
640 640
630 630
620 620
610 610
600 600
Control Saxon
Math (MC & CR) Math Computation
Note: These are adjusted posttest scores after controlling for initial pretest,
MC = mulitple choice, CR = constructed response minority status, attendance, grade, special education, class size,
free/reduced-price lunch, and school.
Eighth Grade
Multiple investigations of the Saxon methodology have also be conducted at the
eighth-grade level to examine the effectiveness of the Saxon Algebra I and
Algebra ½ programs. Results of a 9-week study of Algebra I conducted by Clay
(1998) found that Saxon Math was effective at increasing math achievement
scores on teacher-created criterion-referenced math tests and helped students
overcome an initial math deficiency to bring about greater gains in math
achievement than did a control group using a traditional program.
Crawford and Raia (1986) examined the effectiveness of Saxon Algebra ½
with eighth-grade students in five different middle schools. Achievement on
the California Achievement Test (CAT) for students in Saxon classrooms
Figure 8. Figure 9.
Posttest Comparison of Math Computation Ability Mathematics Achievement Posttest Comparison for 6th Grade
Students using Saxon Math Vs. a Control Group Math Product
800
Math Computation Scale Scores
70
770 60
51.23
761 50 44.01
760
40
750 30
20
740 10
Saxon Approach (N = 65) Traditional Approach (N = 146)
0
Note: Group differences for the adjusted posttest means significantly Saxon Approach (N = 324) Traditional Approach (N = 130)
different at the 95% confidence interval.
A post-hoc effect size was calculated from the exisiting data to be d = .73.
5
3.87
4
…all these results
3
Note: The average gain for the Saxon Math group on Math Computation and Total Math
effectiveness of the score was significaltly higher that the Control group at the 95% confidence interval.
Post-hoc effect sizes were calculated from the exisitng data to be d = .55 for Math
Computation and d = .62 for Total Math.
Saxon Math programs.
was compared to those students in classrooms using a more traditional
approach to math instruction. After controlling for pretest differences, it was
found that the results significantly favored the Saxon program. A second
analysis matching students in the treatment and control groups by
preimplementation achievement level found that Saxon students made
significantly higher gains from pre to post on the CAT Math Computation
subtest and the Total Math score than did control group students (see Figure
10). A final analysis examined achievement on only those objectives covered
by both programs. The analysis found that the results were significantly
different in favor of the Saxon group, indicating that the higher gains in
achievement on the CAT for the Saxon Math students were not due to
performance on objectives that were unique to the Saxon program.
Blair, J. (2000). ETS study links effective teaching methods to test-score gains.
Education Week, 20(8), 24.
Crawford, J. R., & Raia, F. (1986). Executive summary: Analysis of eighth grade math
texts and achievement. Oklahoma City, OK: Oklahoma City Public Schools, Planning,
Research, and Evaluation Department.
Dempster, F. (1988). The spacing effect: A case study in the failure to apply results to
psychological research. American Psychologist, 43, 627–634.
English, H. B., Wellburn, E. L., & Killian, C. D. (1934). Studies in substance memorization.
Journal of General Psychology
Psychology, 11, 233–260.
Gagne, R. M. (1965). The conditions of learning. Austin, TX: Holt, Rinehart, and
Winston.
Gagne, R. M., & Briggs, L. J. (1974). Principles of instructional design. Fort Worth, TX:
HBJ College Publisher.
Good, T. L., & Grouws, D. A. (1979). The Missouri mathematics effectiveness project.
Journal of Educational Psychology, 71, 355–362.
Klapp, S. T., Boches, C. A., Trabert, M. L., & Logan, G. D. (1991). Automatizing alphabet
arithmetic: II. Are there practice effects after automaticity is achieved? Journal of
Experimental Psychology: Learning, Memory, and Cognition, 17, 196–209.
Lafferty, J. F. (1994). The links among mathematics text, students’ achievement, and
students’ mathematics anxiety: A comparison of incremental development and
traditional texts. Unpublished doctoral dissertation, Widener University, Chester,
Pennsylvania.
Mayfield, K. H., & Chase, P. N. (2002). The effects of cumulative practice on mathematics
problem solving. Journal of Applied Behavior Analysis, 35, 105–123.
Nguyen, K., & Elam, P. (1993). The 1992–93 Saxon Mathematics program evaluation
report. Oklahoma City, OK: Oklahoma City Public Schools.
Peckham, P. D., & Roe, M. D. (1977). The effects of frequent testing. Journal of
Research and Development in Education, 10, 40–50.
Pirolli, P. L., & Anderson, J. R. (1985). The role of practice in fact retrieval. Journal of
Experimental Psychology: Learning, Memory, and Cognition, 11, 136–153.
Resendez, M., & Azin, M. (2006, November). Saxon Middle School Math Randomized
Control Trial. Austin, TX: Harcourt.
Resendez, M. & Azin, M. (2007, January). The relationship between using Saxon
elementary and middle school math and student performance on California statewide
assessments. Austin, TX: Harcourt.
Resendez, M., Fahmy, A., & Manley, M. A. (2005, April). The relationship between using
Saxon middle school math and student performance on Texas statewide assessments.
Austin, TX: Harcourt.
Resendez, M., & Manley, M. A. (2005, December). The relationship between using
Saxon elementary and middle school math and student performance on Georgia
statewide assessments. Austin, TX: Harcourt.
Resendez, M., Sridharan, S., & Azin, M. (2006, June). The relationship between using
Saxon elementary school math and student performance on Texas statewide
assessments. Austin, TX: Harcourt.
Resendez, M., Sridharan, S., & Azin, M. (in press). The relationship between using
Saxon Math at the elementary and middle school levels and student performance on
the South Carolina statewide assessment. Jackson, WY: PRES Associates.
Reynolds, J. H., & Glasser, R. (1964). Effects of repetition and spaced review upon
the retention of a complex learning task. Journal of Educational Psychology, 55,
297–308.
Rohm, R. A., Sparzo, F. J., & Bennett, C. M. (1986). College student performance under
repeated testing and cumulative conditions: Report on five studies. Journal of
Educational Research, 80, 99–104.
Rosenshine, B., & Stevens, R. (1986). Teaching functions. In M.C. Wittrock (Ed.),
Handbook of research on teaching: Vol. 3. (pp. 376–391). New York: Macmillan.
Usnick, V. F. (1991). It’s not drill and practice, it’s drill or practice. School Science
and Mathematics, 91, 344–347.
9994053396 2760/12.5M/PR/HORI/10–07