Case Studies: Overview | Dallas | Houston | Plano
Used by a growing number of states and districts, value-added assessment is a new way to measure teaching and learning. Based on a review of students’ test score gains from previous grades, researchers can predict the amount of growth those students are likely to make in a given year. Thus, value-added assessment can show whether particular students have made the expected amount of progress, have made less progress than expected, or have been stretched beyond what they could reasonably be expected to achieve. Using these same methods, one can look back over several years to measure the impact that a particular teacher or school had on student achievement.
Value-added assessment gives educators a powerful diagnostic tool for measuring the effect of pedagogy, curricula, and professional development on academic achievement and provides all K-12 stakeholders a fair and accurate foundation on which to build a new system of accountability. This page and individual case study pages will focus primarily on how to choose an appropriate value-added model and use it as the foundation of an accountability system. For more information on how value-added can be used to drive classroom and school improvement, see the Value-Added as a Diagnostic page.
Student performance on assessments can be measured in two very different ways, both of which are important. Achievement describes the absolute levels attained by students on their end-of-year tests. Growth, in contrast, describes the progress in test scores made over the school year. In the past, students and schools have been ranked solely according to achievement. The problem with this method is that achievement is highly correlated with the socioeconomic status of a student’s family. As a result, this means that teachers and schools have historically been held accountable for factors that are outside of their control. In contrast, value-added assessment measures the growth students make, which can be attributed to the quality of instruction. In effect, it answers the question: how much value did the staff add to the students who attend their school?
Because individual students rather than cohorts are traced over time, each student serves as his or her own baseline or control. As a result, value-added assessment can offer the following benefits:
- Targeted student intervention. Value-added assessment tracks individual students over time. As such, it can show the progress made by particular students and project future performance.
- Accurate assessment of teacher and school impact. This method allows researchers to identify not only the progress made by individual students but also the extent to which individual teachers, schools, and districts have contributed to that progress.
- Fair basis for an accountability system. Unlike achievement at a single moment in time, which is best predicted by family background, value-added progress made by individual students is best predicted by the quality of instruction. As a result, value-added assessment provides a much fairer way to hold teachers and schools accountable for student learning.
- Data-driven school improvement. Value-added assessment provides data for administrators and teachers that can be used to inform instruction and drive school improvement efforts. For more information on these uses, see the Value-Added as a Diagnostic page.
- Contributions to research. Value-added assessment can isolate the impact of teachers and schools on student learning. As a result, researchers can study the behaviors of effective teachers and test the effectiveness of various forms of curriculum and professional development in different contexts.
There are currently a number of value-added models being used across the country. While all of the models track student progress over time, each model varies in how it measures a teacher’s or school’s impact on student learning, as well as how it serves as the basis of an accountability system. The following recommendations are based on current research and publications and provide districts and states with factors to consider when choosing a value-added model.
Statistical Considerations
Value-added assessment has sometimes been confused with simple growth, where a student’s score on last year’s test would be subtracted from this year’s test and attributed to the teacher as the “value” that was “added.” These simple models are an improvement on using absolute achievement to gauge a teachers’ impact on student learning, but they incorrectly assume that all of a student’s growth can be attributed to the teacher. When deciding which value-added model to use, districts should keep the following recommendations in mind. For more information on these statistical considerations, see the websites and tools section.
- Choose a rigorous value-added model. More rigorous value-added models provide a way of isolating the impact of instruction, by separating the growth of students into that which is contributed by the student and that which is contributed by the teacher. Each student receives a projected score based on prior achievement history, and this projected score is compared to the actual score they received at the end of the year. These rigorous models:
- Use multiple years of data. Using multiple years of data can help ensure that the estimate of a teacher’s or school’s impact on student learning is as accurate as possible.
- Include students with incomplete records. Because of absences or mobility, many students have incomplete records. Using all of the available data increases the accuracy of the results.
- Account for the contributions of various teachers. The model needs to be able to account for the fact that multiple teachers may instruct the same student in a given year.
- Design reliable and valid assessments. Assessments used must meet criteria of validity and reliability. They need to be closely aligned with the standards and curricula, have appropriate stretch at the ends of the distribution, and be available in fresh, non-redundant, and equivalent forms. For assistance in developing assessments that meet these criteria, refer to the Integrated Assessment page.
- Develop robust data and reporting systems. Each student and teacher must be assigned a unique individual identifier and teacher IDs should be linked to the students they taught, along with their students’ annual scores on the high-stakes summative exams. These data systems will need to be sufficiently robust to handle student mobility and determine how to apportion student progress to multiple teachers. Click here for additional information on implementing a data and reporting system.
Accountability Considerations
The results of value-added assessment be used to create three levels of teacher, school, and district effectiveness: (1) highly effective – those in which the students exceed expected growth; (2) effective – classrooms, schools, or districts where students on average are receiving expected growth in a year; and (3) ineffective – those where projections fall below what would have been expected. When using these results as the foundation of an accountability system, districts should keep the following recommendations in mind. For more information on how these recommendations have been implemented, see the case study pages.
- Use an external growth standard. To prevent competition, districts and states should not compare teachers’ performance with each other. Rather, they can use an external growth standard to determine levels of performance. For example, Tennessee uses the 1998 statewide average to set an external growth standard, and any teacher who hits that standard has met the expectation. This helps ensure that value-added assessment used for accountability purposes will not cause competition among educators.
- Only use value-added assessment as one of multiple measures. When used as the basis of teacher and school accountability, value-added scores should be part of a balanced system that includes teacher’s and administrator’s performance on observation rubrics. See the Teacher Evaluation page and the Administrator Evaluation page for additional information. Additionally, since value-added assessment can only be used to measure performance in tested subjects, districts and states need to develop mechanisms to include other educators. See the Compensating Educators in the Absence of Value-Added Assessment page for more detail.
- Invest all stakeholders in the system by providing high-quality training. States and districts interested in maximizing the utility and power of value-added metrics must accompany the implementation of a value-added system with high-quality value-added training throughout the school year. Providing additional time for collaboration among educators can help increase investment in the system. For additional information on training teachers, see the Value-Added Training page.
Recently, researchers have become increasingly interested in how to isolate the impact of teachers and schools on student learning, and policy makers have experimented with ways to use value-added modeling as the basis of school and individual accountability systems. It is important to note that researchers disagree over the rigor of various models and their ability to capture the effect of teachers and schools on student achievement. Several of the more recent studies have been documented below, and additional research conducted by specific organizations is also referenced in the websites and tools section.
- Comparisons Among Various Educational Assessment Value-Added Models
(William L. Sanders, The Power of Two – National Value-Added Conference, October 16, 2006)
This paper characterizes the differences among various classroom-level value-added models. Each of the tested models was applied to the same data structure from two different large school districts. The author discusses the advantages and disadvantages of each approach to value-added assessment and pays particular attention to the risks of misclassifying teachers when using the more simple and transparent models. - Evaluating Value-Added: Findings and Recommendations from the NASBE Study Group on Value-Added Assessment
(National Association of State Boards of Education, 2006)
The National Association of State Boards of Education (NASBE) formed a Study Group to assess critical factors that need to be in place when implementing value-added assessment. They discuss the two most common ways value-added assessment has been used as the basis of an accountability system – to evaluate schools and to assess teacher effectiveness. The publication closes with a checklist for policy makers to consider when implementing value-added assessment. - From Data to Bonuses: A Case Study of the Issues Related to Awarding Teachers Pay on the Basis of Their Students’ Progress
(Daniel F. McCaffrey, Bing Han, and J. R. Lockwood, National Center on Performance Incentives, February 2008).
This paper considers the reliability of various value-added models in the context of pay for performance systems. It describes the process of taking a large administrative database of student test scores and class assignments and turning it into bonus decisions for teachers. The choices at each step of the process are illuminated with careful consideration paid to the impact on the types of teachers who receive awards, the choice of performance measures, and decision rules for awarding bonuses. Read the policy brief. - Roundtable Discussion on Value-Added Analysis
(Working Group on Teacher Quality, Washington, DC, October 26, 2007)
In this brief, The Working Group on Teacher Quality sets out to create “a broader understanding of how value-added analysis of student achievement can be used as one indicator of teacher effectiveness and the implications this has for policy and practice.” The brief discusses three main questions: (1) what are the leading models for measuring student achievement gains, and what are the strengths and weaknesses of these models? (2) How is value-added analysis being used in states and/or school districts today to measure teacher effectiveness? (3) What is the best way to explain value-added analysis of student achievement – and how it relates to teacher effectiveness – to various stakeholders? - Would Accountability Based on Teacher Value-Added Be Smart Policy? An Examination of the Statistical Properties and Policy Alternatives
(Douglas N. Harris, National Conference on Value-Added Modeling, April 22-24, 2008)
With annual standardized student testing, it may be feasible to measure the contribution that individual teachers have on student achievement. But will these “teacher value-added” measures help to improve student achievement and learning? This paper addresses this question through a “policy validity framework” that includes three factors: (a) statistical validity, i.e., how well teacher value-added measures actually measure true teacher contributions to achievement; (b) purposes, i.e., whether the measures are intended to signal effective teachers or provide a path to improvement; and (c) costs. - Using Value-Added Models to Identify and Support Effective Teachers.
(Laura Goe, National Comprehensive Center for Teacher Quality, May 2008).
This paper examines how value-added models can be used to identify and support highly effective teachers. The author discusses the benefits and challenges of using value-added models and presents seven strategies – (1) use value-added as one way to identify effective and ineffective teachers, (2) use value-added as part of a teacher’s performance assessment, (3) use value-added to help make differential pay decisions, (4) use value-added in certification decisions, (5) use value-added to improve professional development, (6) use value-added to improve pre-service preparation programs, and (7) use value-added to determine the impact of characteristics and practices on student outcomes. These strategies are accompanied by case studies and relevant resources.
For additional reports and studies, see The Center for Greater Philadelphia’s resource page.
The following resources contain additional information for policy makers to consider when choosing a value added-model to use as the foundation of an accountability system. The websites and tools below provide access to additional research and organizations that have developed value-added models. To learn more about how value-added assessment is being used for accountability purposes, see the case study pages.
Additional research
- The Wisconsin Center for Education Research (WCER) organized a national conference to address some of the most important technical issues surrounding value-added modeling (VAM) as well as their implications for education policy. The research papers, conference, and related activities focused on two interrelated issues: how VAM can be used to identify effective policies and practices (school improvement) and how it can be used to identify effective teachers and schools (accountability). For a complete list of conference papers, click here.
- The National Center on Performance Incentives addresses the use of performance incentives in public education. Many of the Center’s studies have examined the implications of using value-added assessment as the basis of an accountability and performance incentive system. For a complete list of their papers, click here.
Relevant organizations
- SAS® EVAAS® for K-12 builds on the Tennessee Value-Added Assessment System (TVAAS) methodology developed by Dr. William L. Sanders and his colleagues at the University of Tennessee. The EVAAS model is now being used statewide in Tennessee, Ohio, Pennsylvania, and North Carolina, as well as in hundreds of districts across the country.
- The Value-Added Research Center develops, applies, and disseminates value-added and longitudinal research methods to evaluate the performance and effectiveness of schools and teachers, and educational programs and policies. The Center has developed and implemented value-added models in Madison, Milwaukee, and Chicago and provides technical assistance, professional development, and evaluation support.
- Pearson Educational Measurement (PEM) assists state clients in researching, implementing, and supporting growth models. Specifically, PEM developed both the Texas Learning Index and the Texas Growth Index and has advised multiple states, such as California, Florida, New York, New Jersey, Minnesota, Washington, and Utah regarding measuring student growth and value-added models.
- Evaluating Value-Added Models for Teacher Accountability
(Daniel F. McCaffrey, et al., RAND, 2003).
This book clarifies the primary questions raised by the use of value-added modeling for measuring teacher effects, reviews the most important recent applications of value-added modeling, and discusses a variety of statistical and measurement issues that might affect the validity of value-added modeling inferences. The authors close by providing recommendations for the use of value-added models and suggesting directions for further research. - Grading Teachers, Grading Schools: Is Student Achievement a Valid Evaluation Measure?
(Jason Millman, Corwin Press, 1997).
This book examines four high-profile, high-stakes methods for evaluating teaching through learning gains – The Oregon Teacher Work Sample Methodology, The Dallas Value-Added Accountability System, The Tennessee Value-Added Assessment System, and The Kentucky Instructional Results Information System. It closes by providing a synthesis of the criteria for an accountability system and offering a variety of perspectives on how student achievement can be used as a valid evaluation measure. - Value Added Models in Education: Theory and Applications
(Robert W. Lissitz, ed., JAM Press, 2005)
In this book, the editor selected authors who could represent a variety of experiences with Value Added Modeling (VAM). These included theoretical developments such as modeling the effects of test linking on growth analysis, what it means to do VAM applications in a valid way, what has been learned about VAM from applications in local school systems and from analyses of national and state data sets and the application of VAM to important questions in education policy.