Case Studies: Overview | APEP | ISLLC | VAL-ED
The Vanderbilt Assessment of Leadership in Education (VAL-ED), developed under a three-year project funded by the Wallace Foundation, is a multi-component standards-based leadership assessment system for measuring administrators' performance. The results of VAL-ED can be used for both summative and formative administrator evaluation, as well as to guide professional development efforts. VAL-ED has been through three phases:
- Phase 1. In this phase, the designers drew on the research base to develop a conceptual framework for effective leadership and to design an instrument for measuring administrator performance (Porter et al. 2006; Murphy et al. 2006; Goldring et al. 2007).
- Phase 2. A series of pilot studies and other evaluations were completed to create both paper and electronic versions of two forms of the instrument. The instrument was then field tested in 100 elementary schools, 100 middle schools, and 100 high schools in 53 districts and 27 states. Expert panelists were asked to help set performance standards and evaluate the final outcomes (Porter et al. March 2008; Porter et al. August 2008).
- Phase 3. Currently, VAL-ED is being implemented in a number of districts across the country. In addition, the authors are continuing to study the instrument's reliability and validity under a four-year grant from the Institute of Education Sciences.
The VAL-ED assessment system includes components and processes of learning centered leadership, offers valid and reliable instruments and processes for evaluation, and provides resources for interpreting results. These key elements are discussed in more detail below.
Learning Centered Leadership
Learning centered leadership serves as the foundation of the VAL-ED assessment system. Principals are assessed based on their performance on six core components related to student learning and six key processes of effective leadership. Murphy et.al describe the conceptual model and research base undergirding each of these components and processes (Murphy et al. 2007).
Core Components
- High Standards for Student Learning. Individual, team, and school goals for rigorous student academic and social learning
- Rigorous Curriculum (content). Ambitious academic content provided to all students in core academic subjects
- Quality Instruction (pedagogy). Effective instructional practices that maximize student academic and social learning
- Culture of Learning and Professional Behavior. Integrated communities of professional practice in the service of student academic and social learning and a healthy school environment in which student learning is the central focus
- Connections to External Communities. Linkages to family and/or other people and institutions in the community that advance academic and social learning
- Performance Accountability. Leadership holds itself and others responsible for realizing high standards of performance for student academic and social learning. There is individual and collective responsibility among the professional staff and students.
Key Processes of Leadership
- Planning. Principals articulate shared direction and coherent policies, practices, and procedures for realizing high standards of student performance.
- Implementing. Principals engage people, ideas, and resources to put into practice the activities necessary to realize high standards for student performance.
- Supporting. Principals create enabling conditions, secure and use the financial, political, technological, and human resources necessary to promote academic and social learning.
- Advocating. Principals promote the diverse needs of students within and beyond the school.
- Communicating. Principals develop, utilize, and maintain systems of exchange among members of the school and with its external constituencies.
- Monitoring. Principals systematically collect and analyze data to make judgments that guide decisions and actions for continuous improvement.
Valid and Reliable Instruments
The evaluation instrument contains 72 items, which define the core component and key process subscales. Respondents indicate sources of evidence for each of the items and then rate principals on a scale of 1 (Ineffective) to 5 (Outstandingly effective). For example, see the Sample Set of Items for Core Component One.
VAL-ED's items have been shown to meet or exceed widely accepted fairness criteria for assessments (Elliott and Frank 2008). The instrument has been designed so it (1) works well in a variety of settings and circumstances, (2) is construct valid, (3) is reliable, (4) yields diagnostic profiles for formative purposes, (5) can be used to measure progress over time, and (6) predicts important outcomes. Porter et al. 2008 describe the multistage development process the team used to collect validity and reliability evidence (Porter et al. March 2008).
Process of Evaluation
VAL-ED is a 360 degree, evidence-based assessment of leadership behaviors. Multiple evaluators - principals, teachers, and supervisors - are asked to rate the principal on each of the 72 leadership behaviors and provide evidence for their rating. Completing the evaluation takes approximately 20 to 25 minutes. To ensure accuracy, respondent confidentiality must be maintained throughout the process.
Interpreting Results
Results of the VAL-ED instrument are reported on two sets (core components and key processes) of six scales and total score. National norms have been set so principals' scores can be reported in percentile rank. A standard-setting study established performance levels so that the results from these instruments also can be used to determine whether a principal is performing at a "below basic," "basic," "proficient," or "distinguished" level (Porter et al. August 2008).
- Distinguished. A distinguished leader exhibits leadership behaviors of core components and key processes at levels of effectiveness that over time are virtually certain to influence teachers to bring the school to a point that results in strong value-added to student achievement and school learning for all students.
- Proficient. A proficient leader exhibits leadership behaviors of core components and key processes at levels of effectiveness that over time are likely to influence teachers to bring the school to a point that results in acceptable value-added to student achievement and social learning for all students.
- Basic. A leader at the basic level of proficiency exhibits leadership behaviors of core components and key processes at levels of effectiveness that over time are likely to influence teachers to bring the school to a point that results in acceptable value-added to student achievement and social learning for some sub-groups of students, but not all.
- Below Basic. A leader at the below basic level of proficiency exhibits leadership behaviors of core components and key processes at levels of effectiveness that over time are unlikely to influence teachers to bring the school to a point that results in acceptable value-added to student achievement and social learning for students.
Score reports provide an overview of assessment results for core components and key processes, document sources of evidence, and conclude with a summary of principals' relative strengths and areas for growth. Results are displayed graphically and in tabular form and include principals', supervisors' and teachers' mean effectiveness ratings on each scale and total score.
- The VAL-ED Technical Manual is a comprehensive living document that documents all phases of the development process and includes sample evaluation forms and reports.
- The VAL-ED Users' Guide includes tips for conducting assessments and guidelines for interpreting and using VAL-ED results.
- This VALED PowerPoint provides a thorough overview of the VAL-ED program and highlights the development process, characteristics of the instrument, and key purposes and uses.