Advancing Accountability Initial Waiver Analysis: February 2012

The 10 principles of effective accountability systems

Federal and state education reform efforts, including reauthorization of the No Child Left Behind Act (“NCLB”), Elementary and Secondary Education Act (ESEA) Waivers, Common Core State Standards, and Race to the Top, hold promise for turning around our public schools and providing students with a high quality education that prepares them for the future. But, if there is no accountability for student success, then these initiatives become empty promises.

There is no denying that NCLB is not perfect. No law is. But these imperfections are primarily a symptom of the state of educational reform at the time of the law’s passage in 2002. Before NCLB, few states were assessing students annually, and few were disaggregating the assessment data they did have. NCLB’s requirements for annual testing and for disaggregation of results across subgroups changed these norms. As a result, we now have more educational data than ever. We know where serious, systemic problems exist. We have shown a spotlight on the pervasive achievement gap. Armed with these important tools and insights, we are now prepared to develop reform strategies that more directly target school needs and their students. And we are ready to move from a “status” or point-in- time evaluation of performance to measurement of student growth. These crucial next steps in educational accountability must be supported.

Over the last decade, we’ve seen improvements among all student groups, in particular for mathematics. States that embraced the notion of consequential accountability embodied within NCLB saw positive results for their students, especially for African American and disadvantaged students. As an example, African American students increased their National Assessment of Education Progress (NAEP) scores by 21 points in mathematics between 2000-2011 – roughly two grade levels of improvement. Even with this progress, national and international data indicate that our states, districts, and schools are not fully preparing our students for college. Average SAT scores declined across the nation last year, and the U.S. has performed dismally on international comparisons of mathematics, problem solving, and science. And, by 2018, two-thirds of American jobs will require some form of postsecondary education. As a result, we have placed a growing number of our students at risk of graduating from high school – if they graduate at all – ill-prepared for college or a career amid an economic crisis that threatens the future of our nation.

At this time of great consequence, students in every state need a strong accountability system that measures student achievement and holds schools and districts accountable for ensuring that students are prepared for college level work or a good job.

States are in the process of submitting applications to waive certain provisions of ESEA. These waivers can be an opportunity to advance more robust accountability systems that reflect stronger reforms.

However, waivers used in the wrong way can weaken accountability which will only serve to deny students the education they deserve. The evolution of the Department’s waiver guidance as well as a review of the first round of waivers has raised some concerns. In response, the George W. Bush Institute calls on each state to implement an accountability system that reflects the following ten principles.

1. Concrete goals that require schools to prepare an increasing number of students for college level work and reduce the achievement disparity between groups of students.
2. Rigorous content standards that ensure our students graduate from high school ready to do college level work.
3. Performance standards for student achievement that reflect both a basic level as well as an advanced level (representing being on path to postsecondary readiness) of knowledge and skill expected by grade level content standards.
4. Assessments that measure whether students are learning the skills represented by those high standards in:
   1. English Language Arts and math, every year from 3rd– 8th grade, and at the end of key courses in high school; and
   2. Science at least once within each grade span and at the end of key high school courses.
5. These assessments should show whether students are reaching a basic level of achievement as well as whether students’ learning is on, or growing adequately to be on, path to postsecondary readiness upon graduation. The measurements must:
   1. Be valid, reliable, objective and applicable to all students across the state but also must be multiple to reflect the broad range of expectations the standards set for the students.
   2. Be constructed in a fair and technically sound manner to play a significant role in assessing teacher effectiveness in advancing student learning to state goals.
6. Hold schools accountable for how well they are educating all students by having them publicly report their student performance data, disaggregated by subgroup. Accountability decisions for schools must be based on the imperatives of increasing the number of students prepared for college level work and reducing the achievement disparity between subgroups, including English Language Learners (ELL) and students with disabilities. No school should be rated as a high performing school if it does not show gains in the number of on grade level performers and gains in the performance of all subgroups.
7. Parents must have access to this performance data in a way that is transparent and easy-to understand. These data should show parents how well their child is performing in relation to the high standards as well as how other students in the school are performing.
8. Require proven and effective interventions and appropriate consequences for schools that are not making adequate progress to student success as defined by the performance standards. States should have flexibility to target interventions and consequences as they deem appropriate, but consistent with the following principles:
   1. Interventions vary in intensity and type and depending on school performance, with the most intensive interventions occurring in schools that fail to help students reach grade level standards.
   2. Schools should strive to increase regularly the number of students in key subgroups who are on path to postsecondary readiness or growing sufficiently to that goal each year that they will reach it. Success in doing so ought to earn schools higher ratings, honor, and other awards or recognition.
   3. Interventions and consequences are applied consistently across all similarly performing schools.
9. Provide real alternatives and choices for those families whose children are trapped in the lowest performing schools (for example, high-performing traditional public schools, charter schools, online schools, vouchers, and after school tutoring).
10. Detailed plans for continuing to hold schools accountable for advancing student learning each school year and throughout implementation of changes in state assessment and accountability systems, including ESEA waivers, transition to Common Core State Standards and assessments, ESEA reauthorization requirements. States must consider:
    1. The content standards, performance standards, assessments, and performance goals against which schools will be measured each year;
    2. The interventions and consequences the state will employ each year, which schools will receive such interventions and consequences, and how the state will treat schools currently subject to improvement, corrective action, or restructuring under ESEA; and
    3. The choices the state will offer each year to families whose children are trapped in the lowest performing schools.

These principles, taken together, will not only advance state and federal accountability efforts but also ensure that accountability for student success remains at the heart of our educational reform efforts.

And, most importantly, these principles will help states foster a public education system that provides equity, fairness, and high expectations for all students.

Accountability concerns raised by state applications for ESEA waivers

We reviewed the first round of state ESEA waiver applications to find areas of alignment and possible concerns with our ten principles of accountability. Here is what we learned.

Uncertainty and lack of clarity

Many state applications suffer from uncertainty and lack of clarity. For example, some states have failed to provide a deadline for the goals of their accountability system or failed to specify how they will weight the various factors included in their accountability determinations. Other states have not decided which assessment they will adopt or have neglected to establish a timeline for adopting one of the state consortia assessments. Furthermore, several states have to enact controversial legislative changes before certain components of their application can be implemented.

This lack of specificity regarding some very significant accountability issues is worrisome. Did the states have adequate time to think through each part of their waiver plans and their potential impact on schools and students? In light of overstretched budgets and prior commitments to other state and federal reforms (i.e., Race to the Top, School Improvement Grants), do the states have the capacity to implement the promised reforms?

Student subgroups

Over half of the 11 states participating in this first round of waiver applications proposed to base accountability decisions on the performance of a single “super subgroup” consisting of the lowest-performing 25% of students in each school or a melding of some of the traditional subgroups.

Disaggregation of data for purposes of accountability decisions is essential not only to prevent students from being ignored but also to ensure their academic needs are addressed (GWBI Principle 6). As states consider this approach, we believe that the use of a “super subgroup” must include supporting details that insure accountability for key student groups. State applications have failed to provide data to adequately explain what the super subgroup means for individual students. We are also particularly concerned that certain states that have proposed the super subgroup do not even plan to report achievement data disaggregated by the traditional subgroups. Public reporting of disaggregated data empowers parents and other stakeholders with information on how well a school is serving each subgroup of students. Thus, ignoring or overlooking key subgroups (e.g., English Language Learners and students with disabilities) is dangerous and would represent a significant step backward in the effort to hold schools accountable for the achievement of these traditionally underperforming and long ignored subgroups.

Limiting consequences to a certain small percentage of schools

Six of 11 states propose to limit consequences for failing to make adequate progress toward student success to a certain small percentage of schools (i.e., the lowest performing 5% or 15%).

We agree that states must vary the type and intensity of interventions based on the nature and extent of a school’s weaknesses (GWBI Principle 8). We understand that the US Department of Education guidance suggested a 5% threshold for focusing on the worst performing schools. But, this percentage strikes us as an arbitrary cutpoint that, if implemented, would allow thousands of schools to avoid responsibility for educating their neediest students. In accepting this threshold, states have failed to provide a data-based justification for the percentage point cutoffs. Do these states believe that the students in a school at the 5% level should receive intensive state interventions and support while the students in a school at the 6% level are allowed to languish? In other words, is a school at the 6% level that different from a school at the 5% level? Does the state have the capacity to intervene in only a certain percentage of schools? The six applications do not answer these important questions.

In contrast, several other states plan to apply consequences consistently across all similarly performing schools. These states have decided that all schools that do not reach a certain bar or earn a certain grade under the state’s grading system (like an “A-F system”) are failing to make adequate progress toward student success and should be held accountable for this failure. Therefore, the state will intervene in all such failing schools, regardless of the number. This approach seems to hold promise for addressing school improvement needs and also providing focus for state and local educators.

The challenge of improving chronically underperforming schools is a tough one. We’ve both struggled with questions about how to do this work well, and we have many colleagues across the country who are thinking about how best to address this challenge. As we continue to work to promote accountability, we plan to pursue the development of recommendations and solutions that would help states make these changes in thoughtful ways.

Backing away from alternatives and choices for families

Several states propose to eliminate the alternatives they are required to offer to students in failing schools under NCLB; other states have left the decision about alternatives up to their districts or simply failed to address the issue anywhere in their applications.

Improving a really bad school takes time. We see no reason why a student in that school should have to wait for improvement to occur and, in the process, lose valuable learning time (and possibly his or her chance at graduating ready for college or a good job) (GWBI Principle 9). While states and districts should choose which choice model works best for them, eliminating choices and alternatives altogether is not in the best interest for students.

A number of states agree. Some states plan to continue offering alternatives to students in the lowest performing schools through basic school choice plans, after school tutoring, or opportunity scholarship programs available to low-income families. Other states have passed laws (or are working to pass laws) that would increase the number of effective charter schools.

Maintaining accountability during transitions

All 11 state applications discuss the transitions to new content standards, assessments, performance standards, goals, interventions and consequences, and improvement strategies. However, nearly all the states have failed to detail their plans for continuing to hold schools accountable for advancing student learning starting this school year, through next school year, and throughout these transitions (GWBI Principle 10).

We cannot “press pause” on accountability. Implementation of new standards and assessments is not an opportunity to step back from accountability for several years; too many of our students would fall through the cracks during those years. States need to think through – in detail – how the transitions will play out in individual schools. For example, which standards, assessments, and curricula will be used each year? Against which goals will schools be judged each year? How will the state make determinations about poorly performing schools in the interim? What happens to schools that are currently in corrective action under NCLB?

This issue in particular we understand is particularly complicated and raises a number of questions. Our own experience suggests that transitioning from any one aspect of the accountability system to another requires planning and foresight. As we continue to work to promote accountability, we plan to pursue the development of recommendations and solutions that would help states make these changes in thoughtful ways.

Accountability resources available from the George W. Bush Institute

GWBI has committed to help bring about a new era of school accountability, and we would like to be a resource for similarly committed states and districts. Over the next several months, GWBI plans to make the following resources available.

• “Accountability Watch” website. This website will inform the public debate by providing an ongoing analysis of federal and state policy decisions and implementation – including approved and unapproved state ESEA waiver applications – against GWBI's ten principles of accountability. Users will be able to: retrieve state-by-state accountability overviews; understand how these policy decisions are impacting students; and explore real examples of states, schools, and districts that are working to improve student performance and close the achievement gap.
• Assistance for states. GWBI will, upon request, advise states on development of their ESEA waiver applications, state accountability systems, and/or review draft applications for compliance with our ten principles of accountability.
• Research and policy papers. GWBI will help states and districts tackle some of the newest, toughest accountability issues (including effective school choice models and the transition to new standards and assessments) through researching, convening groups of education experts committed to promoting school accountability, and drafting policy papers.