FSA Cut Scores, a Call to Action and the Ingredients in Accountabaloney

On January 6, 2016, at 9AM, Florida’s Board of Education will meet to (finally) decide the cut scores for last spring’s new Florida State Assessment (FSA). This comes after a months long process outlined here. The new cut scores will determine what scores will be necessary for each achievement level, 1-5, on the grade level math and reading FSA exams and the three new math End of Course exams. These cut scores will have lasting impact on school grades, teachers’ pay and student promotion/retention/remediation decisions. (You can watch the meeting live on The Florida Channel and see the agenda here).

Despite public outcry to the contrary, there has been tremendous political pressure from Jeb Bush’s Foundation for Excellence in Education (FEE) to raise the cut scores to match the proficiency levels on the National Assessment of Educational Progress (NAEP), as reflected in their near constant social media campaign, their engagement of the Florida Chamber and Council of 100, and their alignment with Vice Chair of the BOE, John Padget (who famously recommended we submit our children to a cold shower now in anticipation of the hope for a good paying job later).

We want to make it clear that our students and our schools will experience more than just a cold shower with the raising of FSA cut scores to levels beyond grade level proficiency. Florida’s entire Accountabaloney system rests on these proficiency levels… students who do not pass the new third grade English Language Arts (ELA) achievement levels will face retention, high school students not achieving the new levels will be denied their diploma, many, many more students will face remediation, schools will respond with more test preparation to protect their school grade, and much more.

It is essential that the Board of Education contemplate ALL the REAL repercussions of the current Accountability system before raising the cut scores. They should ask for a PAUSE in all accountability policy during the transition to these new scores (as suggested in the FEE panel discussion and outlined here). If the Board votes to raise the cut scores without pausing the repercussions many more students, teachers and schools will be subject to the ramifications of the accountabaloney.

Because we feel the Board needs true understanding of the accountabaloney in Florida’s current system, we are republishing our earlier blog, “A Long and Rather Scary List of the Ingredients in Florida’s Accountabaloney System,” in its entirety, below. We encourage you to share this with our Florida BOE members along with the insistence that they consider the full ramifications of their actions before they set cut scores on January 6th. (You can find the FLBOE contact information here.)

In addition to contacting the Board, please take a moment to voice your concerns with the DOE. The Florida Department of Education is looking for comments on cut scores and school grades. Currently, the only comments posted are those that appear to have been solicited by the Foundation for Education Excellence (evidence here). The DOE needs to hear from YOU! Go the the website and click on “Submit Comment”. 6A-1.09422 for cut scores.

Need advice on what to write? Consider this:

“Our accountability system is based on the premise that test scores reflect grade level proficiency. This fall, months were spent on the FSA cut score process, including educator recommendations, a reactor panel and community input sessions. To throw all that work out the window and insist on raising the bar to NAEP proficiency levels, which are NOT based on grade level proficiency is ill advised, especially given the negative ramifications that “below proficient” test scores have on Florida’s students, teachers and schools. Set the passing levels at grade level proficiency.”

A Long and Rather Scary List of the Ingredients in Florida’s Accountabaloney System

ingredients

SPOILER ALERT: You will not find a lot of reason or common sense.

Parents and taxpayers agree there should be accountability in our school system, but the current test-and-punish accountability system is Accountabaloney; policies put in place are destroying, not evaluating, our schools.

Here, in list form, are the basic ingredients we have discovered in #accountabaloney. We plan to expand on individual accountabaloney ingredients in upcoming blogs.

The primary stated purpose of Florida’s student assessment program is “to provide student academic achievement and learning gains data to students, parents, teachers, school administrators and school district staff.” How well do they achieve that goal?

  • Much of the school year revolves around preparing for state assessments that will only be validated after the students have taken them, if ever. Learn more here.
  • During the Summer of 2015, Alpine Testing Solutions was hired to provide an independent validity assessment of the new FSA tests before the scores could be used. The report only evaluated 5 of 14 grade level tests (grades 3, 6 and 10 ELA and grades 4 and 7 Math) and one of three new math EOCs (algebra 1). This leaves 9 of 14 grade level assessments and two new math EOC unvalidated.  Read the Alpine Report here.
  • Ordinarily, FCAT/FSA results arrive at or after the end of the school year. There is no feedback to students as to what they missed, no time for teachers to correct deficiencies in the school year. These tests DO NOT inform instruction.
  • FCAT/FSA is NOT nationally normed, so results provide parents little information on how their child might perform at a national level. Also, the state test primarily assesses grade level proficiency so performance on the FSA/FCAT not helpful in assessing student performing significantly above or below grade level.
  • Achievement levels change annually, making it nearly impossible for the average parent to understand year-to-year variations in their student’s scores.
  • Achievement levels on the FCAT/FSA are determined by a politically motivated “cut score” process and may have little relationship to developmentally appropriate achievement levels at each grade level. For example, the score required to pass the 3rd grade Reading (ELA) FSA was not determined by assessing what performance would reflect whether a child was reading at grade level prior to administering the exam, but rather by evaluating how students performed on the exam after it was administered.
  • The “cut score” process is politicized to the extent that, for the last week, The Foundation for Florida’s Future, a highly influential, non-profit education reform advocacy group which promotes the expansion of vouchers and the privatization of public schools, has been nearly constantly advertising on social media, encouraging citizens to contact the Florida DOE and influence the cut score process. Specifically, the Foundation wants the cut scores raised, increasing the required passing scores which will result in dramatic increases in the number of children who will fail the state tests. Conveniently, high failure rates for students on state testing will lead to even worse school grades, further supporting the Foundation’s assertion that schools are failing and, therefore, need to be privatized.  Read about it here.
  • The state assessment is a criterion-based test, designed to test how well the specific state standards were taught/learned in each grade. Since the standards vary by grade level, one should question how performance in sequential years actually measures learning gains.

The A-F grading system

  • Central to Florida’s Education Reform agenda was the institution of the A-F School Grade program. Presumably, school grades are calculated so parents and taxpayers can learn how their schools are “doing”. Property values are highly influenced by the local school’s grade. Do these grades reflect what actually constitutes a quality education?
  • By and large, parents do not choose their child’s school based on the “learning gains of the lowest quartile.” According to the August 2015 Phi Delta Kappa International poll, the factors most important to a parent’s school choice include quality of teaching staff, curriculum, student discipline, class size, variety of extra curricular activities, school reputations, proximity to home and school size. None of these things are reflected in Florida’s A-F grades yet, according to decision makers, are all more important to parents than student test scores.   Read the PDKI poll here.
  • Florida’s current school grade calculation is based almost exclusively on test scores in grade level FSA Math and Language Arts and, 5th and 8th grade FCAT Science. Learning gains on Math and ELA FSA are also calculated and represent half of the school’s grade. High Schools score points for graduation rates and student performance of college readiness exams.
  • Because 2015 was the first administration of the FSA, learning gains cannot be calculated this year, leaving half of the school grading calculation without data. Despite calls from Florida’s Superintendents to hold school grades, the DOE is “moving forward” towards issuing school grades based on only half of the grading formula.
  • Extra points are given to Middle Schools who place more students in Advanced Math classes (Algebra or Geometry) and High Schools who place more students in AP, IB, AICE, Industry Certifications and Dual Enrollment courses. Middle Schools and High Schools can also score points if students pass advance course exams, like Math EOCs or AP tests, respectively. This has lead to more students being placed in advanced classes, whether they are prepared or not, sometimes against their will and even against parents’ wishes. In Monroe County, school administrators are paid bonuses to place students in certain AP classes and teachers and students are paid bonuses when students pass the class. In Orange County, middle school students in Honors Math classes found themselves also enrolled in Remedial Math, to allow extra test preparation for the EOC.  Learn about School Grade calculations here.
  • Test scores reflect little more than socioeconomic status. Researchers have been able to predict school grades based on US census data alone. Chris Tienken and colleagues “predicted the percentage of students at the district and school levels who score proficient or above on their state’s mandated standardized tests, without using any school-specific information” making the need for testing obsolete!  For more information, click here.

Costs of Accountabaloney

  • State Assessment data is to be used by the public to assess the cost benefit of the expenditure of taxpayer dollars.
  • Recent estimates by teachers, suggest that at least 25% of the school year is spent in test preparation and test administration.
  • Parents question whether test prep, which pushes out more creative and authentic learning experiences, represents a “free and appropriate education.”
  • The costs of testing and test preparation have increased exponentially yet have demonstrated little positive impact on measures of nationally respected academic achievement (SAT and ACT scores have barely budged during the last 15 years of reform and remain below the national average).  For more information on the decrease of SAT scores, click here.  For more information on ACT scores, click here
  • Test developers, like AIR, have been paid massive amounts of money to develop tests without documented validity or reliability. The state’s contract with AIR required validity testing to be done by the test developer. Commissioner Stewart reported, in March 2015, that the FSA was field tested in Utah and its validity confirmed at that time. Copies of the Utah Field test/validity report have been requested by the Florida Senate but, to date, are not available.
  • Follow the Money: The cost of this year’s FSA test is about $40 per student every time they take it. There is a financial incentive for test developers to fail students requiring retakes. The same companies that make the tests help set the pass/fail rate, create the test prep and remediation programs and publish the curriculum.
  • Large amounts of taxpayer dollars are spent on the technology required for administration of computer based testing. Computer centers created to be used for state mandated testing are, for significant portions of the year, inaccessible to students, including those enrolled in computer science courses. Taxpayers should question whether funding computer labs for the purpose of testing and test preparation, rather than computer education, is an appropriate use of tax dollars.
  • The indirect costs of test focused education are enormous and should also be considered when calculating the appropriate use of taxpayer dollars: lost instructional time, narrowed curriculum, misuse of resources, time spent in test prep, etc. etc.
  • Quality Preschool and Testing Preschoolers: Play based preschools  have lasting benefits when compared to academic preschools, yet the state requires frequent academic assessments in VPK and requires administration of FLCKRS/Kindergarten readiness, which has the primary purpose of rating VPK programs, and is more likely to reward academic preschools.  For more information on the psychological harm done to our youngest learners, please click here.  For more information on the decline of play, please click here.

Testing our youngest learners:

  • Current Florida standards require children learn to read in kindergarten, even though this has been shown to be developmentally inappropriate for many kindergarten aged students and can cause lasting harm for those students. For more info, click here.
  • In order to provide more time for test preparation, many schools across the state have eliminated or curtailed recess, music and art, all known to be critical to early childhood development. For more Info, click here.
  • HB7069 Removed the requirement that each elementary school regularly assess the reading ability of each K‐3 student but still requires districts “to establish a student progression plan that emphasizes reading proficiency in K‐3 and progression based on mastery of the standards “. Though there is no requirement that determination of reading proficiency must be by standardized assessment, when combined with the state’s requirement to evaluate teacher’s based on objective student data, this often results in the administration of multiple standardized assessments throughout the K-2 school years, often computer based, even though such standardized tests in children so young have been demonstrated to be un-valid, unreliable and developmentally inappropriate.  For more info, click here.

3rd grade retention

  • Florida mandates passing the 3rd grade state reading assessment before promotion to 4th grade, even though the preponderance of the evidence show that grade level retention (at any grade level) causes lasting harm to children, decreasing their likelihood of graduating high school by 60% each time they are retained. In Florida, mandatory 3rd grade retention has lead to an increased number of student retention at grade levels leading up to and including 3rd grade. Low income children and children of color are disproportionately affected by these retention practices. Education research provides multiple examples of effective interventions for struggling readers who do not require retention to be successful.  For more information on the impact of retention, click here.
  • Florida loves to celebrate their nation leading 4th grade NAEP scores, which should be considered a direct reflection of the high number of 3rd grade retentions in the state. When you retain the worst readers in 3rd grade, you should expect reading scores in 4th grade to rise, as been demonstrated in other states when similar mandatory 3rd grade retention policies have been enacted.

Intensive Reading:

  • Students with low ELA test scores are placed in Intensive reading interventions, which in many if not most situations involves assigning students to a state approved computer reading program, often remaining on the same or similar programs for years in a row whether the program is helping or not. Districts appear to place students in these classes to provide additional test prep to students they fear will perform poorly on the state assessment. Read about IR here.
  • Across the State, Honors students and those in Advanced Placement literature classes, are simultaneously placed in Intensive Reading Classes. One should question how having “Intensive Reading” on a college bound student’s transcript affects their admissions chances at competitive universities.  Watch here.

Graduation requirements

  • Passing the Algebra 1 End Of Course exam is required for high school graduation even if a student passed his Algebra 1 class and went on to take and pass advanced math classes. A student who passed high school Calculus still requires a passing score on the Algebra 1 EOC to graduate.
  • Passing the 10th grade reading FSA/FCAT is required for high school graduation even if a child passed Advance Placement Literature exams.

State Mandated End Of Course Exams

  • The state requires participation in state-developed End of Course exams in Civics, US History, Biology, Algebra 1, Geometry and Algebra 2. In each case, the test is mandated to be worth 30% of the student’s course grade, and may have a significant impact of a students resulting GPA. Teachers are not allowed to view the tests questions and students and parents cannot review the student’s performance following the test.
  • While all students must take these assessments, districts have tremendous leeway in the interpretation of the leveled test scores (i.e. 1 through 5) and their application to the final course grade. While all districts must calculate the score as 30% of the final grade, the numerical score assigned to individual performance level varies by county… in some districts a level “3” might be worth a B+, in other districts the same level three might be worth a low B or C. Given that the mandated 30% weight can significantly affect a students final grade, students with similar performance in their class and on the EOC may have significantly different course grades due to their geographic location. A district with a more “punitive” calculation of EOC scores could significantly impact a child’s high school GPA, when compared to a child from a neighboring district with a more realistic calculation of EOC scores.

Teacher Pay

  • In 2011, in order to qualify Race to the Top Federal Funding, Governor Scott signed SB736 which required tying teacher pay to test scores., even though studies show that more than 80% of the variation in a student’s tests score is NOT dependent on teacher performance, but rather on circumstances outside the teacher and the school control.  more here.
  • The need for objective test scores for teacher evaluation has led to routine progress monitoring of all students in many districts, even though the state only requires such monitoring for certain struggling students.

Accountability is not uniform across different types of schools: The rules and requirements of testing and accountability vary between types of schools: Traditional Public, Magnets, Charters, Vouchers, Private, Virtual, Duel Enrollments and homeschool, all have varying degrees of test score based accountability.

  • State funded voucher students are required to take some sort of state approved annual assessment, yet need not take the FSA/FCAT/EOC to receive credit for courses, be promoted or graduate.
  • Public school students must pass the Algebra 1 EOC for graduation but, publicly funded voucher students can graduate without fulfilling this requirement.

Impact of state testing on Special Needs Students

  • The developmental age of a child is largely now ignored, even with the support from an Individualized Educational Plan (IEP) in place.  The focus, even for the most disabled children, is on grade level proficiency.  One size does not fit all and this is especially true of special needs children. These children encounter a spectrum of difficulties; children with physical disabilities, like spina bifida or cerebral palsy,  require a very different set of accommodations from an autistic child, for example, yet they are all held to the same standard.
  • Accommodations are not the almighty fix.  IEPs are supposed to be individualized and should serve to put in place whatever supports are necessary for that child to succeed.  Success should be measured very differently from child to child.  You cannot even compare a high functioning autistic child with a child that is lower on the autistic spectrum. The reality is that due to budget cuts, may services/accommodations are not provided or limited. The notion that accommodations alone can solve the issue of a child meeting grade level proficiency is ridiculous.
  • Recent policies from the USDOE defy all common sense and reason, suggesting that all special needs children should be held to the same grade level proficiency standards as their able bodied/minded peers and that the new tests are valid for ALL students with ALL learning needs. Following these policies, only 6% of New York’s special needs children passed their state’s assessment last year. Florida should expect similar results for our most vulnerable children.
    For more information on the current policy implemented for SPED students and the enormous impact on this population, please click here. Or here.
  • Florida law requires families of significantly impaired or terminally ill children to petition the state for a waiver from these tests, which may or may not be granted.  Nobody is exempt from state testing because, as Jeb Bush said, “I think if you don’t measure, you don’t really care.”  Bush goes on to say, “I think you have to assess where students are.”  I guess ignoring a child’s developmental level/age is not part of his equation.  He expects that all children must meet grade level proficiency against all the odds (as long as you have accommodations!)

Congratulations. You have reached the end of our long and scary ingredient list.  If you encounter more #accountabaloney ingredients, and we are certain there are more to be discovered, please share with us in the comments and we will add them to our list.

Accountabaloney cannot be cured by changing from one standardized test to another. We must change the way the test scores are used to create a high quality and just education system. We will be exploring alternatives to test-and-punish accountabaloney in upcoming posts. The first step towards the cure for #accountabaloney is recognizing the extent of the problem.

 

FEE Panel Agrees: Raising Proficiency Levels Demands Accountabaloney Pause

On January 4, 2016, Florida’s State Board of Education will be setting the cut-scores/determining proficiency levels for the Florida Standards Assessment (FSA) that was administered in early spring 2015. The commissioner of Education has reviewed the cut-score levels determined by a panel of education experts, reviewed by a community “reactor panel” and presented to the public. At the last Board of Education meeting, she presented her recommendations, which raised the proficiency bar higher than the educators recommended but apparently not high enough to please several members of the board. On 12/4/2015, the BOE will meet for the last time before the cut score process is complete. You can find information regarding that meeting here.  One would expect the conversation regarding proficiency levels will come up. Will the Board be convinced to agree with the commissioner’s recommendations or will they push for higher achievement levels, resulting in lower passing rates for Florida’s students?

The question “Why Proficiency Matters” is currently a hot topic in reform circles. In this blog, we will attempt to get into the mind of a reformer and see the issue from their perspective. The best place to start is the Foundation for Excellence in Education’s (FEE’s) annual meeting.

On October 22, 2015, education reformers from around the nation converged on Denver, Colorado, for the Foundation for Excellence in Education’s 8th annual Education Summit. Strategy Session VII was focused on “The Proficiency Gap: Why It Matters for Your State.” This session was particularly timely for Florida because, less than a week later, its Board of Education members would be meeting to discuss determining the proficiency levels (or “cut scores”) for the new, 2015 Florida Standards Assessment (FSA). (You can watch Session VII here)

Members of the Florida Board of Education were present during this strategy session so I thought it might be interesting to try to view the session from their perspective, to try to understand what lessons they might have learned and how those lessons might influence the upcoming FSA cut score process. To do that, I am going to ask everyone to bear with me and, for the time being, put on a reformer hat. In doing so, I need you to accept a few basic assumptions. These are a few of the things that essentially all reformers, and most of the audience for this session, believe:

  1. The current standards (Common Core State Standards, Florida Standards, College and Career Ready standards, whatever you choose to call them) are developmentally appropriate, well constructed, internationally benchmarked (not just informed by international benchmarks), rigorous standards that will lead to better academic achievement by all students.
  2. The current state assessments are a high quality, effective and efficient way of determining whether students are on the path toward Career and College Readiness.
  3. The current assessments have been psychometrically evaluated and are fair, valid and reliable for all types of students, including Special Education and English Language Learners.
  4. Raising proficiency cut scores on a state assessment will lead to higher levels of academic performance.
  5. Reformers are on the right path and should not be distracted by naysayers. What reformers are doing is working and we can’t “go backward.”

The debate regarding all of the above will be left alone for the time being (I know it will be difficult; try your best).

Here is a recap of the highlights of “The Proficiency Gap: Why It Matters for Your State” from the 2015 FEE Education Summit:

Introductions were made by the moderator Dr. Christy Hovanetz, Senior Policy Fellow of Accountability, Foundation for Excellence in Education. She explained why she felt now was the right time to have the conversation about proficiency levels (because, having spent so much money on new standards, new tests, new curriculum and professional development, “it would all be for naught” if we didn’t raise the standards). She claimed the current “proficiency gap” (the difference between proficiency levels on state tests and proficiency level on “the Nation’s Report Card” or NAEP assessment) was a result of states not being honest with their students. The proficiency gap, she claimed, was really an “honesty gap”, a result of states, essentially, cheating the system, and it needed to be corrected.

The first panelist was Karen Nussle, Executive Director of the Collaborative for Student Success.  She explained what the proficiency gap was and blamed it on politicians who wanted their state’s scores to “look good.” She explained “there were politicians in the mix, it was hard to be honest.” Ms. Nussle explained “The Collaborative” is a PR firm that could help share information about the honesty gap and she shared a very clever commercial they had produced to influence the FSA cut-score process in Florida, showing darling children saying things like “When I grow up I want a dead end, low wage job.”  Really, the kids are cute and the commercial is funny (I’m pretty sure I heard the distinct chuckle of one of Florida’s BOE members in the audience) and “The Collaborative” can help your state get similar commercials to lobby your Governor to influence the setting of cut scores beyond educators’ recommendations, because “parents deserve the truth.”

The second panelist was Alabama State Superintendent of Education, Dr. Tommy Bice.  Under Dr. Bice’s leadership, Alabama adopted standards 4 years ago, chose to assess students with the ACT Aspire and have raised their proficiency bar to eliminate the honesty gap.  He was present as an example of how the process could be done successfully.  He had lots of interesting things to say, so pay close attention.  During this transition  process he had spent a good deal of time developing relationships with Alabama’s 2 and 4 year colleges and with its business leaders.  He frequently asked them “What skill set is missing in Alabama high school graduates when you get them.”  The overwhelming response was kids graduating from Alabama high schools “lack intellectual curiosity.”  He was not surprised by this because “We have been preparing kids to take a meaningless test rather than to think, solve problems, think analytically and collaborate with peers.”

Dr Bice spent a good deal of time explaining why they chose the ACT Aspire, how they communicated the new expectations to parents and what preliminary growth data looked like.  He, also, shared what he felt was the key to Alabama’s success :

The biggest challenge… is providing this policy safe space in these first few years to not misuse assessment results.  We misused them for decades under No Child Left Behind, to rank and divide and conquer the haves and have nots, we need to use them and we need to use them instructionally to make sure we are improving the practice for our teachers, and improving the learning for our children but we want to be very careful and very protective that we don’t use them for policy agendas that don’t lend itselves (sic) to those things, at least in the first few years and then when we see what this really means to us, we can begin to guide some of the policy decisions that need to be made.”

He repeated that it was important, in Alabama, that they not “misuse these scores in the first few years until we can really see what that growth outcome needs to look like.” During a brief question period, Ms. Nussle pointed out that not everyone will be happy with the new, “more honest scores” but “we are really changing the system here and there is a transition period to that here and there needs to be public understanding of what that involves.” Dr. Bice agreed:

“That’s the most important message… it would be real easy to fall back into how we did it under No Child Left Behind, to put people on lists and rank and divide. We need to use this to really look at what we can do instructionally to move  children in a more accelerated rate to be college or career ready and you don’t do that in a policy environment that uses assessment scores in other ways.”

Next up was Scott Sargrad, Director for Standards and Accountability, Center for American Progress. He spoke about content standards and proficiency standards and explained how, back in 2009, “there was widespread agreement that proficiency standards, and content standards, were too low.” Since then, there has been a lot of work done to increase standards and proficiency expectations.  He explained how raising the proficiency bar can expose previously hidden achievement gaps, allowing more resources and supports to go to the schools that need them. He explained how teachers’ expectations have a significant impact on students. He, also, recognized “serious buy in” has to happen to prevent pushback when raising the proficiency bar. Then (at 31:40) he said:

One of the things that’s really important is… being patient with the scores and not trying to use them too soon.  So, when I was at the DOE, we offered states this flexibility to take a pause on using some of these new results for accountability purposes for schools, for teachers, while they are getting their feet under them. Now not every state took us up on that… but there is also value in taking a little break from the rating and the ranking of the schools and saying we’re going to keep making during the kids are going to get the supports and interventions they need but we’re going to see how these tests play out, we’re going to see how the growth plays out over 2 years, we’re going to see what that means for teachers, see what that means for school.”

Mr. Sargard concluded by noting that we will know whether this works if we see smaller gaps with NAEP (he was hoping to see this in the 2015 NAEP scores), lower remediation rates in college, higher college graduation rates and students getting good jobs after college… then we will know if we have been successful.

Cornelia Orr, former Executive Director of the National Assessment (NAEP) Governing Board, was the final speaker. She shared NAEP research, explained NAEP mapping to state assessments (which is not the same thing as matching percentages) and gave this advice:

“I think you should set your proficiency cut score at a level that will be challenging for students to reach…  My advice is that you use the proficiency level to communicate about student learning and that you use your accountability system to monitor your system and, if changes need to be made there, you do that in the accountability system, or the way that you use those scores, but you set your proficiency mark as high as you can because that is what communicates to students.”

In response to an audience question regarding fairness of these proficiency levels for older students who have not been exposed to Common Core during their entire school careers (from my own Monroe County School Board member, Captain Ed Davidson), Dr Bice reiterated:

What is important “in these first few years of implementation, rather than adjusting scores, is be careful on how we use them, is not use them especially at the upper grade levels as a high stakes, get out of high school sort of assessment, which I would adamantly oppose at this point and we don’t use those in that way in Alabama… we strictly use those to make instructional decisions, to make sure that we’re providing a curriculum and instruction that get kids more college and career ready. We don’t use them in any sort of punitive or any other way for students.”

The final audience question was from Florida BOE Vice Chair, John Padget. He felt this was a really great panel and he reiterated the need for honesty (possibly his question is a plant from the Foundation?). He claimed he has repeatedly seen students who have finished high school and are told when they show up at college “you are not college ready.” He wanted to know what can we do to be honest at an earlier point so students don’t face disappointment at that time. (Going to take off my reformer hat here for a moment to say: personally, I would suggest attending parent-teacher conferences and meeting with the high school counsellor, to get a good idea as to whether your child is on track for success in college… putting the hat back on now). Dr. Bice replied with  some comments about how community colleges actually make a lot of money providing remedial classes and there might be a financial incentive for them to recommend remediation.

Karen Nussle wanted to end on a positive note. According to her, the testing products are better than ever.  Millions of kids have taken the PARCC tests and we can compare data across states.  The tests are great and can only get better  Parents are going to get more information about their kids than ever before. The amount of data collected is fantastic. This is working and we have to keep at it. We need to stop allowing others to distract us from where we are going because this is working.

Lots of applause. Dinner will be served at 6.

Remember, you are still wearing your reformer hat.  As Mr. Padget said there was lots of great info here.  As Ms. Nussle said, we should focus on it and not allow ourselves to be distracted. In the end, there was little debate. All 5 experts seemed to agree on the issues and the main points of the session can be summed up:

  1. Raising the achievement bar to NAEP levels is something we all want.
  2. The key to Alabama’s success  was providing this policy safe space in these first few years to not misuse assessment results.
  3. Test scores should be used to make instructional decisions, ensuring students are provided a curriculum and instruction that will get them more college and career ready.
  4. Guard against using test scores for policy agendas that don’t lend themselves instructional improvements.
  5.  One of the things that’s really important is being patient with the scores and do not try to use them too soon.
  6. Use the proficiency level to communicate about student learning and use your accountability system to monitor your system and, if changes need to be made there, you do that in the accountability system, or the way that you use those scores.
  7. Do not use them, especially at the upper grade levels, as a high stakes, get out of high school sort of assessment.

So that summarizes the Foundation for Excellence’s panel discussion on “Why Proficiency Matters”.  Everyone was pretty much in agreement and Alabama was held out as the shining example of “how to do it right.”

You can take off your reformer hat now.

Fast forward a few days to the October 28, 2015, Florida Board of Education meeting in Orlando (you can watch public comments here). During public comments, Patricia Levesque, the extremely influential C.E.O. of the Foundation for Excellence in Education, advised the board to do two things:

  1. Proceed with the issuing of school grades based on the new FSA. Do NOT listen to the School Superintendents who are asking for a pause.
  2. Do “the right thing” and set “an honest cut score for children.”

So, Ms. Levesque suggested that Florida set a cut score aligned with NAEP scores (an “honest” one) AND issue school grades, in direct conflict with what essentially every Session VII panelist recommended. Why didn’t she listen to her own experts and recommend a pause in the use of test scores for anything except improving instruction during the transition period? Did she miss the strategy session just a few days earlier or does she disagree with its conclusions?

We recommend that Florida use the initial few years of the FSA administration as recommended by the FEE panelists and institute a policy free zone.  This may take the doing of the legislature and/or the Governor but, as Dr. Bice strongly recommended, it is necessary part of the process. A complete re-evaluation of the accountability system should be done.  Listen to the experts and use the FSA scores only to improve instruction. Eliminate the high stakes. With raised proficiency levels the cut scores will no longer reflect expected grade level performance. Do legislators intend to retain and remediate third graders who are reading at or above grade level? Should these tests be used as a high stakes, get out of high school sort of assessment? Does our Board and Department of Education really want our schools and teachers to continue to focus almost entirely on test scores for fear of punitive repercussions or should they use those test scores, as Dr. Bice strongly suggested, “to make instructional decisions, to make sure that we’re providing a curriculum and instruction that get kids more college and career ready”?

One final question: Why would Ms. Levesque advise the Florida Board of Education to do something that her entire panel of experts would disagree with?

Dr_Bice_LACK_intellectual curiosity

Is the Honesty Gap a Distraction?

On October 22, 2015, education reformers from around the nation converged on Denver, Colorado, for the Foundation for Excellence in Education’s 8th annual Education Summit. Strategy Session VII was focused on “The Proficiency Gap: Why It Matters for Your State.” This session was particularly timely for Florida because, less than a week later, its Board of Education members would be meeting to discuss determining the proficiency levels (or “cut scores”) for the new, 2015 Florida Standards Assessment (FSA). (You can watch Session VII here)

The moderator of the session was Dr. Christy Hovanetz, Senior Policy Fellow of Accountability, Foundation for Excellence in Education. She introduced why she felt it was the right time to have the conversation about proficiency levels (because, having spent so much money on new standards, new tests, new curriculum and professional development, “it would all be for naught” if we didn’t raise the standards). She claimed the current “proficiency gap” (the difference between proficiency levels on state tests and proficiency level on “the Nation’s Report Card” or NAEP assessment) was a result of states not being honest with their students. She suggested that, over the last decade, states had been lowering the proficiency bar, lowering the cut scores required to achieve proficiency on their state tests, in order to “achieve” the Adequate Yearly Progress (AYP) required by No Child Left Behind, which had required “all students be proficient by 2014”. The proficiency gap, she claimed, was really an “honesty gap”, a result of states, essentially, cheating the system, and it needed to be corrected.

Is the proficiency gap a result of manipulation of cut scores on states tests, as Dr. Hovanetz (and the rest of the Session VII panel) suggested? In Florida, at least, it appears not.

No Child Left Behind (NCLB) was signed into law in 2001. What was Florida’s “proficiency gap” in the early years of NCLB? If the panel’s suggestion that states deliberately lowered proficiency levels to achieve AYP is correct, than today’s proficiency gap should be greater than that of a decade ago. Historical data, however, shows this is just not true. The 4th grade “gaps” are essentially identical and while the gap for 8th grade reading is slightly larger (23 vs 18), the gap for 8th grade math is significantly smaller (20 vs 30).


 

2003 FCAT % level 3+ 2003 NAEP % above proficient 2003 proficiency gap 2013 FCAT% level 3+ 2013 NAEP % above proficient 2013 proficiency gap
Reading 4th 55 32 23 60 39 21
Reading 8th 45 27 18 56 33 23
Math 4th 51 31 20 61 41 20
Math 8th 53 23 30 51 31 20

2003 FCAT scores from http://www.fldoe.org/accountability/assessments/k-12-student-assessment/history-of-fls-statewide-assessment/fcat/scores-reports/index.stml

2003 NAEP results from http://nces.ed.gov/nationsreportcard/states/

2013 proficiency gaps from whyproficiencymatters.com/florida


If cheating or dishonesty doesn’t explain the gap, what might?

  • NCLB required all states measure all students for grade level proficiency in Math and Reading. State tests were designed to determine if students were performing at grade level. The NAEP assessment, on the other hand, considers “proficiency” to be “solid academic performance for each grade assessed”. The gap might exist solely because NAEP never defined proficiency as “grade level proficiency” but rather something much more advanced.
  • Florida’s state assessment is a high stakes test where, sadly, entire school years focus on achieving high scores, to avoid retention and remediation, graduate high school, earn teacher bonuses and school grades. The NAEP test is a low (no?) stakes test. It is conceivable that at least part of the “honesty gap” is really an “effort gap” in that our test-weary children will surely “try harder” to score well on a high stakes exam. This alone could explain why more children score proficient on state assessments.

What the “honesty gap” really is a public relations campaign to encourage the raising of the bar to aspirationally high achievement levels. It is not an emergency or a sign of cheating. It is, likely, mostly caused by the fact that during the NCLB era, the goal was grade level performance, which, by definition, means average performance and NAEP always measured proficiency at a higher level. Now, the reformer’s stated goal is to eliminate need for college remediation and make all high school graduates “college and career ready”, a much higher achievement level, and reformers hope that NAEP proficiency levels will be an appropriate measure of that (to date, no one has any proof that current standards will make students more “College or Career Ready” but reformers hope they will).

The truth is, since 2001, NCLB required schools to focus on grade level proficiency and states used these test scores to inform parents as to whether their children were performing at or below grade level. Now, with the focus on “Career and College Readiness,” the state has to decide how to impart that information to parents.  Suggesting that states somehow created the “proficiency gap” by manipulating test data and “lowering expectations”, at least in Florida, appears to be inaccurate. Frankly, if Florida did manipulate cut scores to demonstrate false progress, it would have been done with the full knowledge and consent of the Foundation for Excellence in Education, which heavily influences all education policy in Florida.

So, why are reformers, like Dr. Hovanetz, so focused on this “honesty gap”?  We believe reformers, by focusing on the contrived “honesty gap”, have created a false narrative that they are using to further their reform agenda, encouraging the raising of the bar on state assessments regardless of the potential impacts on students and schools. The honesty gap rhetoric creates an emergency situation requiring immediate action. In Florida, it appears the Board of Education will spend significantly more time discussing cut score placement for the FSA than they did evaluating whether the assessment was even valid, fair or reliable.  Nationally, the honesty gap narrative distracts from the obvious failures of the education reform movement: scores are declining on the ACT and SAT, respected college entrance exams, as well as scores on NAEP, itself. After more than a decade of test-focused education reforms, students are graduating from high school “lacking intellectual curiosity”. Dr. Tommy Bice, Alabama State Superintendent, summed it up during the same proficiency panel discussion: “We have been preparing kids to take a meaningless test rather than to think, solve problems, think analytically or collaborate with peers.”

If reformers truly want to improve public education in America, we suggest they spend more time honestly evaluating the apparent shortcomings of their reform agenda and less time accusing others of dishonesty. We will be further addressing “Why Proficiency Matters” (or doesn’t) in upcoming blogs.  Stay tuned…