SB1360: Questions for Senator Gaetz

Somewhere in Florida, Spring 2017:

Anne is a student at school “A”, a perpetually “A” rated school in a relatively wealthy neighborhood in a high performing district.  Her district chose the new SAT for their annual assessment and she scored a respectable 1270, including a 670 in Critical Reading and Writing  and a 600 in Math.  With those scores, she satisfies both the 10th grade ELA (formerly the 10 grade reading FCAT) and the Algebra 1 EOC graduation requirements AND she is exempt from taking the state mandated End of Course exams (EOCs) in Algebra 1, Geometry, Algebra 2, U.S. History and Biology 1.  When she takes those classes, her course grades are determined by her teacher based on her class performance during the school year. She is delighted with the reduced number of state mandated tests she is required to take because it will allow her to focus on her Advanced Placement exams.

Carol attends school “C”, in a less affluent neighborhood in the same district.  She scores a 1170 on her SAT (770 Critical Reading and Math, 400 Math).  Her near perfect 770 in Reading satisfies the State’s 10th grade ELA graduation requirement but her Math and combined scores mean that she will be required to take the state mandated EOCs in Algebra 1, Geometry, Algebra 2, U.S. History and Biology 1.  For each course, her performance on the state mandated EOC will be worth 30% of her course grade. For Carol, little has changed in the amount of testing and she is envious of the students in her class who are not required to take the state EOCs. She will have little time to focus on the AP literature classes that she loves.

Frank is a student in the same district. He has always struggles with math but he is an amazing artist and a creative writer. He dreams of a career as an illustrator or graphic artist. Because of his school’s concerns regarding his ability to pass the Algebra 1 EOC, he is placed in an Oracle Database certification course, instead of his desired Art electives, with the hopes that he can meet the high school graduation requirements that way.

Does this sound like a fair and appropriate system to you? These scenarios are not far fetched, they are based on the details in the proposed legislation in Florida’s SB1360, commonly known as the Seminole Solution.

This, in essence, is the accountability system that  SB 1360, as currently written, describes: It allows students who obtain high enough math and reading scores on a nationally normed assessment, to be exempt from U.S. History and Biology state mandated End of Course exams (EOCs). It allows students who complete FAA Ground School or obtain an Oracle Database certification to be exempt from the high school graduation requirement to pass the Algebra 1 EOC. Fourteen pages of SB1360 (between pages 7 and 21) are devoted to describing the many options for exemptions from state required assessments.

SB1360 is NOT a simple substitution of a nationally normed assessment for the flawed Florida State Assessment (FSA).

SB1360 is, also, NOT a bill that allows parental choice in assessments. It’s the districts choice.  Parents are ONLY allowed to choose if their district chooses an alternate assessment and the parents prefer the FSA. In districts that choose the FSA, parents are given NO choice.

According to Senator Gaetz, he believes this bill will save “the infrastructure of accountability” in Florida’s education system. We are more skeptical.

We appreciate the attempts to minimize duplicative testing and to shorten the “testing season” with a pencil/paper option. We would have understood if, after the disastrous rollout of the FSA, a decision had been made to move to a different testing system (like, perhaps, the ACT system which provides grade level testing through college readiness exams). We are not asking for the complete elimination of testing (which is, of course, required by federal law) but would like the state to move towards a system involving appropriate use of test data, implemented fairly for all students and schools. In that regard, SB1360 is a giant step in the wrong direction.

SB1360 passed through the Senate Education PreK-12 committee (watch it here) with barely a discussion, other than to declare how great it was. In his close on the bill (at 1:23), Gaetz states “if there are things about the bill that are a bit rough, that blame belongs with me”, adding that, if the bill moved forward, they would keep working on it to make it better.

We hope Senator Gaetz was being genuine and is planning to make this bill better. As always, “the devil is in the details.” We hope these questions will guide that process:

Scoring:

  • Are the score reports from the SAT, ACT, and PSAT detailed enough to inform instruction? Can educators use the reported data from these tests to make necessary instructional improvements?
  • Is there an alignment study between ACT Aspire, ACT, SAT, PSAT and FSA that demonstrates these assessments can be used interchangeably? How can scores on different tests be used to fairly rank schools and districts or to measure annual learning gains?
  • When a school administers the SAT or ACT to all its students as a part of our Accountability system, who controls those scores? Are they released directly to the students or will they go to the state for review before being released to the public?
  • If a student scores a 1200 (75% tile) on the new SAT (which is composed of Critical Reading, Writing and Math) they are exempt from both the U.S. History and the Biology state mandated EOC (page 11). Why do scores on unrelated standardized tests exempt students from History and Science EOCs? Again, these EOCs are mandated to be 30% of a student’s grade. Is it fair to the remainder of the students in the class that these students are exempt from the EOC without demonstrating any proficiency in course content?

State mandated End of Course exams:

  • Students who score high enough on an alternate assessment are exempt from taking certain state mandated EOCs. Pages 7-12 of the bill outline the scores necessary on the ACT, ACT Aspire, PSAT and ACT to satisfy the requirements for the state mandated EOCs (Algebra 1, 2, Geometry, Biology and U.S. History). Why are there no comparable FSA scores to allow EOC exemption? In a fair system, all students should have the opportunity for EOC exemptions.
  • How is the requirement that state mandated EOCs are worth 30% of a students course grade fair, when some students are exempted from the EOC completely? How will exempted students’ course grades be calculated? Will this have an unfair affect on GPA calculation for students across the state?
  • If the top test takers are now exempt from taking the state mandated EOCs, what impact will that have on the passing rates of the EOCs? One would predict that if the top 25-50% of test takers were eliminated, and the remaining cohort performed similarly to previous years, then the apparent failure rates will skyrocket (the number of students failing would remain relatively constant but the denominator- the number of students taking the exam- would be dramatically smaller). How will this affect perceived achievement gaps? Will cut scores be adjusted downward to mitigate this? How will that affect the “rigor” of these classes?
  • If the top 25% or more of students are exempted from the Math EOCs because of concordant scores, how will middle school grades be calculated- which depend on performance in Algebra and Geometry EOCs?
  • How are learning gains calculated in a class where only some of the students take the EOC?

Validity:

  • Are the scores referenced in the bill from the old SAT or the new SAT? (Old SAT max score was 2400, new SAT (yet to be administered) max score will be 1600.) We believe this should be clarified in the bill.
  • Has the new SAT been validated? Has it been evaluated for fairness, reliability and validity for Florida’s vulnerable subpopulations of students: ESE, ELL, low income? We believe the answer to this is “not yet.”
  • Why are we considering the use of the new SAT, which has never been given before and has questions regarding its validity? Could this lead to the FSA fiasco all over again, for districts choosing the SAT?
  • Currently, ESE students can receive accommodations during the FSA administration. Are accommodations allowed for the SAT, PSAT, ACT or ACT Aspire? If accommodations are provided for the SAT and ACT, will the resulting scores be usable for college admission? Will a student allowed accommodations on the PSAT be eligible for a National Merit Scholarship?

School grades:

  • How will learning gains be calculated if the administered test changes from year to year in districts?
  • Passage rates on the Biology EOC and the US History EOC are components of the High School Grading Formula. How will this be impacted by student exemptions?
  • How can you honestly and fairly compare middle school based on their advanced math scores when one school might have all students taking the Algebra 1 EOC, another might have mostly students exempt from the EOC based on ACT Aspire scores, and a third might have all their students take the FAA Ground School Industry Certification, avoiding teaching Algebra in that school all together?

VAM – Teacher Evaluations:

  • How will VAM be calculated for teachers when only some of the students in the class are required to take the EOC and the rest are exempt due to a test score they achieved before enrolling in the class?

Parental Choice:

  • In SB1360, if a school district chooses to use an alternate assessment (PSAT, ACT, etc), a parent can elect for their child to take the FSA, instead (keep in mind, there is no other reason to administer the FSA, except for annual testing).  However, if a school district chooses to stick with the FSA, parents are NOT allowed to choose an alternate assessment to satisfy their child’s annual testing requirement (even though every school district will still be administering the ACT, PSAT and SAT). Why not allow parents to choose an alternate assessment?

Exemptions:

  • If SAT and ACT scores have been shown to correlate strongly with socioeconomic level, won’t this system of exempting students with high test scores from multiple other tests, only minimize testing, on average, for more advantaged students and schools?
  • Why doesn’t passing an AP literature exam count as an exemption for the 10th grade ELA graduation requirement?

Funding:

  • How can the transition from primarily FSA/AIR created tests to the use of FSA, ACT Aspire, ACT, PSAT and SAT, with the number of counties choosing any one test varying every year, be a budget neutral endeavor?
  • Currently, schools receive an extra stipend for students enrolled in AP courses. If a student gains credit for taking an AP course just by passing the AP exam, does the school receive the AP stipend? If a student passes the AP exam without taking the associated course, is their AP exam score calculated into their high school’s school grade?
  • Has there been any consideration to having a three-year pause in the repercussions of our Accountability system (including funding implications), while we transition to this more complicated plan?

Alignment to new ESEA law:

  • We have heard there are 6 other states which use ACT Aspire for their annual testing. Do any of those states assess their 3-8th graders with two different tests?
  • Are there any states which use a smorgasbord of assessments for their annual assessment of high school students?
  • Can federal testing requirements to annually assess every student in math and reading be fulfilled by scores from multiple different tests administered in the same year?

As you can see, we have found quite a few things about this bill that seem “a little bit rough.” If Senator Gaetz is serious about wanting to make this bill better, he has his work cut out for him.

At Accountabaloney, we are not against testing; we want appropriate tests used appropriately.  We want an accountability system that can be trusted. SB1360, though well intentioned, will cause chaos in the current, already shaky, accountability system.  Districts will scramble to choose the testing scheme they predict will result in the highest school grades.  The DOE will be overwhelmed with the requirements that they determine comparable scores between a myriad of tests, set cut scores on nationally normed assessments, negotiate/renegotiate contracts with 3 different testing companies that must ultimately be “budget neutral” and develop a system to collect, monitor and compare test data from multiple sources.  And, perhaps most concerning, high performing schools, where many students will be exempt from state mandated EOCs, will see a decrease in testing, but less advantaged schools, where fewer students will reach the threshold for EOC exemption, will see little or no reduction in testing. Is that fair and/or appropriate?

We hope Senator Gaetz and his team can address these issues. As written, this bill will not shore up the accountability infrastructure but will serve to make a shaky foundation even more unstable.

Get out your hard hats.  Watch for falling debris.

SB1360: Baloney on Rye ADDENDUM

This is an addendum to our previous blog, “SB1360: Baloney on Rye is Still Full of Baloney” :

It has been brought to our attention that it is unclear whether the SAT score targets described in SB 1360 reflect scores from the current SAT or from the “newly designed” SAT, which will debut later this year (info here and here). The redesigned SAT will have a maximum score of 1600, compared to the current SAT maximum score of 2400. Since the exam is yet to be administered, the percentile ranking of scores on the new SAT can only be predicted. It is estimated that a score of 1200 (that required to be exempt from Florida’s U.S. History EOC) will be closer to the 75th percentile on the new SAT (not the 15% we stated in our blog).

Additional comparisons with SB1360’s target scores for the ACT, suggest that exemptions for Algebra 1, Geometry and Algebra 2 EOCs may be closer to the 50th to 75th percentiles, respectively. So, SB1360’s required scores may be more “rigorous” than we first thought, but will they be appropriate? It turns out neither the old nor the new the SAT assess math skills beyond basic geometry. Why are we allowing scores on an assessment that does not test beyond basic geometry to exempt students from their Algebra 2 EOC, which covers up to Trigonometry concepts? We hope the Senate Education committees will address this.

Since there are dramatic differences between the performance level associated with the same reported score, we feel SB1360 needs to define exactly which SAT exam (old or new) it is referring to. We also question why Florida would put into statute target scores from an exam that is yet to be administered (even if it does have the same name)? Are Florida students expected to field test the new SAT and then have those scores used for accountability purposes? Remember how well that worked out for the 2015 FSA?

Also, there are significant concerns regarding the math portions of the newly designed SAT, especially for low income and English language learner (more here). The new format of math questions will require higher level verbal and reasoning skills and is predicted to put English language learners and low income children at a significant disadvantage. Given the ever increasing population of low income, English language learner, and immigrant students in our public schools (Miami Dade is currently expecting ~8,000 new immigrant students this school year), why is Florida choosing an exam that would put those students, their schools and districts, at a distinct disadvantage. How is that a fair accountability assessment?

Our initial blog may have underestimated the “rigor” of SB1360’s target scores. If they represent scores from the new SAT, they may be more “rigorous” than we thought. Does this make us feel any better about this bill?

No.

Reviewing the new SAT only raises more questions about the fairness of an accountability system that uses these scores as metrics and in this manner. Students with high standardized test scores (even in subjects unrelated to the course they are taking) will be exempt from taking final exams/EOCs. Students with lower standardized test scores (many who will be immigrants, disadvantaged and/or english language learners), will not only be required to take the exams, but they will be worth 30% of their course grades and (for Algebra 1) passing will be required for graduation. “Smart kids” (often wealthier, white students) will no longer need to take the U.S. History or other state EOCs. They will be exempt from the Algebra 2 EOC based on scores that don’t test the course content; their course grades will reflect their classroom performance and will not suffer from poor performance on the EOC. It appears that students and schools with high test scores (like Seminole County, which has been lobbying hard for this bill, originally calling it the “Seminole Solution”) will require significantly less testing than their less advantaged counterparts.

This does not describe a fair, equitable, uniform education system. This describes the misuse of standardized test scores.

This will not “fix” anything.

This is Accountabaloney.