Algebra 2 EOC Results: Has Florida Jumped The Shark?

fonz_shark

In 1977, in an episode of the sitcom “Happy Days”, Fonzie displayed his bravery by jumping over a confined shark while on waterskis. Ever since then, the phrase “jumping the shark” has come to mean the moment in a TV series that defines the beginning of the show’s decline; a desperate attempt to retain viewership where it becomes obvious to the audience that the show has strayed irretrievable from its original formula. We believe the administration of the 2015 Algebra 2 FSA EOC (Florida State Assessment End of Course exam) will be remembered as the moment when Florida’s Education Accountability System “jumped the shark.”

Over the last week or so, the preliminary FSA EOC student results have been released to parents. Because the cut score process is still in progress, the reports show a percentile ranking for the child and the score breakdown of questions answered correctly in each “reporting category”. The full “horror” of the Algebra 2 EOC is being revealed.

Red flags were raised regarding the Algebra 2 EOC even before it was administered. Prior to the Algebra 2 EOC, every previous state-created EOC had been treated as a baseline administration during its inaugural year. Students would take the new EOCs in their inaugural year but performance on those test would not affect their course grade. The Algebra 2 EOC, however, from day one was planned to be worth 30% of the student’s course grade. Multiple emailed questions regarding this break from protocol went unanswered by the Florida Department of Education last spring.

Students recognized significant problems with the Algebra 2 EOC while it was being administered last spring. Here is a first person account from a 15 year old, gifted, straight A, honor student from Monroe County.

“I took the Algebra 2 EOC last year and it was absolutely ridiculous. I got my scores back and it said that I did better than 83% of the students that took it, but I’m not sure how they could even score such a horrible test. I walked out of the testing room and knew for sure that I failed it, and that they were not going to count it because of how flawed it was. The format that the test was on had many errors, and was filled with information that the teachers were not asked to teach. For example, the test would say “graph this piece wise function” and it would give you the wrong tools to graph it with, so even if you were totally capable of completing the question successfully, you couldn’t.”

We are certain that almost every Algebra 2 student will tell a similar story. The test was a disaster, bringing usually confident high school students to tears.

Before we discuss the results, understand this: Florida (along with the rest of “Common Core” America) has a “standards based education”. Teachers are asked to teach the standards and if students can demonstrate, on their state assessment, that they have learned enough of those standards, they will be deemed “proficient” and “pass” the exam. Florida State Assessments (like the previous versions, the FCAT) are “Criterion Referenced Tests” or CRTs. A CRT should compare a student’s performance to established expectations; if every student meets those expectations, all students would be deemed proficient. Likewise, if no students met the established criterion, all would fail. On CRTs, it is not only possible, but desirable, for every student to pass the test or earn a perfect score. (Remember that, it is important.)

A common example of a CRT is the written test for your driver’s license: if you answer 8 out of 10 questions correctly, you pass the test… even if thousands before you answered 9 or more correctly. (Learn more about Criterion Referenced Tests and their usage here)

The first problem parents should notice when reading their child’s Algebra 2 EOC report is that the score is reported as a percentile ranking, which is an inappropriate way to present results from a criterion based exam. Remember, if all students achieved the established expectations, ALL students would pass. In a criterion based exam, percentile ranking should be irrelevant.

Why would the Department of Education release the percentile ranking of students for a Criterion Referenced Test? Probably, in part, because they don’t expect parents to understand the difference and they are under time pressure to release some information regarding student performance, given that the exam was taken more than 6 months ago. The main reason, though, is that the established expectations for the new FSA have not yet been established and won’t be established until the January 4th State Board of Education meeting (more on that later).

So, despite having NO established performance standards, the Algebra 2 results were released to parents, highlighting a student’s percentile ranking in the state and their performance in individual “Reporting Categories.” Here is what we have learned from reports parents have shared with us:

ALGEBRA 2 2015 FSA EOC results

#correct (out of 56) Total %correct Student percentile ranking
16 25% 63 %tile
18 27% 67%tile
20 36% 71%tile
29 52% 88%tile
43 77% 99%tile

 

Keep in mind that those who took Algebra 2 are essentially all college bound, often honors and/or gifted math students, and this exam is, by statute, worth 30% of their course grade (the 30% was waived after test administration last spring but, by report, will be back in effect next spring). The State’s most gifted math students (99th percentile) still missed one out of 4 questions and students will have passed (if the Commissioner of Education’s current cut score recommendation is approved) missing ¾ of the questions. Something definitely seems amiss.

Remember: “On criterion-referenced tests, it is not only possible, but desirable, for every student to pass the test or earn a perfect score”. So why has Florida created a CRT where its most gifted math student are unable to answer 25% of the test questions correctly?

Of course, the cut scores/expected performance levels have not yet been set for the Algebra 2 exam, so these student reports, actually, say nothing as to whether a student passed or not.

On September 15-17, 2015, at a Rule Development Workshop, the general public got a first look at the impact of potential cut scores. It wasn’t pretty. You can see the complete presentation here.

FullSizeRender-10

To summarize, the Educator Panel recommended cut scores than would fail 69% of students, including 85% of African American students. Since then, the Commissioner of Education has made her recommendations which would (only) fail 64% of students. The ultimate cut scores will be determined by the Florida Board of Education (BOE) on January 4, 2016. Several members of the board have indicated they wanted cut scores that were even higher than the Commissioner’s, failing even more students. Sadly, the process of establishing cut scores seems to reflect political ideology more than sound education judgement. (Learn about the cut score process here)

Prior to the release of the Algebra 2 test scores, on October 28th, I spoke in front of the BOE, and described why I felt “the most egregious example of Accountabaloney is the state’s unwavering assertion that the Algebra 2 EOC is valid.” You can watch my speech here. In summary:

  • In 2015, Algebra 2 had a new set of standards, which now included Pre-calculus and Statistics (learn about the standards here).
  • The commissioner has admitted that no one knows whether it is even possible to teach these standards in a single school year (details here).
  • The Algebra 2 EOC could not have been field tested because Utah does not test Algebra 2 or Trigonometry standards.
  • The Alpine Validity Study completely ignored Algebra 2 in the interest of time (Report here).
  • Honor students reported being unable to answer any questions on day two of this EOC, so it should have been no surprise that the exam’s results were total outliers with almost half of students failing including 85% of African American students. (Report of results here)
  • At the Keep Florida Learning Committee, the department suggested the problem was “the teachers didn’t teach the standards”. (more info here)
  • Crushing the confidence and destroying the GPA of our best and brightest students does NOT make them college and career ready. Blaming it on their teachers does NOT help recruit and retain high quality STEM educators.

I concluded with “the Algebra 2 exam and its standards deserve a complete review.” Seriously, parents shouldn’t have to ask. In the face of falling national math scores (read here), the entire math sequence deserves a review.

Currently, parents and teachers report concern regarding both the pace and scope of the new Algebra 1 and Algebra 2 standards. When Florida switched to the Common Core-like Florida Standards in 2014, much of what used to be taught in Algebra 1 was moved to Pre-Algebra. Algebra 1 now includes statistics and at least a third of the previous Algebra 2 content. Algebra 2 now includes pre-calculus and statistics. The Florida Standards are now out of alignment with the new CCSS-based textbooks. If a teacher teaches from the new, state approved Algebra 1 or 2 textbooks, they will not cover the required standards. This is a problem. Why would the State approve textbooks that do not cover the required course content?

In addition, the amount of content seems to be too much to complete in a single year, especially since the EOCs are given in late April/early May. One Algebra 2 teacher said:

“The problem is that there are TOO MANY standards in the test item specs. It’s simply TOO MUCH to teach, prior to the Algebra 2 EOC which starts in early April to the first week in May. We have to teach at the speed of light to get everything in, and it’s nearly impossible to do so.”

Parents are concerned that the pace does not allow their kids to fully understand concepts and many kids are dropping out of subsequent advanced math courses. Another math teacher wrote:

“Algebra 1 now looks like Algebra 2 and Algebra 2 now looks like Precalculus/Statistics. Regular kids who are not strong in math are drowning. It’s just so developmentally inappropriate. These new standards and new EOC’s are geared towards honors students who excel in math.”

Clearly, when Florida created its new “rigorous” Math standards, moving a significant amount of Algebra 2 content into the Algebra1 course, it created problems beyond the EOCs. By statute, passing the Algebra 1 EOC is a graduation requirement so Florida now requires a mastery of the higher level Algebra 2 content to graduate high school. Was this the legislature’s intention? Also, in the current A-F grading system, middle schools are being graded based on their number of advanced math students (Algebra 1 and Geometry) encouraging increasing student placement into Algebra 1 (Learn about how School Grades are calculated here). Now these very young math students are being asked to master Algebra 2 standards; is this developmentally appropriate? Also, the middle school math teaching credential is for grades 5-9 and DOES NOT include Algebra 2 content. Now that Algebra 2 standards are placed in Algebra 1 courses, most middle school math teachers are not credentialed to teach the content. This is a serious problem.

The current pace, scope and assessment of Math standards has been significantly disrupted by #accountabaloney. We believe re-evaluation of the entire advanced math sequence and math FSA EOCs is warranted.

Here are the concerns that must be addressed:

  1. The overwhelming evidence suggests there is CLEARLY something wrong with this Algebra 2 exam, yet the DOE is releasing test scores, cut score determinations are underway and the test remains on the Spring 2016 schedule. This defies reason. Is the DOE addressing this? Why would they release the scores from such a disastrous test administration? Why even require a state mandated EOC for a non-required class?
  1. Has either the Algebra 1 or 2 EOC been definitively shown to be fair, reliable and valid for special populations including special education (ESE), English Language Learners (ELL), low income or racial subgroups? Where are those reports? 85% of African Americans failed the Algebra 2 EOC… 85% of the best and brightest African American students failed this test… Something is wrong. Also, since passing the Algebra 1 EOC is a graduation requirement, the Algebra 1 EOC definitely needs to be evaluated for fairness for ESE, ELL, etc, students. Where is the evidence this has been done on these new EOCs? (Please don’t accept the answer “the Alpine Validity Study”; it did NOT address these issues.)
  1. Is there any evidence that an accelerated pace through advanced math concepts is a good idea? Why have Algebra 2 standards in an Algebra class or pre-calculus standards on Algebra 2?  Should Algebra 2 mastery be required for high school graduation. Are middle school teachers properly credentialed to teach the current Florida Algebra 1 standards?
  1. We have significant concerns regarding the entire accountability system, extending beyond the math EOCs, at this point. Clearly the “establishing cut score” process has as much to do with political ideology as it does educationally sound practices. We worry that, with the new tests and the new more “rigorous” cut scores, the initial intent of many of the State’s legislated mandates are no longer aligned to the current assessments. For example, mandatory third grade retention is meant to identify and provide interventions for students reading below grade level; is there any evidence that the current cut scores assess grade level proficiency? Will 3rd grade students reading at or above grade level be marked for retention? Likewise, passing Algebra 1 was a challenging requirement for high school graduation but now it appears the EOC is testing a much higher, more challenging Algebra 2 curriculum. Is that what the original legislation intended? We suspect not.

CALL TO ACTION:

We encourage parents and teachers to share this blog with Governor Rick Scott, Commissioner Pam Stewart, the Florida Board of Education and your legislators. Tell them the Algebra 2 EOC is the most egregious example of the problems with the current accountability system. Ask them how parents are expected to have confidence in a system that creates final exams that are so abusive in nature and then seemingly ignores the obvious issues. If the DOE is not evaluating the math standards and assessments, demand they do so. If they are investigating these issues, remind them it would be wise to inform teachers and parents before we lose all confidence and respect.  We need a DOE that can recognize flaws in the current system and work to fix them. Florida needs to re-evaluate both the advanced math sequence and math FSA EOCs.

With the creation and administration of the Algebra 2 EOC, we believe Florida’s Education Accountability has “jumped the shark.” Without dramatic policy reversals, we fear parent confidence in the system will be irretrievable.

Advertisements

Is there a respectful synonym for “punish?”

Accountabaloney Graphic2_SMALLIt has come to our attention that some accountabaloney terms may need to be better defined.  Let’s take a look at the word “punish” and discuss whether that word is a “respectful” word that accurately describes the consequences of high stakes testing in Florida.

On Oct 1, 2015, the  Florida Learning Committee (FLC) convened to discuss everything from accountability to class size. Last winter, in response to outcries regarding overtesting in our public schools, Governor Scott ordered the formation of the FLC, which was to be composed of educators, policy makers and parent representatives from across the state (click here for more info).   During the October 1st brainstorming session, a parent representative, Julia (Megan) Hendricks from Pasco County raised a very important issue, the issue of the high stakes connected to the state assessments.  You can watch entire meeting here (the interaction begins at 1:13:58).

“I would like to piggyback on what several  people have said about the use of assessments, because we are in actuality using it, the FSA, for much more than informing instruction, I mean there are some pretty high stakes attached. I think we’re all pretty aware of what those are as far as evaluating teachers, schools, retention…. so I’d like to even go further and be even a little bit more bold and say that we should reconsider using standardized test to evaluate and punish students, teachers, and schools.”

Commissioner Pam Stewart, immediately, interrupted the brainstorming session (which, I believe, is against “brainstorming rules”) to interject “…we at the State are not punishing…”

When Ms. Hendricks repeated her concern, Brian Dassler responded (in what seems quite condescending):

Response from Brian Dassler, Deputy Chancellor for Educator Quality at the Department:

“I’m going to say that punishing is probably not a respectful use of the word, Megan, because I don’t think it would be appropriate to assign intent there. If you would like to amend that word to something else that would capture what you are trying to say, but in a way that would be  respectful…”

Ms. Hendricks, struggling to find a “respectful” synonym for “punish”, replies that she chose the word “punish” because “that’s the way it is perceived by parents“.

Is “punish” used appropriately in discussion of Florida’s high stakes accountability system? Let’s look at its Google definition:

FullSizeRender

 

 

 

 

 

 

 

 

 

 

According to Google, punish means “to treat (someone) in an unfairly harsh way” as in “raising the achievement levels after a test was given would punish low income, children of color.”

As an aside, while investigating for this blog, we also learned that “rigorous” is a synonym for “punishing”, and, well, rigorous is one of Pam Stewart’s favorite, most often used words… perhaps we’ll save the discussion of rigor for a future blog post.

Why would parents feel the state uses FSA scores in a punitive way? What are the ways that Florida’s system appears to punish or penalize teachers, students or schools?

Here are a few examples:

If this is not punishing, or at least the threat of punishment, then we respectfully ask the DOE to please explain what they would consider is punishing?  The DOE may consider these unintended consequences, but at the end of the day, the enormity of the imposed consequences can no longer be viewed as “unintended.”

The high stakes attached to our Florida assessments ARE a huge problem and they ARE punishing students, teachers and schools, whether the DOE finds the terminology respectful or not.  If the Department of Education wants to retain any measure of respect, open discussions regarding the punitive nature of these high stakes tests are vital.  Ms. Stewart’s nitpicking of the use of the word “punish” was not respectful to Ms. Hendricks, especially since “punish” is a word that is widely used and understood throughout the state and nation to accurately represent what is happening in our schools.

Commissioner Stewart appears to be trying to hide the issue of high stakes under the rug, where nobody can see it.  However, for most parents, the problems associated with punitive high stakes are front and center. By continually choosing to ignore legitimate concerns from parents and educators across the state, it is Commissioner Stewart and the FLDOE who are acting disrespectfully.

If the DOE doesn’t want to be accused of “punishing” then it should become less punitive. If they want to regain respect, they should begin by respecting the real concerns of their stakeholders.

Across Florida, parents and educators are discussing the need to remove the high stakes from the state’s standardized testing. We respectfully request the DOE join us in that conversation.

Many thanks to Ms. Hendricks for boldly representing parents and choosing a word that accurately reflects our concerns regarding the use of FSA scores.