FSA Technical Report: Looking For What Isn’t There

“Looking For What Isn’t There,” the Brief Version:

Almost since the day the Department of Education signed the contract with test developer American Institutes for Research (AIR), the validity of the Florida Standards and the Florida Standards Assessments (FSA) have been questioned. First, we were told it had been field tested and validated in Utah. When no documentation was forthcoming, legislators demanded an Independent Validity study. They hired Alpine Testing Solutions, a company anything but independent from FSA creator AIR, and the validity study was released in 8/31/15. The results were anything but reassuring, evaluating only 6 of 17 new assessments and finding the use of individual test scores “suspect” but, strangely, supporting the use of scores for the rating of teachers, schools and districts. At the time, we were advised to “look for what isn’t there” and we found no evidence the tests were shown to be valid, reliable or fair for at risk sub-populations of students. “Looking for what isn’t there” seemed like good advice, so when the 2015 FSA Technical Report was released, I started looking…

In a nut shell, despite its 897 page length, there seems to be a LOT that isn’t in the 2015 FSA Technical Report. To summarize, here is a list of the most obvious omissions:

  • Though the report clearly states that validity is dependent on how test scores will be used, this report seems to only evaluate the validity of the use of scores at the individual student level. The use of test scores to evaluate schools and districts is mentioned but there is no evidence those uses were ever evaluated.
  • The use of student scores to evaluate teachers (via VAM scores) is completely ignored and is left off the “Required Uses and Citations for the FSA” table, despite such use being statutorily mandated.
  • Despite the previous FCAT 2.0 Technical Report‘s concerns questioning “the inference that the state’s accountability program is making a positive impact on student proficiency and school accountability without causing unintended negative consequences,” no evaluation of these implication arguments are made for the FSA (and I don’t believe that is because there ARE, in fact, no unintended consequences).
  • Missing attachments to the report include: multiple reported appendices containing statistical data regarding test construction and administration, any validity documents or mention of the SAGE/Utah field study, any documentation of grade level appropriateness, any comparison of an individuals performance on both a computer and paper based test.
  • Results from the required “third-party, independent alignment  study” conducted in February 2016 by HumRRO (You guessed it! They are associated with AIR and they have a long history with the FLDOE).

Who is responsible for these documents that create more questions than they answer? Why aren’t they being held accountable? Why, if virtually our entire Accountability system is dependent on the use of test scores, isn’t it a top priority to ensure these tests are fair, valid and reliable? When Jeb Bush said “If you don’t measure, you don’t really care,” was he speaking of test validity? Because, it appears the FLDOE really doesn’t care.

Want more details? Our full blog is here:

FSA Technical Report: Looking For What Isn’t There

In early April, 2016, the FSA Technical Report FINALLY was published.  You can read it here.  The Florida Department of Education (FLDOE) publishes a technical report annually following state testing.  In general, these reports review general information about the construction of the statewide assessments, statistical analysis of the results, and the meaning of scores on these tests. Per the 2014 FCAT Technical Report,  these reports are designed to “help educators understand the technical characteristics of the assessments used to measure student achievement.” Usually, the report comes out in the December following the spring test administration.  This year, the FSA report was expected in January, following the completion of the “cut score process”  by the Board of Education on January 6, 2016. Still, there were significant delays beyond what was expected.

When you look at the new FSA report, the first thing that you notice is the format of the report is completely different from the previous technical reports. The 2014 FCAT report was published as one volume, 175 pages long, and referenced a “yearbook” of appendices that contained detailed statistics on the various assessments for the given academic year. The FSA report was published in 7 volumes, totaling 897 pages, and 5 of the seven volumes reference multiple appendices, which contained specific statistical data regarding test construction and administration (like Operational Item Statistics, Field Test Item Statistics, Distribution of T Scores, Scale Scores, and Standard Errors), that are NOT attached to the report (This is the first thing I found that wasn’t there).

What is the definition of  Validity?

The two reports have slightly different definitions of “validity.” The 2014 FCAT Report (page 126) defined validity this way:

“Validation is the process of collecting evidence to support inferences from assessment results. A prime consideration in validating a test is determining if the test measures what it purports to measure.”

The 2015 FSA (Volume 4) report’s definition is more convoluted (page 4):

“Validity refers to the degree to which “evidence and theory support the interpretations of test scores entailed by proposed uses of tests” (American Educational Research Association [AERA], American Psychological Association [APA], & National Council on Measurement in Education [NCME], 2014). Messick (1989) defines validity as “an integrated evaluative judgment of the degree to which empirical evidence and theoretical rationales support the adequacy and appropriateness of inferences and actions based on test scores and other modes of assessment.”

Both of these definitions emphasize evidence and theory to support inferences and interpretations of test scores. They are perfect examples of the most obvious difference between the two reports.  The 2014 report is written in relatively clear and concise English and the new, 2015 report is verbose, complicated and confusing.  Seriously, has the definition of validity changed? Is it really necessary to reference sources defining validity?  Why does this remind me of when John Oliver said “If you want to do something evil, put it inside something boring”?

More importantly, does either report actually measure what it purports to? Both definitions caution that it is really the use of the test score that is validated, not the score itself.

Let’s take a quick look at how the FSA scores are used:

FullSizeRender-22

Table 1 (2015 Volume 1 page 2) delineates how FSA scores are to be used.  In addition to student specific uses, like 3rd grade retention and high school graduation requirement, required FSA score uses include School Grades, School Improvement Ratings, District Grades, Differentiated Accountability and Opportunity Scholarship.

Interestingly, this list does NOT include Teacher Performance Pay (or VAM calculations) yet the use of student test scores is clearly mandated in F.S. 1012.34(3)(a)1. For many, the use of student scores to evaluate teachers is one of the most contentious parts of the state’s accountability system. Is this an oversight or is there reluctance to put the use of VAM to the validity test? Does it matter to the FLDOE whether VAM is a valid assessment of teacher quality or do they plan on using it regardless?  Remember what Jeb said? “If you don’t measure, you don’t really care,” and it appears the FLDOE does not care.

So, did these Technical Reports validate their respective tests for these uses?

Not that I can tell.

Neither report includes (as far as I can tell) evaluations confirming the use of these tests for the determination of grade promotion or high school graduation. Indeed, the Alpine Validity report cautions against the use of individual scores calling them “suspect” for some students. There appear to have been no attempt to document that FCAT or FSA test scores can accurately rate schools, districts or teachers.

In fact, the 2014 FCAT 2.0 Technical cautioned such use on page 137.

“At the aggregate level (i.e., school, district, or statewide), the implication validity of school accountability assessments can be judged by the impact the testing program has on the overall proficiency of students. Validity evidence for this level of inference will result from examining changes over time in the percentage of students classified as proficient. As mentioned before, there exists a potential for negative impacts on schools as well, such as increased dropout rates and narrowing of the curriculum. Future validity studies need to investigate possible unintended negative effects as well.”

The “Summary of Validity Evidence” in the 2014 Report is telling.  While they conclude that the assessments appeared to be properly scored and the scores could be generalized to the universe score for the individual, they had significant concerns regarding the extrapolation and implication arguments (emphasis mine):

“Less strong is the empirical evidence for extrapolation and implication. This is due in part to the absence of criterion studies. Because an ideal criterion for the FCAT 2.0 or EOC assessments probably cannot be found, empirical evidence for the extrapolation argument may need to come from several studies showing convergent validity evidence. Further studies are also needed to verify some implication arguments. This is especially true for the inference that the state’s accountability program is making a positive impact on student proficiency and school accountability without causing unintended negative consequences.”

In April 2015, I emailed Benjamin Palazesi, from the FLDOE, asking if such “further studies” were ever done to verify the implication arguments, as suggested in the FCAT 2.0 Report. His response? “Since the FCAT 2.0 Reading and Mathematics and Algebra and Geometry EOC Assessments are being replaced by the Florida Standards Assessments (FSA) in these subject areas, there are no plans to conduct a criterion study on these assessments, and we will evaluate the need for additional studies for FSA.”

Hmmm, there is no mention of implication arguments at all in the FSA Report. Do you think they believe there are no unintended negative consequences due to the state’s accountability program? Maybe they don’t read our blog… unintended consequences seem to be a speciality of Florida’s accountabaloney system. Eventually, the FLDOE will need to recognize and address the many unintended consequences of their current system or such consequences can no longer be considered “unintended.”

The validity of the FSA has been in question since it was announced its questions would be “borrowed” from Utah’s Student Assessment of Growth and Excellence (SAGE). On March 4, 2015 (watch the questioning here at 58:26 or read about the “fall out” here), Commissioner Pam Stewart testified in front of the Senate Education Appropriations Subcommittee and proclaimed that the FSA was field tested in Utah and that it was “absolutely psychometrically valid and reliable.” At the time, Ms. Stewart promised to provide documentation to the Senate Education Subcommittee members. Some school board members from Utah were also interested in these documents, as they had not yet received any formal documentation regarding the validity or reliability of their own state test, SAGE (see letter from the Utah School Board here). No documents were ever delivered and, SURPRISE, there is no evidence of a “field test” in Utah or any SAGE Validity documents in this 2015 Technical report, either. (Now might be a good time for Commissioner Stewart to apologize for misleading this Senate Education Subcommittee.)

The 2015 Technical report does include both the legislative mandated Alpine Validity Study (Volume 7 Chapter 7) AND the Alpine presentation to the Senate (Volume 7 Chapter 6).  Remember the Alpine Validity Study, because of time constraints, chose not to assess validity for 11 of the 17 FSA tests, including the Algebra 2 and Geometry EOC. The Alpine study also did NOT assess validity or fairness for at-risk populations of students, like ESE or English Language Learners.

Another thing missing from these reports is any assurance that the level of performance tested is grade level appropriate. Neither technical report compared student performance on the FSA/FCAT to performance on a nationally normed test.  There is no measurement as to whether the 3rd grade Reading FSA, for example, actually tests 3rd grade reading levels (yet students are retained based on its results). This, I believe, is a major concern for parents and  has been seemingly disregarded by the state in the pursuit of “rigor.” Again, “if you don’t measure, you don’t really care” and the FLDOE appears not to care if children who can actually read at a 3rd grade level have been marked for retention.

There is a brief mention (1 paragraph) of statistical fairness in items (Volume 4 page 60), utilizing Differential Item Functioning (DIF) analysis. “DIF analyses were conducted for all items to detect potential item bias from a statistical perspective across major ethnic and gender groups” (Male/female, White/African American/Hispanic, English Language Learner and Students with disabilities). DIF was also used in the 2014 report, but there it seems to have been used to eliminate biased questions that were being field tested.  In the 2015 report, the DIF analysis is implied to assure fairness across subpopulations.

In section 5.2, Volume 1, page 20, DIF analysis is described (emphasis mine).

“Identifying DIF was important because it provided a statistical indicator that an item may contain cultural or other bias. DIF-flagged items were further examined by content experts who were asked to reexamine each flagged item to make a decision about whether the item should have been excluded from the pool due to bias. Not all items that exhibit DIF are biased; characteristics of the educational system may also lead to DIF. For example, if schools in certain areas are less likely to offer rigorous Geometry classes, students at those schools might perform more poorly on Geometry items than would be expected, given their proficiency on other types of items. In this example, it is not the item that exhibits bias but rather the instruction.”

I am not a psychometrician, but I do wonder how a test that is used to rate not only students but also schools, can determine a question is not biased because students came from a low performing school; especially since many “low performing schools” contain an overrepresentation of students from at-risk sub-populations. Regardless, I suspect that determining whether individual test questions are biased is not the same thing as evaluating whether a test is fair and valid for those at-risk populations.

Recent reports demonstrated that students who took the paper/pencil version of the PARCC test obtained higher scores than those who took the computer version. Was this evaluated for the FSA? Not in this Technical report, where the only evaluation of the paper/pencil test appears to be content alignment for students with accommodations .

One more thing to add to our list of things not in the report: page 20, section 3.5 of Volume 2 of the 2015 report  states “A third-party, independent alignment study was conducted in February 2016. Those results will be included in the 2015-2016 FSA Technical Report.” According FLDOE Deputy Commissioner, Vince Verges, that report is being completed by Human Resources Research Organization (HumRRO) who (SURPRISE!) were thanked, along with AIR, in the acknowledgements of the Alpine Validity Study as “organizations and individuals that serve as vendors for the components of FSA that were included in the evaluation.” Indeed, HumRRO has a long history with the FLDOE (read about it here). Seriously, the DOE needs a dictionary with the definitions of “third-party” and “independent” because HumRRO might be neither.

After looking what isn’t in the FSA Technical Report, I have come to a few conclusions:

  1. There was never a valid field test in Utah.
  2. The Alpine Validity Study was incomplete.
  3. These technical reports are more about describing test construction and administration than confirming validity.
  4. There remains no evidence that the FSA is fair, valid and reliable for at-risk subpopulations of students or the myriad of uses outside of the individual student.

Yet, we continue to use the FSA to retain students, deny diplomas and rank teachers, schools and districts. That is accountabaloney.

Why has so little time been spent assuring the validity of these tests? The FSA is the cornerstone of Florida’s education accountability system.  Why hasn’t serious attention been paid to assuring its validity?  Could it be because, as Jeb Bush has said, so many times, “If you don’t measure, you don’t really care.” I am beginning to believe that is true and am wondering who should we hold accountable for that?

SB1360: Baloney on Rye ADDENDUM

This is an addendum to our previous blog, “SB1360: Baloney on Rye is Still Full of Baloney” :

It has been brought to our attention that it is unclear whether the SAT score targets described in SB 1360 reflect scores from the current SAT or from the “newly designed” SAT, which will debut later this year (info here and here). The redesigned SAT will have a maximum score of 1600, compared to the current SAT maximum score of 2400. Since the exam is yet to be administered, the percentile ranking of scores on the new SAT can only be predicted. It is estimated that a score of 1200 (that required to be exempt from Florida’s U.S. History EOC) will be closer to the 75th percentile on the new SAT (not the 15% we stated in our blog).

Additional comparisons with SB1360’s target scores for the ACT, suggest that exemptions for Algebra 1, Geometry and Algebra 2 EOCs may be closer to the 50th to 75th percentiles, respectively. So, SB1360’s required scores may be more “rigorous” than we first thought, but will they be appropriate? It turns out neither the old nor the new the SAT assess math skills beyond basic geometry. Why are we allowing scores on an assessment that does not test beyond basic geometry to exempt students from their Algebra 2 EOC, which covers up to Trigonometry concepts? We hope the Senate Education committees will address this.

Since there are dramatic differences between the performance level associated with the same reported score, we feel SB1360 needs to define exactly which SAT exam (old or new) it is referring to. We also question why Florida would put into statute target scores from an exam that is yet to be administered (even if it does have the same name)? Are Florida students expected to field test the new SAT and then have those scores used for accountability purposes? Remember how well that worked out for the 2015 FSA?

Also, there are significant concerns regarding the math portions of the newly designed SAT, especially for low income and English language learner (more here). The new format of math questions will require higher level verbal and reasoning skills and is predicted to put English language learners and low income children at a significant disadvantage. Given the ever increasing population of low income, English language learner, and immigrant students in our public schools (Miami Dade is currently expecting ~8,000 new immigrant students this school year), why is Florida choosing an exam that would put those students, their schools and districts, at a distinct disadvantage. How is that a fair accountability assessment?

Our initial blog may have underestimated the “rigor” of SB1360’s target scores. If they represent scores from the new SAT, they may be more “rigorous” than we thought. Does this make us feel any better about this bill?

No.

Reviewing the new SAT only raises more questions about the fairness of an accountability system that uses these scores as metrics and in this manner. Students with high standardized test scores (even in subjects unrelated to the course they are taking) will be exempt from taking final exams/EOCs. Students with lower standardized test scores (many who will be immigrants, disadvantaged and/or english language learners), will not only be required to take the exams, but they will be worth 30% of their course grades and (for Algebra 1) passing will be required for graduation. “Smart kids” (often wealthier, white students) will no longer need to take the U.S. History or other state EOCs. They will be exempt from the Algebra 2 EOC based on scores that don’t test the course content; their course grades will reflect their classroom performance and will not suffer from poor performance on the EOC. It appears that students and schools with high test scores (like Seminole County, which has been lobbying hard for this bill, originally calling it the “Seminole Solution”) will require significantly less testing than their less advantaged counterparts.

This does not describe a fair, equitable, uniform education system. This describes the misuse of standardized test scores.

This will not “fix” anything.

This is Accountabaloney.

Algebra 2 EOC Results: Has Florida Jumped The Shark?

fonz_shark

In 1977, in an episode of the sitcom “Happy Days”, Fonzie displayed his bravery by jumping over a confined shark while on waterskis. Ever since then, the phrase “jumping the shark” has come to mean the moment in a TV series that defines the beginning of the show’s decline; a desperate attempt to retain viewership where it becomes obvious to the audience that the show has strayed irretrievable from its original formula. We believe the administration of the 2015 Algebra 2 FSA EOC (Florida State Assessment End of Course exam) will be remembered as the moment when Florida’s Education Accountability System “jumped the shark.”

Over the last week or so, the preliminary FSA EOC student results have been released to parents. Because the cut score process is still in progress, the reports show a percentile ranking for the child and the score breakdown of questions answered correctly in each “reporting category”. The full “horror” of the Algebra 2 EOC is being revealed.

Red flags were raised regarding the Algebra 2 EOC even before it was administered. Prior to the Algebra 2 EOC, every previous state-created EOC had been treated as a baseline administration during its inaugural year. Students would take the new EOCs in their inaugural year but performance on those test would not affect their course grade. The Algebra 2 EOC, however, from day one was planned to be worth 30% of the student’s course grade. Multiple emailed questions regarding this break from protocol went unanswered by the Florida Department of Education last spring.

Students recognized significant problems with the Algebra 2 EOC while it was being administered last spring. Here is a first person account from a 15 year old, gifted, straight A, honor student from Monroe County.

“I took the Algebra 2 EOC last year and it was absolutely ridiculous. I got my scores back and it said that I did better than 83% of the students that took it, but I’m not sure how they could even score such a horrible test. I walked out of the testing room and knew for sure that I failed it, and that they were not going to count it because of how flawed it was. The format that the test was on had many errors, and was filled with information that the teachers were not asked to teach. For example, the test would say “graph this piece wise function” and it would give you the wrong tools to graph it with, so even if you were totally capable of completing the question successfully, you couldn’t.”

We are certain that almost every Algebra 2 student will tell a similar story. The test was a disaster, bringing usually confident high school students to tears.

Before we discuss the results, understand this: Florida (along with the rest of “Common Core” America) has a “standards based education”. Teachers are asked to teach the standards and if students can demonstrate, on their state assessment, that they have learned enough of those standards, they will be deemed “proficient” and “pass” the exam. Florida State Assessments (like the previous versions, the FCAT) are “Criterion Referenced Tests” or CRTs. A CRT should compare a student’s performance to established expectations; if every student meets those expectations, all students would be deemed proficient. Likewise, if no students met the established criterion, all would fail. On CRTs, it is not only possible, but desirable, for every student to pass the test or earn a perfect score. (Remember that, it is important.)

A common example of a CRT is the written test for your driver’s license: if you answer 8 out of 10 questions correctly, you pass the test… even if thousands before you answered 9 or more correctly. (Learn more about Criterion Referenced Tests and their usage here)

The first problem parents should notice when reading their child’s Algebra 2 EOC report is that the score is reported as a percentile ranking, which is an inappropriate way to present results from a criterion based exam. Remember, if all students achieved the established expectations, ALL students would pass. In a criterion based exam, percentile ranking should be irrelevant.

Why would the Department of Education release the percentile ranking of students for a Criterion Referenced Test? Probably, in part, because they don’t expect parents to understand the difference and they are under time pressure to release some information regarding student performance, given that the exam was taken more than 6 months ago. The main reason, though, is that the established expectations for the new FSA have not yet been established and won’t be established until the January 4th State Board of Education meeting (more on that later).

So, despite having NO established performance standards, the Algebra 2 results were released to parents, highlighting a student’s percentile ranking in the state and their performance in individual “Reporting Categories.” Here is what we have learned from reports parents have shared with us:

ALGEBRA 2 2015 FSA EOC results

#correct (out of 56) Total %correct Student percentile ranking
16 25% 63 %tile
18 27% 67%tile
20 36% 71%tile
29 52% 88%tile
43 77% 99%tile

 

Keep in mind that those who took Algebra 2 are essentially all college bound, often honors and/or gifted math students, and this exam is, by statute, worth 30% of their course grade (the 30% was waived after test administration last spring but, by report, will be back in effect next spring). The State’s most gifted math students (99th percentile) still missed one out of 4 questions and students will have passed (if the Commissioner of Education’s current cut score recommendation is approved) missing ¾ of the questions. Something definitely seems amiss.

Remember: “On criterion-referenced tests, it is not only possible, but desirable, for every student to pass the test or earn a perfect score”. So why has Florida created a CRT where its most gifted math student are unable to answer 25% of the test questions correctly?

Of course, the cut scores/expected performance levels have not yet been set for the Algebra 2 exam, so these student reports, actually, say nothing as to whether a student passed or not.

On September 15-17, 2015, at a Rule Development Workshop, the general public got a first look at the impact of potential cut scores. It wasn’t pretty. You can see the complete presentation here.

FullSizeRender-10

To summarize, the Educator Panel recommended cut scores than would fail 69% of students, including 85% of African American students. Since then, the Commissioner of Education has made her recommendations which would (only) fail 64% of students. The ultimate cut scores will be determined by the Florida Board of Education (BOE) on January 4, 2016. Several members of the board have indicated they wanted cut scores that were even higher than the Commissioner’s, failing even more students. Sadly, the process of establishing cut scores seems to reflect political ideology more than sound education judgement. (Learn about the cut score process here)

Prior to the release of the Algebra 2 test scores, on October 28th, I spoke in front of the BOE, and described why I felt “the most egregious example of Accountabaloney is the state’s unwavering assertion that the Algebra 2 EOC is valid.” You can watch my speech here. In summary:

  • In 2015, Algebra 2 had a new set of standards, which now included Pre-calculus and Statistics (learn about the standards here).
  • The commissioner has admitted that no one knows whether it is even possible to teach these standards in a single school year (details here).
  • The Algebra 2 EOC could not have been field tested because Utah does not test Algebra 2 or Trigonometry standards.
  • The Alpine Validity Study completely ignored Algebra 2 in the interest of time (Report here).
  • Honor students reported being unable to answer any questions on day two of this EOC, so it should have been no surprise that the exam’s results were total outliers with almost half of students failing including 85% of African American students. (Report of results here)
  • At the Keep Florida Learning Committee, the department suggested the problem was “the teachers didn’t teach the standards”. (more info here)
  • Crushing the confidence and destroying the GPA of our best and brightest students does NOT make them college and career ready. Blaming it on their teachers does NOT help recruit and retain high quality STEM educators.

I concluded with “the Algebra 2 exam and its standards deserve a complete review.” Seriously, parents shouldn’t have to ask. In the face of falling national math scores (read here), the entire math sequence deserves a review.

Currently, parents and teachers report concern regarding both the pace and scope of the new Algebra 1 and Algebra 2 standards. When Florida switched to the Common Core-like Florida Standards in 2014, much of what used to be taught in Algebra 1 was moved to Pre-Algebra. Algebra 1 now includes statistics and at least a third of the previous Algebra 2 content. Algebra 2 now includes pre-calculus and statistics. The Florida Standards are now out of alignment with the new CCSS-based textbooks. If a teacher teaches from the new, state approved Algebra 1 or 2 textbooks, they will not cover the required standards. This is a problem. Why would the State approve textbooks that do not cover the required course content?

In addition, the amount of content seems to be too much to complete in a single year, especially since the EOCs are given in late April/early May. One Algebra 2 teacher said:

“The problem is that there are TOO MANY standards in the test item specs. It’s simply TOO MUCH to teach, prior to the Algebra 2 EOC which starts in early April to the first week in May. We have to teach at the speed of light to get everything in, and it’s nearly impossible to do so.”

Parents are concerned that the pace does not allow their kids to fully understand concepts and many kids are dropping out of subsequent advanced math courses. Another math teacher wrote:

“Algebra 1 now looks like Algebra 2 and Algebra 2 now looks like Precalculus/Statistics. Regular kids who are not strong in math are drowning. It’s just so developmentally inappropriate. These new standards and new EOC’s are geared towards honors students who excel in math.”

Clearly, when Florida created its new “rigorous” Math standards, moving a significant amount of Algebra 2 content into the Algebra1 course, it created problems beyond the EOCs. By statute, passing the Algebra 1 EOC is a graduation requirement so Florida now requires a mastery of the higher level Algebra 2 content to graduate high school. Was this the legislature’s intention? Also, in the current A-F grading system, middle schools are being graded based on their number of advanced math students (Algebra 1 and Geometry) encouraging increasing student placement into Algebra 1 (Learn about how School Grades are calculated here). Now these very young math students are being asked to master Algebra 2 standards; is this developmentally appropriate? Also, the middle school math teaching credential is for grades 5-9 and DOES NOT include Algebra 2 content. Now that Algebra 2 standards are placed in Algebra 1 courses, most middle school math teachers are not credentialed to teach the content. This is a serious problem.

The current pace, scope and assessment of Math standards has been significantly disrupted by #accountabaloney. We believe re-evaluation of the entire advanced math sequence and math FSA EOCs is warranted.

Here are the concerns that must be addressed:

  1. The overwhelming evidence suggests there is CLEARLY something wrong with this Algebra 2 exam, yet the DOE is releasing test scores, cut score determinations are underway and the test remains on the Spring 2016 schedule. This defies reason. Is the DOE addressing this? Why would they release the scores from such a disastrous test administration? Why even require a state mandated EOC for a non-required class?
  1. Has either the Algebra 1 or 2 EOC been definitively shown to be fair, reliable and valid for special populations including special education (ESE), English Language Learners (ELL), low income or racial subgroups? Where are those reports? 85% of African Americans failed the Algebra 2 EOC… 85% of the best and brightest African American students failed this test… Something is wrong. Also, since passing the Algebra 1 EOC is a graduation requirement, the Algebra 1 EOC definitely needs to be evaluated for fairness for ESE, ELL, etc, students. Where is the evidence this has been done on these new EOCs? (Please don’t accept the answer “the Alpine Validity Study”; it did NOT address these issues.)
  1. Is there any evidence that an accelerated pace through advanced math concepts is a good idea? Why have Algebra 2 standards in an Algebra class or pre-calculus standards on Algebra 2?  Should Algebra 2 mastery be required for high school graduation. Are middle school teachers properly credentialed to teach the current Florida Algebra 1 standards?
  1. We have significant concerns regarding the entire accountability system, extending beyond the math EOCs, at this point. Clearly the “establishing cut score” process has as much to do with political ideology as it does educationally sound practices. We worry that, with the new tests and the new more “rigorous” cut scores, the initial intent of many of the State’s legislated mandates are no longer aligned to the current assessments. For example, mandatory third grade retention is meant to identify and provide interventions for students reading below grade level; is there any evidence that the current cut scores assess grade level proficiency? Will 3rd grade students reading at or above grade level be marked for retention? Likewise, passing Algebra 1 was a challenging requirement for high school graduation but now it appears the EOC is testing a much higher, more challenging Algebra 2 curriculum. Is that what the original legislation intended? We suspect not.

CALL TO ACTION:

We encourage parents and teachers to share this blog with Governor Rick Scott, Commissioner Pam Stewart, the Florida Board of Education and your legislators. Tell them the Algebra 2 EOC is the most egregious example of the problems with the current accountability system. Ask them how parents are expected to have confidence in a system that creates final exams that are so abusive in nature and then seemingly ignores the obvious issues. If the DOE is not evaluating the math standards and assessments, demand they do so. If they are investigating these issues, remind them it would be wise to inform teachers and parents before we lose all confidence and respect.  We need a DOE that can recognize flaws in the current system and work to fix them. Florida needs to re-evaluate both the advanced math sequence and math FSA EOCs.

With the creation and administration of the Algebra 2 EOC, we believe Florida’s Education Accountability has “jumped the shark.” Without dramatic policy reversals, we fear parent confidence in the system will be irretrievable.