What Is the Goal of the Algebra 1 EOC?

It’s that time of the year: all across Florida, the children are crying, stressed and overwhelmed. Why? It is Algebra 1 EOC time.

Since 2010, Florida’s middle and high school students have been administered Algebra 1 End of Course (EOC) exams. We have argued that the Algebra 1 EOC has the highest stakes of all: not only does it count as 30% of a student’s course grade, it is a high school graduation requirement, a requirement for admission to many magnet schools, calculated into school and district grades, etc, but, this year, schools will be fined 1/6 of a student’s per pupil funding (about $1,000) if the child does not pass their EOC. Those are high stakes!

How do students react to the stress? This is what parents have to say:

  • “A 7th grader (earning high school credit) took the Algebra 1 EOC/FSA and didn’t finish the test. She is stressed and terrified of receiving a failing grade (she has always been an outstanding student). She said there was material on the test her teacher never covered and she is concerned because it makes up 30% of her grade and could lower her future high school GPA.”
  • “My 8th grader is set to take her honors Algebra I EOCs tomorrow and Wednesday. She’s so stressed and scared she was bursting into tears all weekend.”
  • “My 8th grader is taking Alg 1 tomorrow and Wednesday. The kids are all afraid to fail the test — Crying, stressed and overwhelmed. The prep work prior to the test was brutal. Sent home with a 40 pg packet, then a 12 page packet trying to help prepare them. This on top of other FL standards tests and EOC. The teacher told them they were all going to fail.” (emphasis mine)

After students take the test, they tell us this:

  • “The questions on the EOC were confusing. I left the test feeling tired, stressed out, and not confident that I did well on it at all.”
  • “This is a test of “gotcha”-type questions. There are a mishmash of questions and they are often more of a function of how to use a computer than how to do math.”
  • “My son took his Algebra 1 EOC yesterday and is feeling the same way. He too has had an A all year and said that he knew nothing on the test. This is his first year in public school and he was so upset. He said that he felt like he was set up for failure and by his teacher. How sad that an 8th grader who works so hard at school now feels this way.”

So, we have a ridiculously high stakes test, students are stressed and overwhelmed, and they describe the questions as confusing and a test of computer skills more than math knowledge. Some children are blaming their teachers, feeling they have been set up for failure. Interestingly, the Florida Department of Education (FLDOE) has responded similarly. Commissioner Pam Stewart said (during the Keep Florida Learning Committee) that if anything on the test doesn’t match what’s being taught, it’s the fault of the teachers who just don’t understand what they’re being asked to do. Is that true? Is the test is fine and the teachers are the problem? What if the problem is not the teachers, but, rather, the test or the standards? During the Keep Florida Learning Committee meetings, Alachua County Superintendent, Dr. Owen Roberts, repeatedly questioned the validity of the instructional standards. While they are busy placing blame on the teachers, what is the FLDOE doing to assure the validity of the EOC, its associated standards and its many high stakes uses? Is the Algebra 1 EOC designed to confirm student proficiency or teacher failure?

2015 was the inaugural administration of the new Florida Standard’s based Algebra 1 EOC, created by American Institutes of Research (AIR).  As we have described previously (read here), the AIR tests were never field tested and they lack completed validity studies. During the summer of 2015, Alpine Testing Solutions’ “Independent” Validity Study did little more than confirm Algebra 1 test questions were aligned to the standards, with no mention of fairness, reliability or validity for at risk sub populations of students; hardly (in our opinion) an appropriate evaluation for a test with such high stakes.

The 2015 FSA Technical Report reveals some interesting finding regarding the Algebra 1 EOC.

On page 4 of Volume 2 of the 2015 FSA Tech report, the number of field tested items on each assessment. Students should know that 10 of the 68 are field test items and will not be counted for or against them.

13164224_1316906234991721_271044146925004399_n

 

This is also from the 2015 FSA technical report and shows the number of technology enhanced items that had to be replaced when writing the paper/pencil version of the test.

13165928_1316897624992582_1324943782525090398_n

Nearly a third of the Algebra 1 test questions are technology enhanced, requiring specific computer skills to answer them correctly. This is important because my district (Monroe) informed the school board that students performed worse on the technology enhanced items than the standard items. This brings up the question “is this test assessing math skills or computer skills?” Are students who do not have computers at home at a disadvantage on this graduation requirement? Neither the technical report nor the Validity Study addressed this. We believe this should be addressed.

There is no report comparing student performance on the FSA Algebra 1 EOC to any established, normed Algebra 1 assessment. This would be vital to assuring the FSA EOC tests what it purports to (an essential ingredient of validity).

This is the most alarming thing we discovered:  The Technical Study confirmed the EOC is ridiculously hard, yet it continues to be administered with the hope that students will, someday, acquire the skill needed to pass it. In the meantime, student self-confidence, emotional well being and GPAs (not to mention school finances) will just have to suffer.  Here is what we found:

The Algebra 1 (and 2) EOC was shown to be more difficult for the test population, with an atypical standard error curve.

From Volume 4 FSA Technical Report:

FSAV4Figure 3 SEM

 

 

Figure 3 show, for grades 3-7, the SEM curves show the expected “U” shape.

 

 

FSAV4Figure4SEMEOCs

 

 

Figure 4 shows the right side of the “U” has flattened out for Algebra 1 and Algebra 2 (and grade 8 above), indicating these exams are more “challenging” for the tested population.

 

From Page 18, Volume 4:

“For most tests, the standard error curves follow the typical expected trends with more test information regarding scores observed near the middle of the score scale. However, there are two general exceptions. In grade 8 Mathematics and for both Algebra EOC tests, the test is maximized at a higher point along the ability scale. This suggests the items comprising these tests are somewhat challenging relative to the tested population for this initial test administration. Because the testing of Algebra 1 is relatively new and the statewide testing of Algebra 2 is entirely new to these populations, this atypical curve is not unexpected. As students continue to learn these required skills, it is probable that this SEM curve will shift to reflect the expected, normally distributed SEM curve over time.”

What does this mean in English? We are not statisticians but we think this tells us that the 2015 FSA 8th grade Math and BOTH Algebra EOCs DID NOT fit the expected SEM curves, demonstrating the items on these tests are more challenging for the tested population. Is this the result of students not knowing the required skills or is this a problem with test design? No one knows because there was no comparison to an established, valid assessment. No one knows because there was no evaluation as to whether the test assesses math skills or computer skills.

The SEM results should surprise no one. The “curve” on the algebra test has always been huge, with students passing while answering less than 50% of the questions correctly. If you add technology enhanced “gotcha” questions to the previous EOC, you should expect even worse performance. The question is whether it is appropriate, or necessary, to test Algebra 1 students this way. If the DOE believes students will adjust to the new exams in time, should the high stakes be waived while allowing students to catch up on the required skills? Is there a deficit in the earlier (Common Core) math standards that contributes to the poor performance on these exams? Has anyone even looked? We hope these questions are being asked.

We believe, in a standards based system, tests could be written to assess essential Algebra knowledge without making students cry.

What is the goal here? Students can be challenged without being crushed. Exams can be rigorous without being punitive. A high school graduation requirement should not be full of “gotcha” questions. Accelerated middle school math students should feel proud of their accomplishments, not demoralized. No test scores are that important.

Is the goal of this EOC to assure student proficiency or to blame teachers for their inadequacies? Is this exam an educational tool or a political weapon?

Destroying children’s confidence should concern the Department of Education.  It is not enough to wait and see if students acquired new skills in the future. There are children taking these tests now. They are stressed and crying now. Waiting a year or two will only assure next year’s students are equally tortured.

We urge you to contact your Florida legislators and demand that these exams, and the high stakes attached to them, undergo review. Creating a test that demoralizes even the most academically talented students and then blaming their teachers for the outcome is accountabaloney.

Just prior to publishing, this was reported in the Tampa Bay Times: “Florida Department of Education leaders will be setting up an online forum this summer to collect ideas about how to revise the state’s educational accountability system.” They will have their work cut out for them because there is A LOT in need of revision. The problems with the Algebra 1 EOC are just the tip of a very large iceberg.

Advertisements

PROJECT: Trying to Find Y

Trying to Find Out Y_FACEBOOKParents, Teachers, Students:

We need your help!

We have significant concerns regarding Florida’s current advanced math sequence and the associated EOCs.  We have written several blogs expressing some of our concerns (Algebra 1, Math Problems, Algebra 2).  We even travelled to Tallahassee for a face to face meeting with the Department of Education in January.  At that time, DOE officials felt certain that the low scores on last year’s Algebra 2 EOC were the result of teachers being unfamiliar with the new standards and that student scores would improve this year.  We were less confident.

As students start to take the Math EOCs this year, we are not seeing much reported improvement over last year.  Indeed, there seems to be more concerns regarding the content of the Algebra 1 EOC. Students are, again, feeling crushed and demoralized. We feel something needs to be done.

We are concerned with the appropriateness of the standards.  We hear from teachers that the students, immersed in our test focused, Common Core classrooms, don’t have the firm grasp of basic skills needed to be successful in higher math courses. We hear stories that it may not even be possible to teach the course content in the time allowed.  We have seen the pacing guides that include the teaching of new material in late May, well past the EOC administration. We don’t understand why a state assessment is even needed for courses that are not graduation requirements, like Geometry and Algebra 2. We hear stories of these EOCs destroying students’ GPAs and confidence. We believe 30% of the course grade is much too high.

If you are a parent or a student, please share your stories with us. (If you are a teacher, please read to the end of this post. There are questions specifically for you, there.) We strongly encourage everyone to, ALSO,  share your story with your local school board, your state representative (who votes on the legislation that mandates these exams), the Commissioner of Education and the State Board of Education (who set these policies). They need to understand what a “math mess” there is in Florida. We have included email addresses below.  The overriding question we would love students and parents to address is this:

  • The Math FSA EOCs are required for all students enrolled in Algebra 1, Geometry and Algebra 2. They are mandated by state law to be worth 30% of the course grade.  What do you want the legislators, who mandated these exams, to know about them?

In addition, consider discussing the following:

Parents:

  • How has preparing for the Math FSA EOCs affected your child?
  • Do you feel your child was well prepared?
  • What factors contributed to your child’s state of preparedness.
  • What was your child’s experience with the EOC?
  • Did you child require a one-on-one tutor in order for your child to stay on track?

Students:

  • Please describe your experience preparing for and taking the Math FSA EOC.
  • Do you feel your teacher prepare you well?
  • Were there items on the test that were not covered in class?
  • Were test questions easy to understand or confusing?
  • How did you feel when you left the test?
  • When do you expect to learn your score?
  • Do you worry about the impact of this exam on your overall GPA?
  • For some students, entrance into magnet programs rely on passing certain EOCs.  Were you in this predicament?  How did you feel knowing that you had extra pressure to do well on this EOC test in order to gain entrance into your selected high school?

Where to send your stories:

Please send your stories to Accountabaloney by posting, or private messaging us, via our Facebook page, Accountabaloney , or emailing us directly at accountabaloney@yahoo.com. Please, please, please send your stories to your state legislators, the Commissioner and the Board of Education. Email lists are below.  It is imperative that they hear these stories and understand the impact of their legislation and policies.

Education Commissioner, Pam Stewart – Commissioner@fldoe.org

State Board of Education Members:

Chair, Marva Johnson – Marva.Johnson@fldoe.org
Vice Chair, John Padget – John.Paget@fldoe.org
Tom Grady – Tom.Grady@fldoe.org
Rebecca Fishman Lipsey – Rebecca.Lipsey@fldoe.org
Gary Chartrand – Gary.Chartrand@fldoe.org
Andy Tuck – Andy.Tuck@fldoe.org
Michael Olenick – Michael.Olenick@fldoe.org

Follow this link to determine your State Senator and Representative: https://www.flsenate.gov/Senators/Find

If you are a Math Teacher in Florida, we need your expertise.  Please share your experiences/concerns with us.

  • Do you feel that your pacing guides properly paced the curriculum?
    What percentage of time did you dedicate to test prep verses actually teaching?
  • Did you hold review sessions outside of the regularly scheduled class time?
  • Did your students have access to Math Nation/Algebra Nation?
  • Do you feel like the books you are using properly aligned to the test content?
  • Did you feel like you have the proper time to properly truly teach the standards in the time allotted?
  • Did the FSA testing season changes of schedule impede your ability to properly prepare your kids?
  • What were some of the greatest challenges you faced this year?
  • Do you feel it is fair to have the EOCs count for 30% of the student’s course grade?
  • Have Math teachers in your area been discussing this?  How do you believe teachers can most effectively impact the current situation?
  • Would you be interested in participating in a focus group to address these issues?

Please comment or private message us via our Facebook page or email us at accountabaloney@yahoo.com OR fill out the form below.  We strive to keep teacher involvement confidential.

Thank you for contributing to our project: Trying to Find Y.

Stay tuned, here, for updates.

 

Student Achievement and FSA Validity: More Accountabaloney

A test is only valid if the results are used in a valid way. If a test’s results are used in anyway other than their intended use, the validity of that use must, also, be confirmed. This, essentially, sums up the lack of validity of the entire Florida Education Accountability system: tests that may have once been be valid for one use are now being used for a myriad of uses they were never intended for. Our children are sitting for tests whose results are used to determine promotion and graduation eligibility, rank and punish teachers, evaluate schools and districts and, because of school grades, determine local property values. Are these valid uses of the FSA? Has anyone bothered to check?

Following significant concerns regarding the validity of the new Florida State Assessment (FSA), legislators, during the 2015 session, ordered an independent verification of the new state assessment. Last summer, Alpine Testing Solutions, in partnership with EdCount, completed a partial, so called “independent”, verification of the psychometric validity FSA.  (You can read the report here, and our comments on it’s incomplete nature here). In the report, Alpine describes the relationship of validity to the intended use of assessment results.

“The process of evaluating an assessment and its associated validity evidence is directly related to the intended uses of the scores. Validity refers to these specific uses rather than a global determination of validity for an assessment program. As such, it is possible that the validity evidence supports one specific use of scores from an assessment while is insufficient for another.”

For example, a spelling test might be a valid way of measuring a student’s spelling abilities.  That same spelling test would tell you nothing about the child’s math skills, knowledge of U.S. History or ability to drive a car. Likewise, a student’s driver’s test is an insufficient measurement of that student’s spelling skills.

Table 2 (page 27 of the final Alpine report) provides a summary of these intended uses of the FSA (as provided to Alpine by the FLDOE) and notes the uses for which modifications have been made for 2014-15 as the first year of the program.

12821626_1260551510625721_8063714972586503064_n

The stated intended uses of the FSA include: teacher evaluation, school and district grades, school improvement rating and state accountability.  The ONLY listed uses for students are 3rd grade promotion, graduation eligibility (for both grade ELA and Algebra EOC) and student grades (for the math EOCs which are worth 30% of the students course grade). Please note: nowhere in Table 2 is “provide student academic achievement and learning gains data to students, parents, teachers, or schools” mentioned, yet this is reported to be the primary purpose of the student assessment program. In fact, according to the chart, there are NO intended uses for individual students, at all, for grade level FSA testing from grade 4 to grade 9.

I want to repeat that: according to the Alpine Validity Study, there are NO intended uses for individual students, at all, for grade level FSA testing from grade 4 to grade 9!

So, not only did the Alpine Study fail to evaluate 11 of the 17 new FSA assessments and fail to assess whether any of the tests were fair, valid or reliable for vulnerable sub-populations (read more here), but, according to Table 2, it EVEN failed to evaluate  whether the results could be used for the (reported) primary purpose of the state assessment system: evaluating student achievement.

It bears repeating that the 2014 Florida Statewide Assessments (FCAT 2.0) Technical Report (released in 12/2014), on page 137, suggested that further studies were needed to verify the inference “that the state’s accountability program is making a positive impact on student proficiency and school accountability without causing unintended negative consequences.” (Emphasis mine.) It will be difficult, I think, to evaluate that if the validity of the use of individual student scores is never determined.

When school administrators tell you the FSA is about providing information about student achievement and learning gains, show them this.  Providing information about student achievement and learning gains does not appear to be a priority of the current system; ranking and punishing teachers and schools does. Ask the “powers that be” when is Florida’s Accountability system going to take a serious look at the appropriate use and the current misuse of test scores?  When will they verify whether the current system is making a positive impact on student proficiency (dropping ACT and NAEP scores suggest otherwise)?  When will the myriad of unintended consequences be addressed? Until then, this is not a valid accountability system; it is all a bunch of accountabaloney.