2016 FSA Administration: Ready or Not, Here it Comes

FSA Picture_thumb

On February 25, 2016, my son’s 3rd grade Florida State Assessment (FSA) score report was delivered to his school, (ironically) by express delivery, and I was finally given a copy demonstrating his percentile performance in last year’s assessment.  For some reason, never explained, his scores were late reported by the state. He is scheduled to take the 4th grade FSA Writing component on March 1st (5 days after we received last year’s results). True story.

Last Spring, the inaugural administration of the Florida State Assessment (FSA) was marred by computer glitches, cyber attack and validity questions.  Round Two begins on February 29, 2016. Has anything changed? Is Florida ready?

Last Spring: Frozen Screens. Service denials. Cyber attacks. Server Crashes. Log-in failures. This year: Low Expectations.

The computer-based FSA had problems from the start. “From the get-go, the FSA test was riddled with technical problems, including a cyber attack from an unknown source and a slow-moving AIR server which left thousands of students with error messages in their language arts and mathematics tests.” (read about it here and here)

The Florida Department of Education did launch an investigation into the server attack that, presumably, had left thousands of students across the state unable to take the new FSA, but 6 months later, the Department still had no answers to who was behind the server attack or why it took place (details here). Is that investigation still ongoing? Are adequate safeguards in place?

In September, State Education Commissioner, Pam Stewart, announced that she would seek liquidated damages from American Institutes for Research, or AIR, the creator of the FSA (damages which are clearly delineated in the State’s executed contract and collection of such is required, “if applicable”, by Florida State Statute). Nevada has successfully recovered $1.3 million in damages from AIR for similar testing disruptions there (details here). Recently, an attorney from the FLDOE refused to comment on the status of this, stating it was “pending litigation,” but we can find no public record of any pending litigation. Is the State still pursuing this?

This year, expectations for FSA computer-based testing are being set low, as reported here.

“The Florida Department of Education, its testing vendor American Institutes for Research, along with districts and schools, have taken several steps to prevent such troubles. Those include expanding bandwidth, upgrading defenses against outside attacks and improving testing software.

Even with such moves, though, the department warned that students still might encounter interruptions beyond their control. And that, said FairTest public education director Bob Schaeffer, could hurt some children.

Imagine the impact when the screen goes blank for a seventh-grader taking a civics test required to get out of middle school, Schaeffer said. “For an emotional adolescent to experience that, it’s a scary situation.”

Yet there’s almost no way to guarantee trouble-free computerized testing on a stage as large as Florida’s, experts said.

So… expect glitches.  Lots of them.

Earlier this month, students across Florida, when asked to participate in district “infrastructure testing,” reported wide-spread issues, in multiple counties, with the system: screens freezing, difficulty loading, bugs in the program, unexpected shut downs.  It appears there is still “work to be done.” Similar situations in Tennessee, led that state to “pull the plug” on this year’s computer testing (read about it here). What would it take for Florida to pull the plug and return to paper and pencil tests, as recommended by many of Florida’s superintendents?

There are more reasons than just technical issues to question continuing with computer testing. Reports (here and here) from Illinois, New Jersey and Maryland have shown students who took the 2014-15 PARCC exams (similar to Florida’s FSA) via computer tended to score lower than those who took the exams with paper and pencil—a discovery that prompts questions about the validity of the test results and poses potentially big problems for state and district leaders. Why evaluate student proficiency with an assessment that under-represents their abilities?

Were similar results found with the Florida assessments?  Did students perform better on paper tests than computer tests? Is the comparison even being made?  No one knows because the 2015 FSA Technical Report, due out in January, has yet to be released.

Technical Report not complete

The 2015 FSA Technical Report, usually published by January following a test’s administration, is still (per personal communication with the FLDOE on 2/25/16)  “undergoing final reviews and will be available shortly.” In the past, FCAT Technical Reports (past examples can be found here) focused on test validity and reliability, “key concerns for establishing the quality of an achievement test such as the FCAT.” Also in the past psychometric analysis was the major focus of these reports. Without a completed report, is Florida proceeding with this year’s FSA administration in the absence of proof of psychometric validity?

Interestingly, the 2014 Florida Statewide Assessments (FCAT 2.0) Technical Report (released in 12/2014), on page 137, suggested that further studies were needed to verify some implication arguments. “This is especially true,” it read, “for the inference that the state’s accountability program is making a positive impact on student proficiency and school accountability without causing unintended negative consequences.” (Emphasis mine.) I am, especially, eager to learn whether Florida completed these “further studies” because it appears there are LOTS of unintended consequences in the current accountabaloney system.

An Incomplete and Not Independent Validity Study

Questions regarding the validity of the FSA began to be asked last March, when Commissioner Stewart testified before a Senate Education Committee that the FSA had been validated in Utah and she promised to provide the senators with those reports.  No such reports were ever delivered.  This prompted legislators to pass a bill requiring the department to hire an independent company to verify whether the FSA was a valid tool to assess the academic achievement of Florida’s students.

Let’s take a little pause, here, to review how to determine whether a test is valid or reliable.  This article outlines the process (you can substitute “FSA” for “SBAC” and the article would still, mostly, hold true):

“With real scientific educational research, a group of independent researchers, not bought off by any billionaire, would select a large group of representative students, such one thousand 8th graders randomly chosen from an urban school district. These students would be randomly assigned to an experimental group and a control group. A survey would be given to each group to make sure that the groups were matched on important characteristics such as free and reduced lunch status and ELL status. Each group would be given a series of tasks such as completing the 8th grade NAEP Math test, the 8th grade SBAC Math test, the 8th Grade MAP test, the 8th Grade MSP test and/or the 8th Grade Iowa Test of Basic Skills. These would then be compared against comparison measures such as teacher grades in their previous and current year math courses. The actual test questions of every test would be published along with student scores for each test question on each test. Objective analysis and conclusions could then be made about the reliability and validity of various measures using these carefully constructed norming studies. This results and conclusions would be peer reviewed and quite often the entire study could and would be replicated by other independent researchers at other major Universities. “

Suffice it to say, Florida “independent” validity study didn’t do that.

Partial, hardly independent, FSA Validity Study

Florida hired Alpine Testing Solutions to perform the mandated validity study.  Alpine partnered with EdCount, a partner of AIR, the test creator.  The project team contained previous AIR employees.  So much for the independent part…

The Alpine Report was presented to the FLDOE on August 31, 2015, after allowing the DOE to review and make suggestions regarding two earlier drafts of the report (more here). The final report was released to the public on September 1st. The FLDOE announced the study showed the FSA to be “valid,” a claim that was challenged by educators (more here and here). The report recommended against using the results from the computer-based assessments as the sole factor in determining individual consequences, such as whether students should be promoted, retained or remediated, but felt those same scores COULD be used to evaluate teachers and schools. Not surprising, many wondered how a test, not found to be accurate in measuring student achievement, could be used to rank teachers, schools and districts.

While many debated the contents of the Alpine report, what was NOT in the report was equally interesting.  Alpine was charged with reviewing the grade 3-10 English Language Arts (ELA) exams, the grades 3-8 Math assessments and the Algebra 1, Algebra 2 and Geometry End of Course (EOC) exams. Because of time constraints, however, the report ONLY evaluated ELA exams for grades 3, 6, and 10, Math exams for grades 4 and 7 and the Algebra 1 EOC. This leaves 11 of the 17 new FSA exams (ELA grades 4, 5, 8, and 9, Math grades 3, 5, 6, and 8, and Algebra 2 and Geometry EOCs) UNEVALUATED and possibly invalid. When will those tests be evaluated? Before the next FSA administration? That seems unlikely, since the next administration begins on Monday (2/29/16).

Also missing from the Alpine Report is any evaluation regarding whether the FSA is valid, fair or reliable for vulnerable populations, such as special needs students, english language learners and other at-risk populations, as outlined by Dr. Gary Thompson here.

“…Due to the limited time frame for developing the FSA, item reviews related to content, cognitive complexity, bias/sensitivity, etc. were not conducted by Florida stakeholders.” (Alpine Testing Solution, Inc. Validity Report P.35)

Per Dr. Thompson, “Neither Utah nor Florida has produced validity documents suggesting that either the SAGE or FSA high stakes academic achievement tests can validly measure achievement in vulnerable student populations, or that the current testing accommodations allowed or banned, are appropriate or fair.”

Without determining fairness for vulnerable sub-populations of students, continuing to use FSA scores to retain, remediate or prevent students from graduating is unconscionable. How can an accountability system be based on an invalid or unfair measurement?

So, in summary:

  • Expect computer glitches. Keep your FSA expectations LOW.
  • Don’t hold your breath for any liquidated damages from last year’s FSA fiasco, despite being spelled out in the AIR contract and mandated by statute.
  • State assessments, in general, are not appropriately validated. Will Florida ever evaluate whether, like the PARCC assessment,  FSA scores are worse on computer then pencil and paper?
  • The validity of the FSA is, at best, incomplete; the Alpine Study failed to evaluate 11 of 17 new assessments.
  • There are no documents suggesting the FSA can validly or fairly measure achievement in vulnerable student populations, yet those students suffer the same consequences as their more advantaged classmates.
  • Don’t expect the FSA Technical Report to address the unintended negative consequences of the current system.
  • My son’s test scores, delivered just 5 days before this year’s test, will not inform his instruction.
  • The Accountabaloney will continue until further notice.

Or, in other words, “Same song, second verse, could get better but it just gets worse.”

FSA testing starts Monday, whether we are ready or not.

 

 

 

 

CALL TO ACTION: Stop the Poorly Designed CBE Pilot (SB1714)

CALL TO ACTION. IMMEDIATE ATTENTION REQUIRED

This Thursday, February 25, 2016, the Florida Senate Appropriations Committee will hear SB/CS1714, the Competency Based Education (CBE) Pilot Study Bill, in its last stop before it hits the Senate Floor.  The companion bill in the House has already been passed.  We have joined a state-wide, bipartisan coalition of grassroots organizations in a Call to Action attempting to stop the passage of this bad bill, which allows the advancement of CBE in four Counties in Florida, with the Commissioner’s ability to spread the program to even more counties, even before any serious public discussion regarding CBE has taken place. (Learn more about CBE here)

At Accountabaloney, we believe this is a poorly designed pilot program and should not be initiated in Florida. Participating districts, and the schools within them, are unlimited in how they can initiate CBE within their systems. By allowing vague descriptions and definitions of the possible CBE programs in each participating county, with few well defined parameters, comparing program effectiveness and attributing that to CBE will be virtually impossible. As currently defined, the results measured will assess little more than test taking skills; hardly the definition of a quality education system. The rapid expansion within and between counties, currently allowed by these bills, could affect millions of Florida’s public school students before the pilot programs’ results have even been tallied. This is NOT the way legitimate pilot studies are designed. Florida’s students are not served by participating as guinea pigs in poorly designed studies.

Florida’s Education Accountability system is still reeling from the rush to implement the Florida Standards and the FSA. Can it withstand another major rushed implementation?

We believe it cannot.

We urge you to join thousands of Floridians and participate in this Call to Action . Please, contact the members of the Senate Appropriations Committee TODAY (their addresses are in the attached blog) . You must take action NOW.

Because of the immediate need for action, consider sending the following to the entire Senate Education Appropriations Committee (benacquisto.lizbeth@flsenate.gov, lee.tom.web@flsenate.gov , flores.anitere.web@flsenate.gov, altman.thad.web@flsenate.gov, gaetz.don.web@flsenate.gov, galvano.bill.web@flsenate.gov, garcia.rene.web@flsenate.gov, grimsley.denise.web@flsenate.gov, hays.alan.web@flsenate.gov, hukill.dorothy.web@flsenate.gov, joyner.arthenia.web@flsenate.gov, latvala.jack.web@flsenate.gov, margolis.gwen.web@flsenate.gov, montford.bill.web@flsenate.gov, negron.joe.web@flsenate.gov, richter.garrett.web@flsenate.gov, ring.jeremy.web@flsenate.gov, smith.chris.web@flsenate.gov):

Vote NO to SB1714! Competency Based Education will be full of daily data collection, and will put our children’s personally identifiable information at risk. I want my children educated by teachers; not data mined and taught by computers.

The pilot study defined in SB1714 is poorly designed and allows the rapid expansion of CBE programs even before data has been collected.  By allowing vague descriptions of CBE programming and measuring little more than test taking skills, the data obtained from these pilot studies will be virtually unusable in the evaluation of quality education programming.

Florida does not need another rushed implementation of a poorly vetted plan to dramatically overhaul our education system.

Vote NO on 1714.

Thank you,

(Name, address, Phone Number)

If you live in Lake, Pinellas, Seminole or Palm Beach counties, the passage of this bill will bring CBE to your schools almost immediately.  Please contact your local representatives, as well as your school board members, and tell them to keep CBE, and it’s accompanying data mining and test prep,  out of your schools.

Call to Action: Competency Based Education

Senate Bill 1714 and its companion House Bill 1365 will allow changes to state education law to put in place competency based education programs.  Other names for competency-based learning [CBE], include performance-based, proficiency-based, and mastery-based education, as well as “customized” and “personalized” learning.  Definitions of CBE are called “a work in progress,” and include life-long data and socioemotional profiling, a significant emphasis on lower level job skills, and a focus on much more subjective psychosocial, non-academic attributes and skills.

While there may have been positive examples of CBE used in classrooms in the past, the bills that are currently being pushed through the Florida legislature completely change the definition of CBE that most educators are familiar with. The technology component will redefine the role of the teacher from educator to facilitator and significantly minimize human interaction in the learning process. We believe students learn best from face-to-face human interaction.

In addition, CBE will allow a massive increase in the amount of personal student data collected and shared both in our public schools and with the Department of Labor. Currently, education is the 4th largest industry and the most data mined industry in America.  The amount of data collected from students and shared by CBE vendor programs (including very subjective psychosocial data), can be one million times more than what is currently collected and shared on Facebook users. One education technology CEO bragged that their company picks up five to ten million data points from Pearson Common Core education curriculum per student per day.  There has been little to no formal discussion regarding the impact of expanding these pilot programs, and this bill gives the Florida Commissioner of Education the ability to expand it to every county, even before any results are reported.

National proponents of Competency Based Education have argued that these programs will eventually allow states to get rid of the statewide standardized assessment, which sounds great until you realize the goal is to replace tests like the FSA with daily, individual, data collection via computer programs.*

If you live in LakePinellasPalm Beach or Seminole County, you are part of the pilot program and this bill will bring CBE to your schools immediately.  For all other counties, it will be fully implemented within five years.

Other very serious problems with this bill and these programs include:
(Detailed quotes and references for these assertions are available here)

  1. These programs emphasize lower level job skills and social emotional skills at the cost of academic knowledge while diminishing the role of teachers and harming the student-teacher interaction.
  2. Gates Foundation-funded programs and technology education programs such as this in general have a long track record of failure and high cost.
  3. There is information showing Gates funding for only two (Lake and Pinellas) of the four counties covered in the bill. This means that the counties (Palm Beach and Seminole) or state taxpayers will likely have to pay for this experiment. The most recent Gates-funded experiment in Hillsborough County was a disaster, and ended up costing taxpayers millions.
  4. The language of the bill will ensure district-wide implementation of CBE before the results of the pilot are completed.
  5. There is no language about parental consent or pulling out of the five year program if students, parents, schools or districts change their minds or the program fails before the pilot ends. 

CALL TO ACTION!

The bill has already passed in the House, and is now being considered in the Senate.   We have just learned that the bill will likely be heard in the full Senate Appropriations Committee which will be meeting on Thursday, February 25th from 10 AM to 5 PM.  It is imperative we take action now. Please e-mail and call the Senators on the Florida Senate Committee on Appropriates (information below) immediately and tell them to vote no to SB1714!  Competency Based Education will be full of daily data collection, and will put our children’s personally identifiable information at risk.

Very important!  In addition to contacting these Senators, please contact your local school boards and superintendents and let them know you want your children educated by teachers; not data mined and taught by computers.  When you send your email, feel free to include a link to this blog post as well as the additional links below.

CS/SB 1714: Competency-based Innovation Pilot Program
GENERAL BILL by Education Pre-K – 12 ; Brandes
https://www.flsenate.gov/Session/Bill/2016/1714

Florida Sentate -Committee on Appropriations – Members

Copy and paste these emails addresses to send a bulk email:

benacquisto.lizbeth@flsenate.gov, lee.tom.web@flsenate.gov , flores.anitere.web@flsenate.gov,  altman.thad.web@flsenate.gov,   gaetz.don.web@flsenate.gov,  galvano.bill.web@flsenate.gov,   garcia.rene.web@flsenate.gov,   grimsley.denise.web@flsenate.gov, hays.alan.web@flsenate.gov, hukill.dorothy.web@flsenate.gov,  joyner.arthenia.web@flsenate.gov,  latvala.jack.web@flsenate.gov,  margolis.gwen.web@flsenate.gov,  montford.bill.web@flsenate.gov,     negron.joe.web@flsenate.gov,  richter.garrett.web@flsenate.gov,   ring.jeremy.web@flsenate.gov,  smith.chris.web@flsenate.gov

Chair: Senator Tom Lee (R)
E-mail – lee.tom.web@flsenate.gov
District Office – (813) 653-7061
Tallahassee Office – (850) 487-5024

Vice Chair: Senator Lizbeth Benacquisto (R)
E-mail – benacquisto.lizbeth@flsenate.gov
District Office – (239) 338-2570
Tallahassee Office – (850) 487-5030

Senator Thad Altman (R)
Twitter: @SenatorAltman
E-mail – altman.thad.web@flsenate.gov
District Office – (321) 868-2132
Tallahassee Office – (850) 487-5016

Senator Anitere Flores (R)
Twitter: @Senator_Flores
E-mail – flores.anitere.web@flsenate.gov
District Office – (305) 270-6550
Tallahassee Office – (850) 487-5037

Senator Don Gaetz (R)
E-mail – gaetz.don.web@flsenate.gov
District Office – (850) 897-5747
Tallahassee Office – (850) 487-5001

Senator Bill Galvano (R)
E-mail – galvano.bill.web@flsenate.gov
District Office – (941) 741-3401
Tallahassee Office – (850) 487-5026

Senator Rene Garcia (R)
E-mail – garcia.rene.web@flsenate.gov
District Office – (305) 364-3100
Tallahassee Office – (850) 487-5038

Senator Denise Grimsley (R)
E-mail – grimsley.denise.web@flsenate.gov
District Office – (863) 386-6016
Tallahassee Office – (850) 487-5021

Senator Alan Hays (R)
E-mail – hays.alan.web@flsenate.gov
District Office – (352) 742-6441
Tallahassee Office – (850) 487-5011

Senator Dorothy L.  Hukill (R)
E-mail – hukill.dorothy.web@flsenate.gov
District Office – (386) 304-7630
Tallahassee Office – (850) 487-5008

Senator Arthenia L.  Joyner (D)
E-mail – joyner.arthenia.web@flsenate.gov
District Office – (813) 233-4277
Tallahassee Office – (850) 487-5019

Senator Jack Latvala (R)
E-mail – latvala.jack.web@flsenate.gov
District Office – (727) 793-2797
Tallahassee Office – (850) 487-5020

Senator Gwen Margolis (D)
E-mail – margolis.gwen.web@flsenate.gov
District Office – (305) 571-5777
Tallahassee Office – (850) 487-5035

Senator Bill Montford (D)
E-mail – montford.bill.web@flsenate.gov
District Office – (850) 487-5003
Tallahassee Office – Same as District office

Senator Joe Negron (R)
E-mail – negron.joe.web@flsenate.gov
District Office – (772) 219-1665
Tallahassee Office – (850) 487-5032

Senator Garrett Richter (R)
E-mail – richter.garrett.web@flsenate.gov
District Office – (239) 417-6205
Tallahassee Office – (850) 487-5023

Senator Jeremy Ring (D)
E-mail – ring.jeremy.web@flsenate.gov
District Office – (954) 917-1392
Tallahassee Office – (850) 487-5029

Senator David Simmons (R)
E-mail – simmons.david.web@flsenate.gov
District Office – (407) 262-7578
Tallahassee Office – (850) 487-5010

Senator Christopher L.  Smith (D)
E-mail – smith.chris.web@flsenate.gov
District Office – (954) 321-2705
Tallahassee Office – (850) 487-5031

Recommended Reading

http://www.flstopcccoalition.org/files/E7DA713C-13D6-4D28-93CA-D376DFA759E6–40BE074B-D2E8-438F-AFCF-7194E326A218/specific-concerns-related-to-sb-1714-and-cbe-cost-quality-and-choice.pdf

https://accountabaloney.wordpress.com/2016/01/26/competency-based-education-destroying-public-schools-one-profitable-data-point-at-a-time/

https://unitedforflchildren.com/2016/02/18/competency-based-education/

http://www.flstopcccoalition.org/blog/concerns-about-competency-based-education.htm

http://dianeravitch.net/2016/02/21/eric-brandon-robot-teachers-for-the-masses/

https://www.alec.org/model-policy/digital-teaching-and-learning-plan/

http://www.competencyworks.org/

This Call to Action is supported by the following grassroots organizations/groups:

Accountabaloney
Broward BATs
Common Core Discussion Group – Florida
Florida Stop Common Core Coalition
Florida BATs 
FLParentsRISE
FLParentsRISE – Pinellas/Pasco Chapter
Fund Education Now
Lake County Against Common Core
Minimize Testing Maximize Learning
National BATs
Opt Out Leon
Opt Out Pinellas
Opt Out St. Lucie
Out Opt Florida Network
Tea Party Network
Uncommon to Our Core – Florida
United for Florida Children

*The original call to action has been edited to reflect the withdrawal of Representative Smith’s Amendment #045043, which suggested CBE in Florida would eventually replace the FSA.

CBE

 

Online “I can smell the education accountabaloney” Toolkit

Do you smell the accountabaloney?  Join us in calling it out.  We are hoping to create a strong online presence of parents and educators that are not afraid to call out the baloney.

Instructions to create your own meme:

CanYouSmellAccountabaloney_FNLClick to download the photo frame with the nose.

If you want to do a group shot, then use the frame without the nose.

Looking forward to a sea of clown noses online.