Funding & Other Baloney Part II: How You Spend It Matters A Lot

In Part I of this series, we questioned whether Florida’s embarrassingly low per pupil spending, in the face of questionable measures of quality, really meant our education system was “efficient” or just cheap. We also asked that the Florida Department of Education stop celebrating high 4th grade reading scores as a measure of high quality, when those scores are clearly the result of mandated 3rd grade retention and any benefit noticed evaporates by the 8th grade assessments. In Part II, we wonder if increased funding might lead to improved outcomes in our public schools.

Part II: How You Spend It Matters A Lot

When parents learn that Florida spends well below the national average in per pupil spending (learn more here and here), many advocate for increased education spending as a way to improve the quality of our public schools. It seems to make intrinsic sense that schools spending significantly more educating their students would, naturally, be able to provide a higher quality education. Indeed, many districts rely on that belief to get voter approval for local sales tax initiatives or increased millages to help fund local schools.

On the other hand, education “reformers” frequently claim there is no relationship between per pupil spending and student outcomes on standardized test scores and they use that idea, frequently, to advocate for low levels of state education funding in the name of “efficiency.”  This appears to be a commonly held belief amongst Florida Board of Education members, the Department of Education and many, if not most, of our legislators.  The idea seems to defy common sense but (much to my surprise) the research on the subject, until recently, actually appeared to support the position.

The idea dates back to 1966, when James Coleman lead a group of sociologists to write a report on educational equality in the United States, commissioned by the U.S. Department of Education. It was a massive study surveying more than 150,000 students, one of the largest in history. The finished report was titled “Equality of Educational Opportunity,” and is common referred to as The Coleman Report.

The results of the Coleman Report are widely reported to show that variation in school resources (as measured by per-pupil spending and student-to-teacher ratios) was unrelated to variation in student achievement on standardized tests. These results have been upheld by further research leading to reformers to report this as a “commonly held” belief. Indeed, Dr. Jay Greene, testified to as much during the recent CSS v BOE trial, pointing out that variations in state funding did not correlate with proficiency levels in Florida’s districts (you can see his testimony here).

Interestingly enough, the Coleman Report didn’t really deny that funding or other school effects matter, but it did argue that other factors are more important (learn more here). “Specifically, the report found that student background and socioeconomic status are much more important in determining educational outcomes than are measured differences in school resources. It also affirmed that differences in schools—and particularly teachers—have a very significant impact on student outcomes.” The study, also, concluded that disadvantaged black children learn better in well-integrated classrooms, leading to the busing of students in attempts to desegregate public schools. Additionally, the Coleman Report, suggested that standardized tests measured cultural knowledge, not intelligence, putting minority students at a disadvantage.

Regardless, the belief became “commonly held” amongst education policy makers that increased funding will not improve education, pointing at high poverty schools with many special needs children requiring expensive programs, yet scoring low on standardized tests, as proof. Many education reformers equate increased funding for public schools, in the face of low standardized test scores, as “throwing good money after bad.”

What if the lack of a connection between improved funding and a better educational outcome is simply because standardized tests scores are not the best measurement of learning?

A recent study addressed the limitations of standardized testing and the results suggest that increased education funding may result in significantly improved outcomes, especially for children of low income families. The 2105 report by C. Kirabo Jackson, Rucker C. Johnson and Claudia Persico is titled The Effects of School Spending on Educational and Economic Outcomes: Evidence from School Finance Reforms.The report asks the question, “Does school spending matter after all?” (You can read more about it here and here.)

“Tracking students born between 1955 and 1985, the researchers isolated the districts where court-mandated reforms would affect spending at those students’ schools; next, they compared those students’ achievement to similar students’ progress in districts that did not receive greater funding. The effects they found were huge and likely causal: A 10 percent increase in per-pupil spending led to higher graduation rates, nearly 10 percent higher earnings in adulthood, and fewer incidents of poverty later in life. The researchers found small effects on students from affluent families but huge effects on poor students.”

So, increased periods of spending had a significant effect on graduation rates AND  narrowed the wage gap between low-income and non-poor students when they reached adulthood:

highergradratenarrowingwagegap

This study showed that the “commonly held belief” may be wrong, demonstrating, instead, that increased education funding can have significant impacts on educational and economic outcomes, particularly for children in low income families.  This suggests that increased funding, whether it results in improved test scores or not, may lead to improved lives. Perhaps standardized test scores are NOT the best measure of the impact of education funding? Maybe narrowing the achievement gap on standardized test scores is not nearly so important as narrowing the economic gap? Maybe a better metric of quality is needed? We think Florida should consider this.

Public education advocates are often accused of wanting to “throw good money after bad.” Rest assured, we are not asking for that, nor were the authors of the 2015 study. The 2015 study authors cautioned that how additional funds were spent “matters a lot.

The authors evaluated the effects of court-mandated spending increases on spending for school support services, physical capital, and instruction. They also estimated the effects on student-to-teacher ratios, student-to-guidance-counselor ratios, teacher salaries, and the length of the school year. What they found was:

“… exogenous increases in school spending are more likely than other forms of school spending to go to instruction and support services. The increases for instruction and for support services (which include expenditures to hire more teachers and/or increase teacher salaries along with funds to hire more guidance counselors and social workers) may help explain the large, positive effects for students from low-income families.”

“While there may be other mechanisms through which increased school spending improves student outcomes, these results suggest that the positive effects are driven, at least in part, by some combination of reductions in class size, having more adults per student in schools, increases in instructional time, and increases in teacher salaries that may help to attract and retain a more highly qualified teaching workforce.”

So, when appropriately used, increased education funding CAN lead to improved outcomes, especially for our most at-risk public school children. Sadly, in recent years in Florida, rather than focusing funding towards instruction and support services, state and federal mandates have directed large portions of school budgets towards things like technology infrastructure, progress monitoring and assessments. In the face of nearly flat education spending, the net result is less funding for the very things shown to improve student outcomes: class size reduction, guidance counsellors, social workers and the increased salaries necessary to attract and retain a high quality workforce.

We urge the Florida DOE to expand its definition of educational outcomes beyond standardized test scores and reconsider funding priorities. Increased funding, when spent appropriately, can have significant, lasting effects for our students, especially those most at-risk. Continuing to ignore the positive effects of appropriate funding is #accountabaloney.

Advertisements

Florida’s Middle School Math Problems: A Perfect Storm

Why did Florida’s 8th grade National Assessment of Educational Progress (NAEP) math scores plummet?  We believe it may be the result of a “perfect storm” created by  Common Core Math and Accountabaloney…

Florida State University Physics Professor, Paul Cottle, in a December 5, 2015 op-ed (read it here) in the Tallahassee Democrat, sounded the alarm regarding recent dismal middle school math performance:

“Florida’s middle schools have fallen off a cliff in math, according to recently released results from the National Assessment of Educational Progress, an exam given to a sampling of students in nearly all states.

When NAEP was administered in 2013, it determined that 31 percent of Florida’s eighth graders were proficient in math. That was below the 2013 national average. But the news this year was much worse: Only 26 percent of the state’s eighth graders were found to be proficient in that subject.

It’s important to note that the national math proficiency rate for eighth graders declined as well – from 35 percent in 2013 to 33 percent in 2015.

But Florida’s decline was the nation’s largest. You might think that Florida’s educational leaders would mobilize an effort to address this crisis in middle school math. But you’d be wrong.”

Mr Cottle goes on to suggest attracting mathematically talented young people to teaching is the solution to our middle school math problems. While that may be part of the solution, we think the problem may go much deeper than teacher quality.

We do agree with Mr. Tuttle, however, when he says, “There is one thing for sure: Pretending the problem doesn’t exist isn’t going to make it go away.”

Lennie Jarratt wrote a column on 12/2/2015, discussing declining NAEP scores since the institution of Common Core State Standards (CCSS). She reported that the 2015 NAEP scores showed “an across-the-board decrease in math test scores,” the first drop in 25 years. She, also, pointed out that, after breaking down the data, there was a greater decline in states that had adopted the Common Core math standards. She voiced concern that the standards might be to blame:

“The math techniques now associated with Common Core-aligned math are solidly entrenched in many public education systems across the nation, even though in 2006 the National Council of Teachers of Mathematics called for an end to these techniques and a return to teaching the basics, i.e. direct instruction and memorization of basic facts. These basics provide a solid foundation for understanding, learning, and building future math concepts. Teachers who use Common Core-aligned math are similar to those who attempt to build a house without a foundation; the house is destined to crumble.”

When the National Council of Teachers of Mathematics are calling for a return to basics, one wonders why policy makers would not listen?

What has changed since transitioning to the Common Core? In a column from the Brooking Institute, Tom Loveless outlines the differences between math instruction prior to and after initiation of the CCSS. In a nutshell, he describes how the CCSS math sequence delays some basic math instruction, resulting in 6th graders now practicing basic division algorithms when they used to be focusing on the study of rational numbers  (fractions, decimals, percentages).

Since Florida Standards curriculum closely aligns to Common Core, this  means that  Florida students are now wasting time working on basic math into middle school, time that should be spent in the study of rational numbers.

Also in Florida, middle school math is being squeezed from the top end as well because there is a greater and greater push to move Algebra and Geometry classes into 6th and 7th grade. Florida’s current A-F school grading system rewards schools that place students into these advanced math courses. Schools have responded by placing more and more students into Algebra 1 whether they are ready or not. Last fall, parents in Orange County were outraged to learn that their middle school Algebra 1 students were simultaneously placed in remedial math courses, presumably to give those students extra time to prepare for the Algebra 1 End of Course exam. Placing students into advanced math when then are not properly prepared seems ill-advised.

In addition, the course content in Florida’s Algebra 1 and 2 courses no longer resembles the courses students took preCCSS. In an attempt to make Florida Standards more “rigorous” than regular CCSS, Florida added dozens of advanced math standards to the upper end of high school math. The trickle down effect has resulted in approximately 1/3 of the previous Algebra 1 content now being taught in pre-algebra, 1/3 of the previous Algebra 2 content now being taught in Algebra 1 and Trigonometry now being taught in Algebra 2. This shifting of advanced content into lower level math courses exacerbates the squeeze on time available to learn traditional middle school math material (fractions, decimals, percentages).

The shift has been so dramatic that we question whether middle school Algebra 1 teachers (with a middle school math credential which covers math content up to grade 9) are teaching out of subject when they are required to teach Algebra 2 content. Florida has a law that says families must be notified if their child’s teacher is teaching out of subject. Is your child’s Algebra 1 teacher certified to teach Algebra 2 content?

The combination of the Common Core Math’s delay of basic math instruction and Florida’s drive to reward increasingly difficult math into middle school has resulted in a perfect storm. It’s intensity is reflected in Florida’s plummeting 8th grade Math scores. It is another clear example of the destructive force of accountabaloney.

NAEP scores sounded the storm warning. We think that Florida’s educational leaders should be mobilizing an effort to address this crisis in middle school math. Do they have any such plans? Can they smell the baloney? We can only hope…

Batten the hatches.

 

DISCLAIMER and a Call for Help from Florida’s Math Teachers: It is not entire clear what or who is responsible for placing Algebra 2 content into Florida’s Algebra 1 course standards.  Is it part of CCSS or unique to Florida Standards? We are not sure. If anyone has information regarding this or any other impacts of accountabaloney on Florida’s math sequence, please contact us.

 

 

 

 

 

 

Honestly? This is baloney!

Honesty_Quote

The blog where we expose the real truth about proficiency, achievement levels and accountabaloney.

Currently, Florida’s Board of Education is contemplating raising achievement levels (“cut scores” or “passing levels”) for the new Florida State Assessment (FSA) to match the much higher “proficient” achievement level on the National Assessment of Educational Progress (NAEP), thus eliminating the so-called “honesty gap”. Floridians have been told that it is important that we are honest with our students now, rather than have them discover their academic shortcomings later. Such an important measurement, one that will affect how many students pass 3rd grade or earn a high school diploma, deserves a serious look.

In general, if you are going to create something called an “honesty gap” and then use it to accuse states of lying to students about their math and reading abilities, you should be extra careful to confirm you are, in fact, telling the truth. It would be pretty bold to create a deceptive measure and then call it the “honesty gap”. Just sayin’.

The Invention of the Honesty Gap

The first reference to this “honesty gap” appears to be in a May 2015 report from Achieve  (read it here, if you must).  Achieve is an organization dedicated to corporate education reform. It helped develop the Common Core State Standards (CCSS), write the Next Generation Science Standards (NGSS) and served as project manager for states in the Partnership for Assessment of Readiness for College and Careers (PARCC). Achieve suggested that state test results and the NAEP often tell “conflicting stories about students’ proficiencies in math and reading.” They called the discrepancy, between state proficiency results and NAEP proficiency rates, a “proficiency gap”, claiming too many states were “saying students are “proficient” when they are actually not well prepared.”

This summer, the Foundation for Excellence in Education, the education reform group founded by Jeb Bush, began promoting this discrepancy as the “honesty gap”, encouraging citizens to learn about the “level of dishonesty in their state” through websites such as whyproficiencymatters.com and honestygap.org. In Florida, they even launched a social media campaign encouraging Floridians to email Governor Rick Scott, asking him to protect children from being “victims of Florida’s proficiency gap” and “ensure we raise the bar so that our children can succeed in college and careers.”

Question: Were Achieve and FEE being honest about this honesty gap? How safe is our children’s education?

A look at NAEP and its achievement levels.

NAEP, also known as The Nation’s Report Card, was developed in 1969 to provide information on how well US students were performing over time. It is frequently referred to as the “gold standard” of student assessments. By 1990, criterion based achievement levels (basic, proficient, and advanced) were added to NAEP and comparisons across states became possible. Initially, NAEP participation was voluntary. Since the passage of No Child Left Behind (NCLB), participation in 4th and 8th grade Math and Reading NAEP assessments has been required as a condition to obtaining Title I funds. NAEP obtains its data by sampling; a sample of 4th and 8th grade students from each state participate in the biannual assessment.

No Child Left Behind, of course, required the annual testing of all students between 3rd and 8th grade with the goal of 100% grade level proficiency for all students, in reading and math, by 2014. Of course, that was an impossible goal.

It doesn’t take much research, however, to realize that the disparity between state proficiency levels and NAEP proficiency level (the “honesty gap”) can, in a large part, be explained by differences in the way each test defines proficiency.

The Center For Public Education explains the discrepancy this way (emphasis added):

From: http://www.centerforpubliceducation.org/Main-Menu/Evaluating-performance/The-proficiency-debate-At-a-glance#sthash.PiWCF4ar.dpuf

Just because NAEP and states label a level of achievement “proficient” does not mean they are defining proficiency the same way. Nor are they necessarily testing the same knowledge and skills.

For example, the National Assessment Governing Board (NAGB), the board that oversees NAEP policies, states that “In particular, it is important to understand clearly that the Proficient achievement level does not refer to ‘at grade’ performance” (Loomis and Bourque 2001).

Unlike NAEP, NCLB requires states to define proficiency in terms of being on grade level. A report developed by the U.S. Department of Education specifically states that “The proficient achievement level [for NCLB] represents attainment of grade-level expectations for that academic content area” (U.S. Department of Education 2007).

A report commissioned by the NAEP Validity Studies Panel found that the most appropriate NAEP level for gauging state assessment results should be the percentage of students at or above the basic achievement level (Mosquin and Chromy 2004) Even U.S. Secretary of Education Margaret Spellings urged reporters to compare state proficiency rates to NAEP’s basic level (Dillon 2005). Moreover, James Pellegrino, lead author in the National Academy of Science’s 1999 evaluation of NAEP called Grading the Nation’s Report Card, suggested that perhaps proficiency for accountability purposes lies somewhere between NAEP’s basic and proficient levels because NAEP’s proficiency level was not developed with accountability in mind (Pellegrino 2007).

To repeat, with NAEP, “proficiency” DOES NOT refer to “at grade performance” and, by law, state tests MUST define proficiency in terms of being on grade level.  In other words, NAEP standards are aspirational, whereas state standards are set for minimal competency. I wonder why, when the distinction between the two definitions of proficiency are so clear, would organizations like Achieve and FEE confuse them?

Is it possible that the only honesty problem we have is coming from those who insist on equating the NAEP and FSA definitions of proficiency? The two are simply not the same.

Additionally, NAEP proficiency levels were NOT developed with accountability in mind but, clearly, FSA test scores are primarily used for accountability purposes. If Florida were to follow FEE’s advice, and raise FSA achievement levels to match NAEP levels, all the ramifications of Florida’s test and punish accountability system would come into play (see our “Ingredients” post here). The establishment of cut scores for the FSA is not made in a vacuum. If fewer children pass the Florida assessment because of higher cut scores, than more children will suffer the mandated repercussions.

Keep in mind that NAEP only tests  students in grades 4 and 8, so proficiency levels for the other grades, particularly grades 3 and 10 where Florida’s high stakes are greatest, would have to be extrapolated from a few NAEP data points.

Matching FSA passing levels to  NAEP proficiency levels, in the atmosphere of Florida’s test and punish accountability system, would be inappropriate and misguided. Predicting the extent to which that might happen, requires an understanding of the relationship between NAEP scores and other student achievement measures.

Student Achievement and NAEP

The Center for Public Education, also, reports interesting data regarding student achievement, international achievement and NAEP scores (details and references here):

  • No country, even those performing highest on international assessments, would have 100 percent of its students reach NAEP’s proficiency level and no country would have 100 percent of its students reach NAEP’s basic level (Phillips 2007).
  • Looking at high school math students, 44% of “A” students in math would score below the NAEP proficient level and 80% percent of “B” students would score below proficient (Scott, Ingels and Owings 2007).
  • 32 percent of seniors who had completed calculus would not reach the proficient level on NAEP (Scott, Ingels and Owings 2007).
  • According to ACT, a score of 22 on the math portion of its college admissions test indicates that a student is ready for college algebra (Allen and Sconing 2005) but almost 80% of students who scored between 21 and 25 on the ACT mathematics assessment scored below NAEP’s proficiency level for math (Scott, Ingels and Owings 2007).
  • As expected, 91% of high school seniors who scored at NAEP’s advanced level in math and 79% of those who scored proficient went on to earn a 4 year bachelors degree(Scott, Ingels, and Owings 2007).
  • Students who performed below proficient also had significant rates of obtaining a 4 year bachelors degree, especially in comparison to the nation where approximately one third of adults receive a four-year degree. A full 50% of students scoring at the basic level went on to receive bachelors degrees and almost 20% of those who performed below basic on NAEP received bachelor degrees (Scott, Ingels, and Owings 2007).

What Florida should expect if our “proficiency gap” is closed:

If nearly a third of calculus students are unable to reach proficiency levels in Math, it becomes clear that NAEP cannot possibly define “grade level proficiency.” When half of students scoring at basic level go on to complete college, it would be dishonest to suggest to those students that they were “not college material.”

If students who reach grade level expectations are still not ready for college, perhaps it is the system that focuses almost entirely on grade level performance on standardized tests that is to blame.

At the October 28th Florida Board of Education meeting, it was suggested that parents needed to have achievement levels better explained to them.  It appears the BOE would benefit from similar instruction. Even the Commissioner of Education (who needed to check an online dictionary for definitions) might need a review.

If Florida’s Board of Education decides to raise that proficiency bar, beyond grade level expectations, to match NAEP’s “proficiency” level and eliminate this so-called “honesty gap”, then board members should expect (based on the data above):

  • Nine year old third graders, reading at grade level, will be marked for mandatory retention, forced to take summer school and additional assessments, and will be labeled as “bad readers”.
  • Just under half of “A” students in math will fail their FSA Math assessment and up to 80% of B students will fail. These students will be made to believe they are “not good” at math and may be required to take remedial classes.
  • Students who might have passed calculus, will take remedial math classes, instead.
  • Large numbers of students, who are fully capable of obtaining a 4 year bachelors degree, will fail to pass the 10th grade FSA ELA, will be labeled “not college ready” and will be assigned to demoralizing remedial reading classes and exam retakes.
  • Test prep will increase for all students as teachers struggle to get all children above grade level, a mathematical impossibility.
  • Teachers and schools could be negatively impacted even if all students are performing at or above grade level (yet below “proficiency”).
  • Children who are performing at a developmentally appropriate level will be told they are poor achievers.

When the ramifications of fewer children passing these tests occur, it will no longer be considered “unintended consequences” but the direct result of these actions by this Florida Board of Education, who will have failed to understand that grade level proficiency and NAEP proficiency are NOT synonymous.

At the end of the day, the most important question the board members should be asking is: “Does the level 3 achievement score, for every subject, every grade, represent developmentally appropriate grade level expectations, as required by NCLB?” Additionally, since it represents the difference between promotion and retention, “Does the Level 2 Achievement Score for 3rd grade reading represent a developmentally appropriate, minimally acceptable, grade level expectations?” Of course, these are questions that can only be answered by educators, not advocates or ideologues. Please, ask the educators.

P.S. There are rarely times when I cannot see both side of an issue. This is one of those times. This Honesty Gap is clearly based on the misrepresentation that the determination of “proficiency” on NAEP and the state’s requirement to monitor grade level proficiency have anything in common other than the word “proficiency”. Could it be that the reformers really don’t understand or do they expect their followers to follow blindly without question? What would happen if states heed the call of FEE, Aspire, and Mr. Padget, and raise state achievement levels to match NAEP’s high bar? Millions of children will fail state tests and the reformers will have “proved” our public schools are failing. These organizations are either seriously ignorant or they are being completely deceitful. I am hoping common sense and true honesty (not this deceitful reformer kind) will prevail and cut scores will be set based on educationally sound principles rather than political baloney.