State ignores big question: Was student test fair?

Senate Education Appropriations Subcommittee Chairman Don Gaetz, R-Niceville.
Senate Education Appropriations Subcommittee Chairman Don Gaetz, R-Niceville. The Associated Press

After two weeks of computer glitches and cyber attacks, a long list of people have a long list of questions about Florida’s new standardized student assessments, starting with this critical one:

Was it even a fair test?

"It's an excellent question — and one the department of education will need to answer empirically," said Senate Education Appropriations Subcommittee Chairman Don Gaetz, R-Niceville.

But it’s also a question that the Florida Department of Education so far seems inclined to ignore. Despite concerns echoed by parents, teachers, superintendents and outside academics, the department has refused to say whether it will follow the path of other states that have analyze results to determine if student scores were affected by similar technical problems.

“The test was not hacked as the attackers never gained access. Since student responses were not accessed, there is no reason not to use the test results,” Meghan Collins, the department’s communications director, wrote in an email.

Collins repeatedly responded to questions about the validity of the tests by saying only that the department was “proud” that 90 percent of students had successfully taken the writing portions of the new assessments as of Friday.

But critics argue that a post-test analysis is vital because the stakes of the results are so high. The test will be used to determine whether high schoolers can graduate, and whether teachers can keep their jobs.

Parents and educators have plenty of doubts after the sputtering launch of the Florida Standards Assessments, a new, tougher and highly controversial replacement for the old Florida Comprehensive Assessment Test.

While there were no problems with paper versions of the tests given to kids in grades four through seven, students trying to take the computer-based for the first time during March ran into an array of problems. A software update by test contractor American Institutes for Research and a cyber attack caused long log-in times, booted some students off mid-test and erased answers in some cases.

“The glitches can make part of the exam, or maybe the entire exam, not totally valid,” said Samuel Perkins, an associate professor and program director at the school of education at Barry University in Miami Shores. “How did they impact the taking of the exams? How did they impact the students nerves and comfort, because we know some students have a phobia of standardized exams?”

Miami-Dade County schools Superintendent Alberto Carvalho believes technical problems created “inconsistency in the testing environment that is not acceptable.”

Some students saw test materials before being kicked offline, and got to return another day to complete the exam. Others may have rushed because they were worried they would get kicked off the system again. Still others had no problems at all taking the online tests.

A handful of other states have run into technical problems with tests and reaction to the issue has varied.

In Kansas, a pilot of new state tests last year followed almost exactly the same pattern as in Florida. First, there were issues with test servers and later a cyber attack.

A review of the results showed that students who suffered technical difficulties skipped more questions than students who didn’t: 15 percent compared with 1 percent, said Marianne Perie, director of the Center for Educational Testing and Evaluation at the University of Kansas, which created and administered the tests. Other comparisons showed more discrepancies.

Kansas ultimately decided not to release student test scores, a step that required federal approval.

“You don’t want to hold someone accountable for performance that was impacted by a technical glitch. That’s not fair for anybody,” Perie said. “Incorrect information is worse than no information.”

Conversely, Minnesota decided to stick with its test scores after state tests administered by American Institutes for Research — the same company handling Florida’s tests — got off to a rocky start. An independent review there found student’s results were not impacted.

Even before bugs bedeviled Florida’s tests, there have been questions about the validity of the test. It wasn’t fully field tested, only certain questions were. And that was done in Utah – a state with little in common with Florida.

Such geographical differences can matter. For example, Florida educators have in the past taken issue with a test question that required students to know that a moccasin is a type of shoe, whereas students here are more familiar with snakes of the same name.

The tests are also based on new, harder education standards.

Still, so far, the state has decided the tests will count in high-stakes decisions from the very first year — and lawmakers haven’t budged on the issues either, despite the questions. A proposal to hold off on issuing school grades, which also are based on the tests, failed to make it out of a house panel on Thursday after Republicans disagreed.

Still, Senate education appropriations subcommittee chairman Gaetz said he has yet to determine "the extent to which the test's integrity had been compromised." He said he was meeting regularly with state Education Commissioner Pam Stewart on the subject.

"We want to know when the DOE knew and what they knew it," he said.

Many superintendents and parents, along with teachers unions, say that students shouldn’t be punished for what amounted to a test run of a critical test.

“Our kids are lab rats,” said Miami-Dade PTA President Joseph Gebara.

“It’s unfortunate that, because of the politics involved, the state can’t just back up and say: ‘You know what? We were wrong’,” he said. “It’s the egg-in-the-face moment they all want to avoid.”

The sputtering roll-out and resistance to even analyzing its impact has been a boon for at least one group in South Florida:

Interest in the Opt-Out movement, a national group of parents boycotting high stakes tests, has steadily grown with the testing problems, said Rosemarie Jensen, a Broward County parent.

“Parents are concerned that kids will bare the consequence of something that is just a totally failure on the part of the state,” she said. “In general, they don’t trust what’s coming out of Tallahassee.”

Follow @Cveiga on Twitter.

Related stories from Miami Herald