Florida students and their teachers are held to account based the scores on the high-stakes FCAT tests. School funding is partially contingent on test performance. Robert Krampf, a Florida science educator, has been reviewing the test-prep materials given to teachers in order to refine his own curriculum and prepare his students.
However, the test-prep materials were very poor. They consist of multiple-choice questions with more than one correct answer. For example: "This sample question offers the following observations, and asks which is scientifically testable: 1 The petals of red roses are softer than the petals of yellow roses; 2 The song of a mockingbird is prettier than the song of a cardinal; 3 Orange blossoms give off a sweeter smell than gardenia flowers; 4 Sunflowers with larger petals attract more bees than sunflowers with smaller petals."
The curriculum guide says that the correct answer is 4, but 1 and 3 are also correct. Krampf asked FLDOE's Test Development Center for clarification, and the Center told him that although the question had three answers, only one was "correct" in the context of the curriculum -- that is, students would only have learned about testing 1, and not about the chemistry needed to test 3, or the observational methodologies to test 4. This is just dumb. It means that the test doesn't distinguish between students who misunderstand the curriculum, students who are making guesses, and students who have progressed beyond the curriculum. In other words, the test can't tell you anything useful about the students' understanding or the teachers' methodology.
The question about isn't an isolated example, apparently. Krampf reports finding many examples like this from all parts of the test, some of which weren't just bad test-design, but factually incorrect; for example, the test defines a "predator" as "An organism that obtains nutrients from other organisms." As Krampf points out, "By that definition, cows are predators because they obtain nutrients from plants. The plants are predators too, since they obtain nutrients from decaying remains of other organisms."
I wonder how many students got "wrong" answers on the FCAT because their teachers taught them too much. How many "F" schools would have higher grades if those scientifically correct "wrong" answers were counted as correct answers. How many "B" schools would get the extra funding that "A" schools get, if those scientifically correct "wrong" answers were counted as correct answers?
We may never know the answers to those questions. The Test Item Specifications are the guidelines that are used to write the test questions. If the Science FCAT test is reviewed by the same Content Advisory Committee that reviewed the Test Item Specifications, then it probably has similar errors. But as much as I would LOVE to check the accuracy of the questions from the actual Science FCAT, I can't. Teachers, scientists, and the general public are not allowed to see actual test questions, even after the tests have been graded and the penalties for those grades have been imposed.
Standardized testing is usually a mess. High-stakes standardized testing is usually a bigger mess. But even by those standards, the FCAT science tests are a disaster, and the lack of transparency and accountability in them means that they're doomed to fail Florida's students for a long time to come.
I write books. My latest is a YA science fiction novel called Homeland (it's the sequel to Little Brother). More books: Rapture of the Nerds (a novel, with Charlie Stross); With a Little Help (short stories); and The Great Big Beautiful Tomorrow (novella and nonfic). I speak all over the place and I tweet and tumble, too.