Sen. Leising has introduced SB 58 which appears to be a reaction to the revelation that last year’s ILEARN test did not produce reliable metrics. Under this legislation, a school’s performance rating for 2018-2019 can’t be any lower than 2017-2018, “additional consequences” for school improvement may not be applied for the 2018-2019 school year, and ILEARN scores can’t be used for a teacher’s performance evaluation. “Additional consequences” has to do with provisions under IC 20-31-9 providing for escalating consequences for each year that a school is in the lowest performance category (e.g. state takeover or closure in the 4th year.) Note: I’m going off the digest with for the proposition that the “additional consequences” are paused. I can’t quite parse the language of the bill to reach the conclusion that the clock stops running for the 2018-2019 school year – my reading suggests that the 2017-2018 performance will be substituted unless the school wants the 2018-2019 performance to be applied. But the language is pretty dense, so I might be misreading something.
Sen. Leising has also introduced SB 59 which provides that ILEARN scores (and other “objective measures of student achievement and growth”) may not account for more than 5% of a teacher’s total performance evaluation. I should be a little tempered in my critique here – it’s an improvement on the status quo. That said, I would strongly argue that ILEARN is not an objective measure of student achievement or growth. Fact is, we don’t know what the hell it is measuring. Its metrics – which, lest we forget, cost the State $40 million – are unreliable. ILEARN suggests that West Lafayette, the 20th best STEM school in the nation, has 40% of its students lacking proficiency in math or English. If the test missed that badly in West Lafayette, it can’t be trusted in other schools either. Using this test as even 5% of a teacher’s evaluation is too much. Furthermore, even if we had a standardized test that was a reliable measure of student growth and achievement, using them as a measure of teacher performance is problematic since there are too many confounding variables for us to mistake correlation for causation. Usually those test results end up being pretty well correlated with a student’s socioeconomic status.
Not a bad start, but seems like there is room for improvement here. I’m an enthusiastic amateur when it comes to education law – I’d love to see the opinions of people who know the ins and outs of this stuff.
We need to remember that legislators and appointed public servants who know nothing about testing, statistics or education are actually pretty easy to fool. Give them classy presentations and pages of language that sounds new and sophisticated and you have them. The Indiana iLearn page is a great example. This happens at all levels, from K through graduate school, where someone who has to make a decision is presented with numbers and it’s a slam dunk. I was in a university setting, which housed a number of very knowledgeable statistical people but who the administrators did not bother to consult. The stat people could have told them that they were about to buy a bill of goods, but that never happened. They bought and used junk that any student in a beginning statistics course could have told them should not have been purchased.. I’m sure the same thing happened in Indiana, where people think that being an elected anti-intellectual who managed to graduate from high school or degree where one learns to be feel entitled is better than someone who might know about the product in question. A call to the educational psychology or psychology departments at Purdue or Indiana will probably confirm my suspicion that they were never consulted about this. Reading the Indiana iLearn page, I found no data and nothing that actually told me about this test. Just sales talk.
Reading your last paragraph motivated me to weigh in on this mess, but I couldn’t find any data about validity or reliabillity at all, let alone validity and reliability under the wide range of circumstances this thing is supposed to be ready to take on. There should be extensive data for every circumstance, comparing iLearn performance with some known test someplace. The question is “valid in relation to what?” and then “what is the validity in relation to what under given circumstances?” If the Lafayette STEM school ranks in the bottom half of whatever is being compared, there is something wrong with the kids, the school or the test. I strongly suspect it’s not the first two. It measures something, for sure, but anybody who knows anything about kids and learning can tell you that this is just plain awful. The people of Indiana have been conned, not just by a worthless test but by people who have allowed themselves to be fooled, who were probably too arrogant to consult someone who actually knows about this sort of thing and legislators who think they are able to make decisions about complex issues solely because they are elected. We got what we should have expected. The numbers haven’t lied..
The underlying theme of these tests is that they don’t trust the teachers and administrators. I think the teachers will tell you, by and large, that they know where their students are strong and where they are weak. I think the administrators will tell you, by and large, that they know which teachers are effective and which are not. As I understand it, the greatest need for tests are of the type that are short, quick, and produce prompt results that can be used in the middle of the term to identify what concepts and skills the students are struggling with.
The testing regime is, by and large, the product of a notion that teachers and administrators are incompetently lazing around and not to be trusted to tell parents and members of the public the truth about students or the system. Instead, you need to throw buckets of cash at testing companies to provide objective data.
Most of the tests are not designed to be diagnostic, in the sense that they answer the question, “What skills are lacking that this child needs to know next or what do we need to know to fill the gap(s) preventing him/her from progressing?” Those questions really can’t be answered psychometrically because test analysis is not focused on educational needs so much as it focuses on item validity, ability to predict total score and item stability (reliability), among other noninstructional issues. Psychometrics is not education. It’s measurement of large ideas and constructs, especially in the state tests. If you look at the test subscale labels, they mention abstract skills and theoretical constructs but the actual subtest scores are so unreliable and difficult to make into concrete skills that they are almost impossible to use. (If you have the chance, check out the reliability coefficients of the subtest scales. They tend to be low, and serve as the basis of ways to determine whether a score is what it is or whether the “true score” is actually much higher or much lower. The actual range is usually incredible, and only serves to show how useless they are.) That is the ugly truth that the testing people don’t discuss, ostensibly because the area is pretty difficult for the public to understand, but the final answer is that the state test is not designed to help teachers become more efficient at what they do. When interpreting these things, it’s probably best to throw out the highest and lowest scaled scores and go somewhere down the middle. The labels provide a deceptive veneer of sophistication and applicability. Skills identification is best done by teachers and personnel through interviews, actual performance. and the teacher’s experience in the area. Even then, managing that incredible amount of information with 30 kids and trying to address the needs on anything resembling individual instruction, even when they are known, is a huge and very difficult task. Trying to apply standardized test scores to the actual teaching/learning environment is a fool’s errand. In the end, tests serve as an administrative tool to determine allocation of resources.
Personally, I usually believe that legislators’ beliefs that teachers are lazy and incompetent and personal projections of their own incompetence. Furthermore, it’s interesting that the Phi Delta Kappa educational survey done since forever has consistently shown that the public rates their own local schools with an “A”, the schools in the next county with a “B” and in the rest of the state with a “C” or “D”. Everyone else’s school is a dump but my local school, and that shines.
Your knowledge of what goes into these tests is so far beyond mine that a lot of what you are saying here is pretty opaque to me. But, I do grasp the gist that the margin of error on some of the measures the tests offer up is so great as to not be worth reporting – let alone carry the freight that lawmakers put on the tests.
The “everyone else’s school is a dump but my local school, and that shines” makes me think of a similar phenomenon with lawyers. Most people like their own!
It seems we’ve always had a legislature whose members actually believe that there are simple answers to the most complex problems, but are reluctant to try to understand what is actually going on if it seems to lie outside their agenda. (What is that theory that suggests the less people know the more they think they know?) That makes them vulnerable to con artists who offer the quick and easy solutions that fit their demands. Money seems to attract con artists. Whatever legislators decide either simply doesn’t solve “the problem” or further adds to the difficulty in solving or even understanding it. When we are dealing with people, there is nothing wrong with solutions that end with more questions, but somehow legislators believe that a problem can be completely fixed quickly and easily without struggle. So, we end up with tests and program schemes that are ineffective or bring on corruption, and financial procedures that don’t fund equitably and serve to drive out good people who refuse to work in an irrational environment. They don’t need this in a job that is already hard enough. This isn’t conservative government. It’s ignorant government that currently seems to fit the notion of “conservative”. And it rules education at the behest of a public that believes we don’t have to know much to know everything!
“Ignorant government” is the perfect description of the current state legislature. They choose to not understand the harm caused by all of this testing. Mr. Cook knows of the consequences very well as a former superintendent, but for two of his terms, he has chosen to go along with the GOP caucus and take part in this harmful policy. Now suddenly he has an epiphany about testing? I feel that he was only motivated by the prospect of losing an election in 2020.