Adaptive testing

[Beebs52]

Put here because sure it'll get political. Apparently SATs are using it. I think it's bs. I can see an employer using it, like MAT, etc, but not for knowledge based fact things, like that...what do ya think?

[mrkelley23]

Put here because sure it'll get political. Apparently SATs are using it. I think it's bs. I can see an employer using it, like MAT, etc, but not for knowledge based fact things, like that...what do ya think?

Depends on what you mean. In the testing biz, adaptive testing usually means that the test adapts the difficulty to how the student is performing. So if I keep getting questions right, the difficulty keeps ramping up until I get one wrong. The goal is to make testing shorter and more accurate. I've oversimplified it above, of course, but from my experience, it works pretty well. My personal experience meant that a test with a 3 hour window only took me about 45 minutes. I've seen students significantly cut their test time, as well.

[Beebs52]

Put here because sure it'll get political. Apparently SATs are using it. I think it's bs. I can see an employer using it, like MAT, etc, but not for knowledge based fact things, like that...what do ya think?

Depends on what you mean. In the testing biz, adaptive testing usually means that the test adapts the difficulty to how the student is performing. So if I keep getting questions right, the difficulty keeps ramping up until I get one wrong. The goal is to make testing shorter and more accurate. I've oversimplified it above, of course, but from my experience, it works pretty well. My personal experience meant that a test with a 3 hour window only took me about 45 minutes. I've seen students significantly cut their test time, as well.

I'm confused as to what are the measurement criteria. What does it mean if I do better on an "easier" fact based test as opposed to one I decide to ride through in more difficulty? I guess I coulda put this over on main bored. I just expected the "not all students have same background", etc. Which still doesn't answer the first question. I find this all interesting. Made me ruminate on my sat scores.

[mrkelley23]

Put here because sure it'll get political. Apparently SATs are using it. I think it's bs. I can see an employer using it, like MAT, etc, but not for knowledge based fact things, like that...what do ya think?

Depends on what you mean. In the testing biz, adaptive testing usually means that the test adapts the difficulty to how the student is performing. So if I keep getting questions right, the difficulty keeps ramping up until I get one wrong. The goal is to make testing shorter and more accurate. I've oversimplified it above, of course, but from my experience, it works pretty well. My personal experience meant that a test with a 3 hour window only took me about 45 minutes. I've seen students significantly cut their test time, as well.

I'm confused as to what are the measurement criteria. What does it mean if I do better on an "easier" fact based test as opposed to one I decide to ride through in more difficulty? I guess I coulda put this over on main bored. I just expected the "not all students have same background", etc. Which still doesn't answer the first question. I find this all interesting. Made me ruminate on my sat scores.

Yeah, it's not about the inherent bias in standardized tests at all. And the College Board has done a better job of addressing that in the last couple of decades. But it's not "wokeness" or whatever you want to call it -- it's documented several ways by double blind methodology. A study in the 1990s, I think it was, showed there was a greater correlation between shoe size and college success than SAT scores. And it really shouldn't be surprising that a test written by a bunch of white, upper-middle class folks might have have cultural references that would not be familiar to people of other socioeconomic and ethnic statuses.

As far as measurement criteria, allow me to oversimplify again. Let's say you take a test of 100 questions over 4 areas of some discipline. There are 25 questions on each area, with five levels of difficulty. In the past, everyone would answer all 100. Some kids would get some of the harder ones right by chance. Over a large population, it would even out (some kids would miss those same questions by chance) to make it statistically valid. But some kids would benefit who didn't really deserve to.

Now we switch to adaptive testing. I ask you a level 1 question, and you get it right. Same with level 2 and 3, but you miss the level 4 question. So I fall back to a few level 3s. Statistically, you're not going to get 3 of those right in a row, unless you really know that level 3 stuff. If you do, we go back to level 4 and try a few of those. Instead of 25 questions in each discipline, I might ask you only 8 or 9. And arrive a truer idea of what you actually know.

Does that make any sense?

[kroxquo]

Put here because sure it'll get political. Apparently SATs are using it. I think it's bs. I can see an employer using it, like MAT, etc, but not for knowledge based fact things, like that...what do ya think?

Depends on what you mean. In the testing biz, adaptive testing usually means that the test adapts the difficulty to how the student is performing. So if I keep getting questions right, the difficulty keeps ramping up until I get one wrong. The goal is to make testing shorter and more accurate. I've oversimplified it above, of course, but from my experience, it works pretty well. My personal experience meant that a test with a 3 hour window only took me about 45 minutes. I've seen students significantly cut their test time, as well.

I'm confused as to what are the measurement criteria. What does it mean if I do better on an "easier" fact based test as opposed to one I decide to ride through in more difficulty? I guess I coulda put this over on main bored. I just expected the "not all students have same background", etc. Which still doesn't answer the first question. I find this all interesting. Made me ruminate on my sat scores.

And as far as "easier" facts, here is an an example. If this was a test on US Presidents and assuming it is all multiple choice, Level 1 might be "Who is the current US President?"; Level 2 might be "What was the name given to Franklin Roosevelt's legislative social program?"; Level 3 might be "What was 'Manifest Destiny'?"; Level 4 might be "Why did Lincoln suspend the right to habeus corpus?" Making some of the questions open-ended (i.e. short answer), especially at levels 3 & 4 would also increase difficulty although make the test harder to grade.

[silverscreenselect]

If this was a test on US Presidents and assuming it is all multiple choice, Level 1 might be "Who is the current US President?"

Uh, oh. Half the Republicans taking the test would miss this one.

[Beebs52]

I guess it doesn't make much difference overall except for testers' convenience, especially if you're brilliant.
Disclaimer-I ain't. I did pretty well but that was in the day it was 800/800 for a perfect score, which I didn't get. What is it now?