Adaptive testing

If it's going to get the Bored heated, then take it here PLEASE.
Post Reply
Message
Author
User avatar
Beebs52
Queen of Wack
Posts: 14978
Joined: Mon Oct 08, 2007 11:38 am
Location: Location.Location.Location

Adaptive testing

#1 Post by Beebs52 » Sat Mar 09, 2024 6:03 pm

Put here because sure it'll get political. Apparently SATs are using it. I think it's bs. I can see an employer using it, like MAT, etc, but not for knowledge based fact things, like that...what do ya think?
Well, then

User avatar
mrkelley23
Posts: 6268
Joined: Mon Oct 08, 2007 6:48 pm
Location: Somewhere between Bureaucracy and Despair

Re: Adaptive testing

#2 Post by mrkelley23 » Sun Mar 10, 2024 9:18 pm

Beebs52 wrote:
Sat Mar 09, 2024 6:03 pm
Put here because sure it'll get political. Apparently SATs are using it. I think it's bs. I can see an employer using it, like MAT, etc, but not for knowledge based fact things, like that...what do ya think?
Depends on what you mean. In the testing biz, adaptive testing usually means that the test adapts the difficulty to how the student is performing. So if I keep getting questions right, the difficulty keeps ramping up until I get one wrong. The goal is to make testing shorter and more accurate. I've oversimplified it above, of course, but from my experience, it works pretty well. My personal experience meant that a test with a 3 hour window only took me about 45 minutes. I've seen students significantly cut their test time, as well.
For a successful technology, reality must take precedence over public relations, for Nature cannot be fooled. -- Richard Feynman

User avatar
Beebs52
Queen of Wack
Posts: 14978
Joined: Mon Oct 08, 2007 11:38 am
Location: Location.Location.Location

Re: Adaptive testing

#3 Post by Beebs52 » Mon Mar 11, 2024 3:32 pm

mrkelley23 wrote:
Sun Mar 10, 2024 9:18 pm
Beebs52 wrote:
Sat Mar 09, 2024 6:03 pm
Put here because sure it'll get political. Apparently SATs are using it. I think it's bs. I can see an employer using it, like MAT, etc, but not for knowledge based fact things, like that...what do ya think?
Depends on what you mean. In the testing biz, adaptive testing usually means that the test adapts the difficulty to how the student is performing. So if I keep getting questions right, the difficulty keeps ramping up until I get one wrong. The goal is to make testing shorter and more accurate. I've oversimplified it above, of course, but from my experience, it works pretty well. My personal experience meant that a test with a 3 hour window only took me about 45 minutes. I've seen students significantly cut their test time, as well.
I'm confused as to what are the measurement criteria. What does it mean if I do better on an "easier" fact based test as opposed to one I decide to ride through in more difficulty? I guess I coulda put this over on main bored. I just expected the "not all students have same background", etc. Which still doesn't answer the first question. I find this all interesting. Made me ruminate on my sat scores.
Well, then

User avatar
mrkelley23
Posts: 6268
Joined: Mon Oct 08, 2007 6:48 pm
Location: Somewhere between Bureaucracy and Despair

Re: Adaptive testing

#4 Post by mrkelley23 » Mon Mar 11, 2024 9:09 pm

Beebs52 wrote:
Mon Mar 11, 2024 3:32 pm
mrkelley23 wrote:
Sun Mar 10, 2024 9:18 pm
Beebs52 wrote:
Sat Mar 09, 2024 6:03 pm
Put here because sure it'll get political. Apparently SATs are using it. I think it's bs. I can see an employer using it, like MAT, etc, but not for knowledge based fact things, like that...what do ya think?
Depends on what you mean. In the testing biz, adaptive testing usually means that the test adapts the difficulty to how the student is performing. So if I keep getting questions right, the difficulty keeps ramping up until I get one wrong. The goal is to make testing shorter and more accurate. I've oversimplified it above, of course, but from my experience, it works pretty well. My personal experience meant that a test with a 3 hour window only took me about 45 minutes. I've seen students significantly cut their test time, as well.
I'm confused as to what are the measurement criteria. What does it mean if I do better on an "easier" fact based test as opposed to one I decide to ride through in more difficulty? I guess I coulda put this over on main bored. I just expected the "not all students have same background", etc. Which still doesn't answer the first question. I find this all interesting. Made me ruminate on my sat scores.
Yeah, it's not about the inherent bias in standardized tests at all. And the College Board has done a better job of addressing that in the last couple of decades. But it's not "wokeness" or whatever you want to call it -- it's documented several ways by double blind methodology. A study in the 1990s, I think it was, showed there was a greater correlation between shoe size and college success than SAT scores. And it really shouldn't be surprising that a test written by a bunch of white, upper-middle class folks might have have cultural references that would not be familiar to people of other socioeconomic and ethnic statuses.

As far as measurement criteria, allow me to oversimplify again. Let's say you take a test of 100 questions over 4 areas of some discipline. There are 25 questions on each area, with five levels of difficulty. In the past, everyone would answer all 100. Some kids would get some of the harder ones right by chance. Over a large population, it would even out (some kids would miss those same questions by chance) to make it statistically valid. But some kids would benefit who didn't really deserve to.

Now we switch to adaptive testing. I ask you a level 1 question, and you get it right. Same with level 2 and 3, but you miss the level 4 question. So I fall back to a few level 3s. Statistically, you're not going to get 3 of those right in a row, unless you really know that level 3 stuff. If you do, we go back to level 4 and try a few of those. Instead of 25 questions in each discipline, I might ask you only 8 or 9. And arrive a truer idea of what you actually know.

Does that make any sense?
For a successful technology, reality must take precedence over public relations, for Nature cannot be fooled. -- Richard Feynman

User avatar
kroxquo
Posts: 3061
Joined: Sun Feb 17, 2008 12:24 pm
Location: On the Road to Kingdom Come
Contact:

Re: Adaptive testing

#5 Post by kroxquo » Tue Mar 12, 2024 5:40 am

Beebs52 wrote:
Mon Mar 11, 2024 3:32 pm
mrkelley23 wrote:
Sun Mar 10, 2024 9:18 pm
Beebs52 wrote:
Sat Mar 09, 2024 6:03 pm
Put here because sure it'll get political. Apparently SATs are using it. I think it's bs. I can see an employer using it, like MAT, etc, but not for knowledge based fact things, like that...what do ya think?
Depends on what you mean. In the testing biz, adaptive testing usually means that the test adapts the difficulty to how the student is performing. So if I keep getting questions right, the difficulty keeps ramping up until I get one wrong. The goal is to make testing shorter and more accurate. I've oversimplified it above, of course, but from my experience, it works pretty well. My personal experience meant that a test with a 3 hour window only took me about 45 minutes. I've seen students significantly cut their test time, as well.
I'm confused as to what are the measurement criteria. What does it mean if I do better on an "easier" fact based test as opposed to one I decide to ride through in more difficulty? I guess I coulda put this over on main bored. I just expected the "not all students have same background", etc. Which still doesn't answer the first question. I find this all interesting. Made me ruminate on my sat scores.
And as far as "easier" facts, here is an an example. If this was a test on US Presidents and assuming it is all multiple choice, Level 1 might be "Who is the current US President?"; Level 2 might be "What was the name given to Franklin Roosevelt's legislative social program?"; Level 3 might be "What was 'Manifest Destiny'?"; Level 4 might be "Why did Lincoln suspend the right to habeus corpus?" Making some of the questions open-ended (i.e. short answer), especially at levels 3 & 4 would also increase difficulty although make the test harder to grade.
You live and learn. Or at least you live. - Douglas Adams

User avatar
silverscreenselect
Posts: 23274
Joined: Mon Oct 08, 2007 11:21 pm
Contact:

Re: Adaptive testing

#6 Post by silverscreenselect » Tue Mar 12, 2024 7:10 am

kroxquo wrote:
Tue Mar 12, 2024 5:40 am
If this was a test on US Presidents and assuming it is all multiple choice, Level 1 might be "Who is the current US President?"
Uh, oh. Half the Republicans taking the test would miss this one.
Check out our website: http://www.silverscreenvideos.com

User avatar
Beebs52
Queen of Wack
Posts: 14978
Joined: Mon Oct 08, 2007 11:38 am
Location: Location.Location.Location

Re: Adaptive testing

#7 Post by Beebs52 » Tue Mar 12, 2024 2:03 pm

I guess it doesn't make much difference overall except for testers' convenience, especially if you're brilliant.
Disclaimer-I ain't. I did pretty well but that was in the day it was 800/800 for a perfect score, which I didn't get. What is it now?
Well, then

Post Reply