This is a follow-up to my posting of September 23rd . As information has continued to pour in on the webct-users and webct-admin listserves I follow, this software defect just gets weirder and weirder. The good news is that we’re pooling our observations and that Blackboard is being extremely attentive.
Recapping the ISSUE: Students are saving answers to tests. The fact that they’re doing so is being recorded in server application logs. But NOT in the database so that the answers are displayed AND graded by the application and displayed back to teachers and students.
The SCOPE: It’s being reported far and wide across institutions. From Saskatchewan to Warrnambool, from Sheffield to Boulder, with Tuscaloosa and Des Moines in between. We’re running Bb Vista 8.0.3 and Bb Vista 8.0.2. But, unlike I posted earlier, it’s now been seen in Assessments which do NOT contain paragraph-style answers. It’s been seen on various windows desktops (XP and Vista). I’ve heard of one case (New Mexico State) on Safari 4 (does this mean a Mac? because this may be the only case). It’s been reported with the PC browser Firefox (3.0.13; 3.0.14; 3.5.2; 3.5.3) and using various versions of Java (1.6.0_11; 1.6.0_15; 1.6.0_2). I’ve searched my archives …I can’t find any IE instances … It definitely involves some client-side interaction between browser and other desktop installed items.
RULED OUT: Load balancers. Multi-casting. Respondus Lock-down browser. Student error. Resetting the Assessment.
CURRENT SUSPECTED CAUSE: webserver.logs are demonstrating in these cases there is a single POST saveAllResponse followed by one or more GET saveAllResponse entries. It is suspected that the Firefox browser, under certain conditions (yet unknown) is sending a GET request which Bb Vista software versions 8.02 and 8.03 are misinterpreting.
HOW OFTEN ARE THE ASSESSMENT RESULTS LOST LIKE THIS? From all the webserver logs submitted to Blackboard so far, which contain records of missing assessments AND records of assessment results recorded, they are calculating that the error is occurring 8 out of every 1000 tests taken, for a 99.2% reliability rate (and a nearly 1% error rate). It may seem trivial at first, but when 10,000 tests are taken and 80 students have blank scores… you can see the problem is HUGE.
For now I suppose, we advise students to use IE when taking exams. I should at least cut down on the incident rates if not eliminate them altogether.
I want to give Kevin Lowey of the University of Saskatchewan a huge huge thank you from all of us. He’s really helped define what this issue is and how it’s happening. Kevin is an occasional blogger here. If you too would like to be able to think like Kevin, perhaps reviewing his brain would be helpful. We love you, Kevin!