Here at Notre Dame we’re finishing up a project we called “CMS Replacement Project.” We started last May (2007), hit our stride in the fall, and have been synthesizing, collating and analyzing data since then. The synthesized data was presented to the Course Management System Steering Committee (CMS SC) last Friday. It was tremendous.
TechSmith’s Morae product was used to capture the data generated by 20 Faculty completing around 30 use cases in 3 different CMSs. Interviews were conducted with vendors and with institutions utilizing those 3 different CMSs (narrowed down by whether they use the same SIS we do, SCT Banner). Campus visits by the vendors and representatives gave us an idea of their forward vision. 25 students were summarily recruited, through the strategic use of Starbucks and iTunes gift cards, to spend 20 minutes captured by Morae again, with the 3 different systems.
All video captures were chopped into tasks where time on task and mouse clicks on task could be measured. Participants were classified as Novice, Intermediate, or Expert. Which CMS they did first was rotated.
If that weren’t enough data, we also ran complete pilots with volunteer Faculty and the courses they taught last semester, as well as our STEP distance ed folk. This part meant entering into pilot contracts with those those hosted those systems for us.
We spent money on this replacement project to determine who has the next contract at Notre Dame.
I hope to be able to tell you who the winner is by the end of March. And look forward to Educause for our report on the method we used. It has been exciting at times, tedious at others…