Making this easy: LMS Evaluations

My brain just did a flashback as my fingers poised over the keyboard ready to begin this post. The song, “War is a Science,” from Pippin has started to syncopate through my skull:

the rule that every gen-er-al
kno-ws by he-art:
it’s smarter to be lucky
than it’s lucky to be smart!


I value ‘smart’ (O how I value smart!) but  in my experience we overthink LMS evaluations.

It’s not about a Request for Proposal process. It’s not about a comparison of features. It’s not about the best software package out there. It’s not even primarily a decision of open source vs proprietary, although this exercise may help you characterize your institution’s culture as one or the other and that will get you started down the right track …

LMS Evaluations are like any other decision you have to make for your institution. It’s about trusting that the software you choose matches the way your institution does things.

It’s a cultural decision. Wasn’t always. But these days the market is mature enough that all these packages (Canvas, Sakai, Blackboard, Desire2Learn, Moodle) can do pretty much the same thing. It’s the way they do them that you care about. It’s the way your institution plans to use and support the software that will make the implementation project a smashing success or an unadopted disaster.

This is what you need to plan for and do:

1). Select the group of people at your institution that you trust to make this decision for you. Is it an already existing faculty committee? Maybe its composed of appointees to be representative of each college or department, with central IT or the Library thrown in because they have to run it?

2). Create and follow whatever rigor or metrics these people will need to document and communicate their decision for maximum buy-in.

That’s it.

No kidding. Institutions often publish their final LMS Eval reports. Read them. The variety they represent is as wide as the cultures of the institutions that created them. They’re not all smart. But the successes are the lucky ones who chose software that matches their institutions’ culture.

12 thoughts on “Making this easy: LMS Evaluations

  1. Keep in mind, the culture of politicians and institution administrators is not the same as the culture of students, faculty members, or technologists.

    1. I agree with sneezypb. There are many subcultures in academe. Making sure they are properly represented in technology decision making can be a real political challenge. It may not be an issue as much anymore if lms have indeed been commoditized (which isn’t absolutely clear). But in the past (at least in my region) an interest in expediency and minor differences in usability often eclipsed concerns about openness.

  2. Luke,
    Your statement, “an interest in expediency and minor differences in usability often eclipsed concerns about openness,” identifies a cultural quagmire at your institution. As a technical implementor you may prefer open source because you see advantages for it, either professional or institutionally, while other subcultures at your institution may not see those advantages. If open source is pushed through despite a positive climate for acceptance, it may end in disaster. This makes my point. If open source is chosen against this tide and then ultimately perceived to be a bad decision, it is NOT the software’s fault.

    1. Agreed: a particular outcome, no matter how great it appears shouldn’t be chosen if the only way to get there is by overriding or ignoring fair procedures. To take an example from youth: we’re willing to give the larger half of a cookie to our classmate as long as we’re the one who gets to cut it. We practiced that rudimentary form of justice because we knew, even as kids, that fair procedures allow us to live more comfortably with unequal and/or undesirable outcomes.

      Still, even if we need to honor fair procedure, that doesn’t mean we shouldn’t try to convince our university colleagues of the virtues of openness and the possibility that it’s a value all academic sub-cultures actually share [ even if a particular subculture doesn’t realize it yet 🙂 ].

  3. What a great discussion! Finally, we’re seeing more journalist say exactly what Laura said, “It’s not about the software features.” But I agree with the comments that follow, if it’s a cultural decision – which culture has final say? While participatory governance is a time-honored principle in higher ed, the RFP process is insufficient when schools are choosing a provider partner to chart their future academic technology strategy. Laura, when you say, “It’s the way your institution plans to use and support the software that will make the implementation project a smashing success or an unadopted disaster” – that is the most accurate statement! From the vendor side, schools are struggling to achieve a unified vision; therefore their purchasing decision making is immediately flawed. Our advice: choose the provider that understands your needs. For thirteen years, as an ‘influenced source’ provider, WebStudy continually evolves to meet customer needs.

  4. I think the biggest challenge is that all types of faculty, and all departments, have different needs for the software. So the question is how can we accomodate most instructors without alienating of confusing others.

    I’m sure many instructors are also fatigued with systems constantly being switched.

  5. Laura – Thanks for a great post. Right on the money.
    My own post on this subject (to which you referenced above), is simply stressing (a) the fact that the institution has its own interests, (b) these interests have little to do with education per se, and everything to do with education management. (Which is important, too.) And (c) the importance of the institution’s interests will make it very difficult to move beyond the existing LMS model – which some people advocate.

Comments are closed.