menu
The Future of Digital Education

A collection of podcasts and blog posts discussing the role of technology in education

Blog Post

What Could Possibly Go Wrong? Assessing the Challenges of Digital Assets – Steven Furino

whatcouldgowrong

I first authored an online course in 2010 and offered that same course in Fall 2010. I have been involved with online education ever since in a number of capacities: author, teacher, encourager, funder, administrator, champion, and colleague of DigitalEd. The Faculty of Mathematics at the University of Waterloo now offers more than 60 degree credit courses online, a graduate degree online and has courseware available to roughly 120,000 high school students.

My experience has made me somewhat skeptical in general. Though I continue to be thrilled by the enthusiasm of those joining the move to digital assets, there has been a certain amount of naiveté in that enthusiasm. As a result, when people advocate for this or that particular digital initiative, it occasionally causes others to ask, “What could possibly go wrong?”

Realistically, there is a long list of things that can go wrong. I will share some of those with you in the hopes that enthusiasm, tempered by experience, leads to large gains for everyone involved in education. For this post, I will focus on things that can go wrong from the teaching side. My comments are not specific to particular software, but rather speak to the larger effort of developing and deploying digital assets for education.

There can be tremendous gains made using technology, but that technology alone will be hard pressed to create those gains in learning. We need to align our technology with wisdom from cognitive science and psychology, with the social structures of schools and neighborhoods, and with reductions in economic class differences that deprive significant fractions of the population of both technology and the motivation to study. The following are areas to pay close attention to when developing and implementing digital content.

Online education is a great idea. I am sure all of my colleagues will participate.

A tenured research faculty member has an inordinate number of demands on their time. At this point, developing high-quality digital assets is still time consuming and of uncertain advantage in many situations.

Do not feel the need to bring everyone on board at the beginning. Almost certainly, there will be early adopters. If the experience of the early adopters is positive, there will be a natural tendency to move towards strategies that have demonstrable, sustainable advantages. This makes it all the more important to have early wins.

This won’t take long.

This depends on the production quality you want and the degree of compliance with local standards and accessibility legislation. Well produced courses take time. Courses built with interactivity in mind take time. Courses built in such a way that they can be maintained take time.

There are always trade-offs. If demand is high enough, then production quality can be traded off against timely production. But the rules of the game can change. The costs of complying with accessibility legislation forced the University of California at Berkeley to withdraw over 20,000 free online educational audio and video publications from their website.

At Waterloo we have the advantage of an in-house production team that consists of about 80 staff members. At other places, the author needs to be a subject matter specialist, a web developer, an audio and/or video editor, a copyright specialist and have sufficient distance from her or his own work to do quality assurance. It is possible and has been done, but the sustainability of this “heroic model” of development is questionable. Like textbooks, it is often more cost and time efficient for an institution to license, or to require students to license, existing assets rather than build their own.

Free is best.

Free is certainly good for students and politicians in the short term. It is not clear how we build a sustainable business model around free though. Technology companies are not charitable organizations and relying exclusively on funding from foundations or special government grants is not viable in the long term because of the unreliability of those funds.

In building digital assets we need to be very conscious of our complete set of costs and of how we can meet those costs over time. At many institutions, there is a way to get seed money up front to experiment. The more difficult part in my experience has been to find ongoing financial support.

Students will learn so much more with digital assets.

It is possible. But for that to happen, we need to do much more than place our notes on the web. Humans learn by doing. Mathematics is not a spectator sport so we need the ability to allow students to carry out experiments with mathematical objects. Can we rotate a three dimensional surface to find local minima? Can we modify parameters of a system and observe changes in the system’s behavior?

Humans also rationalize. Being able to ask a mathematically sophisticated question, receive a mathematically sophisticated answer, and give real-time feedback allows the student to be honest with themselves about what they know and what they don’t know. If we can repeatedly ask questions in a low stakes environment, then students can practice and learn at their own speed. This is where tools like Möbius Courseware and Möbius Assessment excel.

Students will work through all of the material.

No they won’t. And it doesn’t matter whether the content is digital or physical. Do you read every page of every document you are expected to review? Content needs to be designed so that students gain, and see the gain, of working through all of the content. If possible, we need to use social expectation and pressure to encourage students to spend enough time with the content to become competent. Möbius Courseware can help you track the behavior and accomplishments of each student and intervene as necessary.

Students are so interested in learning they won’t cheat.

As Associate Dean I was responsible for student discipline. I am familiar with a huge array of illicit resources available to students. We have to assume that in all of us there is a predisposition to take moral shortcuts. If you have any doubt about that, let me simply cite the difference in income tax collected when governments moved to collect income tax at source rather than relying upon us to report our income accurately. It is also not a surprise that an overworked student who has always done well in the past and is desperate to do well in the present might resort to techniques they would never have previously considered.

It doesn’t take much to deter most shortcuts and simultaneously emphasize learning. First, make cheating take effort. If all of us have the same questions on an electronic quiz, it is enough to have a smart person take the quiz first and distribute the questions and answers. Möbius Assessment and Möbius Courseware allow us to randomize questions and make that basic strategy far less successful. Second, be clear about expectations of academic integrity and support those expectations with invigilation and consequence. Something as simple as an automated checking of IP addresses of students taking quizzes can alert instructors to anomalies that should be investigated.

I’ll keep the digital assets up to date.

Most people want to keep assets up to date but don’t, in large part because of the amount of work involved. As we gain more experience with how students use digital assets, we can fairly quickly identify what should be changed to support learning. Having the underlying technology support agile development and deployment of digital resources is a huge advantage.

Summary

So what advice do I have for someone interested in building digital assets? Simply put, begin. Pick a small project that is valuable as a standalone project but one that could be easily incorporated into a larger project. It will help to have someone to work with. You may want to start with someone else’s assets and modify those. Be patient with yourself and those working with you. Expect to take a lot of time. When you deploy the assets, get feedback and iterate. Tools like Möbius Courseware allow us to develop and deploy in ways that are learning effective, cost effective, replicable and shareable. Our focus should be on the creation of digital assets that can support and sometimes transform existing educational and social structures rather than target the elimination of those structures.

Steven Furino, Assistant Dean, Online Studies
University of Waterloo

Matt
written by: Matt