2.1 | Defining our research questions

How we defined our research questions.

The cultural probes

In order to define the problem, we needed to synthesize the data we achieved through the cultural probes and the expert interview. We began with the cultural probes by laying all of the tasks from all five probes in five respective piles and go through all of the answers one pile at the time while systematizing them in a simple form:

Results from the Circle Around the Word task:
– Stupid, Confused, Annoyed
– Confused
– Stupid, Confused, Annoyed
– Boring
– Confused, Satisfied, Smart

Results from the Pick a Face task:
– Bad / sad
– Bad / sad
– Happy
– Happy
– Okay happy

Results from the Expectations while using Communication Platforms task:
– «That» everyone » is on the platform and that it works well. Will also use the one that people are most available on.»
– «That it works properly.»
– «That I get replies quickly and that the platform works well and quickly.»
– «The fact that the platform is fast, can send files, seldom downtime, easy to use, no restrictions on what to share.»
– “Possibility of a group chat, that they are easy to use, that no problems occur while using them. Also, the ability to send more than just text, such as pictures and files. »

Results from the Weekly Basis Communication Platforms task:
– Facebook, Snapchat, SMS, Discord, Instagram, Twitter.
– Discord, Messenger, Snapchat.
– Facebook, WhatsApp, Instagram, Email.
– Facebook, Steam, Discord, Instagram, Snapchat, Canvas, Email, YouTube.
– Facebook / Messenger, Instagram, Snapchat, Vipps chat, in game chat functions like League of Legends Client chat, Email, SMS, YouTube.

Results from the Statements task:
1. «I always know where and when I have lectures thanks to Canvas.»
– Agree
– Neutral
– Strongly disagree
– Agree
– Neutral

2. «It’s easy to organize for group work in Canvas.»
– Neutral
– Strongly disagree
– Disagree
– Neutral
– Strongly disagree

3. «It is easy to respond to messages from teachers / lecturers.»
– Neutral
– Agree
– Disagree
– Disagree
– Disagree

4. «I can find my grades and feedback from my professors in Canvas quickly.»
– Agree
– Strongly agree
– Neutral
– Neutral
– Neutral

Results from the Complete the Sentence task:
«What I wish it was possible to do in Canvas is…»
– “I want a neatly organized Canvas where all subjects have the same structure and name and are in the same language. Better and more explanatory titles would have been nice, too. »
– «Don’t use more than one scroll bar.»
– “Instant chat with teachers.”
– “Organize the website the way you want”
– “Chat functions with those you are working with on group projects + Opportunity for co-writing (better).”


When we had gone through all of the probes, we were able to see a few patterns emerging, so our next step was to categorize the probe answers by organizing them in clusters.

The results from the cultural probes, organized in clusters.

Canvas/LMS was the main cluster in the centre from which we derived the results into three sub-clusters; Communication, Emotions, and Wishes/Improvement suggestions.

From Communication, we divided the probe answers further. Many answers brought up features we usually find on social media platforms, such as chat, so obviously we had to have a SoMe sub-cluster, as well as an Expectation sub-cluster. From the probe answers, we noticed that the expectations the people were having to LMS was often connected to the ability to communicate, like file sharing and chatting. Other expectations were connected to the more technological aspect of LMS, such as user-friendliness and stability. Chat functions were also echoed in the Wishes/Improvement suggestions sub-cluster, but this cluster also brought up another exciting suggestion: The possibility to organize the LMS yourself.
For the Emotions cluster, we noticed that a majority of the probe answers were those of negativity when dealing with their current LMS (Canvas).

The expert interview

Trying to synthesize data from the expert interview was a more challenging task, but even here we saw a clear pattern emerging as we analysed the interview.

The expert interview organized in clusters.

We noticed that we could easily divide mr. Bisseberg’s main statements into two major sub-clusters: What is good with Canvas, and what is bad.

The good features are mostly technological, like constant bugfixes and updates, and little downtime, just to mention a few. Should the service be down for some reason, then Canvas can provide good support.
The bad features are often based on the user experience, such as a less than optimal user-friendly design, limitations in regards of co-operating and custom design the LMS for your own needs. «The «one size fits all» doesn’t fit all» statement sums up this aspect rather accurately.  It made us wonder, though: Would it be possible to hold on to everything that makes Canvas a good technical user experience, but simultaneously change and improve upon what makes it bad?

1.2 | Expert interview

Mr. Bisseberg is an engineer, working at Østfold University College, and THE go-to-person with all things Canvas and LMS related!

Name: Per Olav Bisseberg
Location: His office at campus Remmen – Face to face interview.
Occupation: Head Engineer at the IKT department of HIOF.
Other Experience related to this: Canvas Top Admin.

In this interview we wanted to collect data about LMS (mainly Canvas) and the expert user’s experience and position towards these and their functions.

We wanted to gather as much data according to this to help us understand how and why we could improve these.

Interviewer(Q): «What do you do in your position here?»
Per Olav Bisseberg (A): » I am the Canvas top admin, and technically, that means I’ll have to make sure all integrations towards other systems are working. If Canvas goes down for some reason it’s my job to get it up and running again. I am the the one that keeps in touch with the supplier, and I’m involved with the operating of Canvas here.»

Q. «What experience do you have with LMS like Canvas?»
A: » I got a lot of experience with LMS overall. I am also in a national team that works towards better systems for students when it comes to school related platforms.»

Many of [Canvas’] features are acceptable, but as I see it, could have been done better


Q: «What is your perspective when it comes to group work in Canvas? Does a lot of people come in these doors with group related questions?»
A: «No, not really. Most of the cases gets cleared by the IT-Support team. What comes through my door is usually more major issues that has to be failchecked, or searched for, or in some cases, notified to other parties so they can fix it. But from experience, we had a intro project in Canvas the fall of 2017 and throughout the spring of 2018, and there were a lot of questions about almost everything from both the staff and users that were chosen for this pilot, but ever since, the rate of support cases have decreased majorly. Most cases from students where there are problems, are coming from huge projects or classes that have tried to use the «Rate Each Other» function in Canvas. That function is not always working properly, and Instructure (company name of Canvas owners) are working overtime to get a new solution out for this. I’m saying, and you might quote me on this if you’d like, I am not a Canvas evangelist, I do see a lot of problems with Canvas as it is today, but compared to all the other options we have, Canvas is the least faulty learning platform.
Back when we used the Fronter (another LMS), the situation was a lot worse, and we encountered a lot more problems. It was also up to each teacher to actually use it. Today we use Canvas and that is mandatory for all teachers that all the information they hand out should go through there, as well. Canvas puts the students in focus and the teachers next.»

Q: «What do you think is the biggest challenge with Canvas at HIOF today?»
A: «The biggest challenge we face with Canvas today is for the teachers to use it correctly so that it get to be a good, easy and pleasant experience for the students.»


Canvas puts the students in focus and the teachers next.


Q: «HIOF has a lot of future teachers under training, and I know that they are very focused on using digital solutions, what are your prospects there?»
A: «They’ve built up a big IKT section that’s looking into how IKT works with school and the educational worth around it.»

Q: «With your experience with Canvas, are there any functions that you’re missing as today?»
A: «On the admin side of things I do miss a lot, mostly tools for us as admins, but overall I think Canvas is good. There are small problems here and there, like the Wiziwig box where you write comments and such does not support the writing of code as things might get unorganized and highlights ect. might not be shown. That is a thing I’m missing, they said they were going to do something about that, but that was back in 2017 and is probably not as high on their priority list. I do wish they had used some more time on creating user experience and interaction design. It is a lot, especially for the teachers, that is a bit clunky that with better design could have been avoided, and down to the more fundamental things that they don’t have one set of iconography. Their symbols aren’t coherent. There is also a lot of Google Translate integrated that’s not always working properly. Like recordings got translated to «registrering» in Norwegian that caused some problems. If the iconography and the translations had been better it would probably save us for a lot support cases.»

Q: «Do you have any data of what is the most used devices to use Canvas with?»
A: «Yes, we do actually. And our data tells us that approximately 47% of all our interactions are from Smartphones. We do have some future problems there that we need to handle, as we got a lot of teachers that uses Canvas primarily as a information channel towards their students, works great on a smartphone, but we also got a lot of people that uses it as a file system, and that doesn’t always work so good on smartphones as they might not be able to read the specific type of document on their phone. If we had everything as PDF it would make it less of a problem, and if they had everything on Canvas as pages, then this would have scaled down while your on the phone as well.»

Q: «Do you have any other data that could help us in our search of making Canvas better?»
A: «The last information we gathered were the interaction data from 2018. We haven’t done any data gathering in 2019, and that is primarily because of the ruleset of something called GDPR that tells us what kind of information we are allowed to use. We want to make sure what data we can look at and what we can analyze until we get a signal from UNIT that it is legally okay for us to do so. A new wind that is coming is called Learning Analysis, that uses all these data to determine what learning methods that have good effect, what might not have such a good effect, how the stream of student are and so on, but we can not go into this data yet. America and China has gone to the extreme when it comes to learning analysis as their laws around personal privacy is lacking. In China this has gone to far, as liking a particular band might not get you accepted to certain schools as they have data that says their former students that also liked that band did not do so well.»


I wish Canvas was only a basic site.


Q: «What is Canvas best at?»
A: «I’m a critical person, so I usually see things in systems that usually doesn’t work way before I see what’s actually working, but I do think it works great in the way that teachers don’t really need a lot of training nor time to use Canvas to inform or share material needed for their lectures. Other than that, there is very many things that are okay, not good, but okay. Many of its features are acceptable, but as I see it, could have been done better, but that is probably because my developer background kicking in. They got to get some credit on their uptime and the fast communication from school to instructure and vica versa is really good. They have extremely good procedures when it comes to fixing problems and pushing updates out. Every third week they send out a new update, and the size of these updates are regularly huge and comes with a lot of fixes.»

Q: «Is it anything you miss in Canvas that you think could be implemented?»
A: «In many ways I feel like Canvas already have too many functions. There are so much information, so many things that you can do and to keep in mind. So i understand that for many teachers and students Canvas can become a bit overwhelming. I wish Canvas was only a basic site; a platform for teachers to inform about their course, deliver assignments and get reports on those assignments – and that’s it. That you could instead add other services to your basic Canvas platform. Canvas uses something called L-10, that allows us to integrate a lot of tools to their site. If it was up to me i would have putted a lot less functions on Canvas and made sure those were operating at a high level, and then have each institution get the additional functions they themselves wanted to add.»»

Q: «Thank you for taking time talking to us on the matter of Learning Management Systems and Canvas!»
A: «You’re welcome.»


Findings: Students are not the only ones having problems when it comes to navigate Canvas. Canvas seems to have a lot of problems, but less than other competitors LMS. It would probably be hard for us to find information regarding Canvas and stats in for the Norwegian region. Almost half of all interactions with Canvas is by smartphones, even though it ain’t compatible for many things there (not all phones can open all the different files, the website would look weird on some phones etc.) Good uptime on Canvas, good support from Instructure. Canvas might be over complicated – can we make a clean version with the L-10 plugins? Some restrictions to what type of files teachers can post might benefit smartphone users.


In addition to our expert interview with mr. Bisseberg, we also wanted to have a student’s point of view, and sat down with our fellow student Eric Michael Lea Paulsen for an Average User Interview.

Location: The Library at campus Remmen – Face to face interview.
Occupation: Student – studying Digital Media and Design at HIOF.
Other experience related to this: Studied Photography and Filmmaking at a Folk high school in Norway for one year – mainly focused towards filmmaking.

In this interview we wanted to collect data about LMS (mainly Canvas) and the average users experience and position towards these and their functions from a student’s point of view. We wanted to gather as much data according to this to help us understand how and why we could improve these on the user’s -the student- conditions.

Interviewer (Q): «What experiences do you have with learning platforms like Canvas and group projects while being a student?»
Eric(A): «I have no previous experience when it comes to using Canvas, but I’m trying to adapt to using it and I think it’s a good tool for making groups and doing group projects. We never did use much of electronic learning platforms , only Itslearning a bit at high school. It worked quite well back then, we never really had any major issues with it. When it comes to group projects, I’m fairly new at that, but when we’ve done group projects earlier it has mostly gone through Facebook, or some other social media platform we had at hand.»

Q: «When you used Facebook, were there any specific tools you used more than others doing group work?»
A: «I mostly just used it to start a chat and to share files.»

Q: «From the experience you have accumulated so far, what seems to be the most recurring problems when you use Canvas?»
A: «For me it’s got to be that it is very confusing and messy at times. Especially when they put the Informatics Class tasks with our tasks, so we get a lot of tasks that are irrelevant to our courses and it creates a lot of confusion. Many times I will have Canvas notifying me of tasks that I haven’t delivered yet, and even though I understand at once that they aren’t a part of my class, it puts quite the fright in you. You don’t want a notification saying «not delivered» on your frontpage all the time. That for me is quite irritating. I do also feel the interface in Canvas could need some work, it is just too much stuff going on there and makes it kind of hard to find what you’re looking for. I think some video guides would be really useful for first time users.»

Q: «Have you participated in any Canvas learning courses?»
A: «At the start of the year they tried to show us how Canvas worked, but the teacher doing so had a hard time finding stuff himself, so it was a kinda poorly presented and did not give us much information. Didn’t get really much out of it, so we had to look around a lot by ourselves to find things. The same goes for when teachers shows us print screens and so on from Canvas from their side of the view and tells us to do something that looks completely different on our view of Canvas. A «show as student» button would be a nice thing.»

Q: «What do you think Canvas does best?»
A: «What is Canvas good at? It’s a good file sharing tool in class and to give students their assignments. Of course, it’s kinda messy to find the correct ones at times. I’m not even sure if it there is a sorting button there, but if it is, it is hidden quite well.»

Q: «Where do you use Canvas the most? On you computer or your phone?»
A: » It’s mainly the app for Canvas on my phone that I use the most. When I’m sitting down and working on a assignment I usually tend to use the computer.»

Q: «Is there any reason you favour the one over the other at certain tasks?»
A: «I think it’s easier to use the app on the phone, and it’s more organized compared to how it looks on in a web browser, as it’s a bit more compact. When it comes to the files, they are more organized on the web browser.»

Q: «Do you have any wishes for what Canvas could include in the future?»
A: «I don’t really have any wishes at this time. A sorting button to sort between deadlines, when things are due and such. A multi-sorting button.»

Q: «You said you had used Facebook and such for group projects before. Any reason your group choose to use that line for communication?»
A: «The main reason we were using that platform for communication was that it’s a platform we used a lot more regularly, as most of used it in a social setting as well and made all of us available more often. I gather Canvas is only used when people have their minds set on studying and might leave it at that, making some information either go very slow innside the groups or that some information would even be lost to some. Again, most of us were using Facebook for social activities as well and had a higher chance of getting the info to each other at all hours.»

Q: «If you had access to all the chat functions and group functions that’s on your social apps, would you be more active at Canvas?
A: «Well, it’s something about Facebook and those things that gives you the opportunity to be 90% focused on work and still have that 10% for other social interactions. It’s both hard and draining to sit down and work a 100% with your work, and I feel the mindset when doing so is very demoralizing. By putting all the functions we use in Canvas we might be more focused on the task at hand, but our energy drain and the feeling of logging into Canvas would be harder for me to do. A small part of my brain would probably tell me to check up on the social activities on other sites while I’m logged into Canvas if it was that way. »

Q: «Have you used any digital tools to work in group projects before?»
A: «I’ve used Microsoft Teams to some extent. In high school we mostly used Google Docs and that worked very well. I tend to use to this day as it’s free and is very easy to use. Co-operating and co-writing in Google Docs works well.»

Q: «What feelings do you recon with logging into Canvas?»
A: «Well, i feel like i usually comes into Canvas with a lot of motivation to work that quickly decreases as I’m having a hard time finding what i need to find to start my work.»

Q: «Thank you for answering all of our questions, and we hope your studies goes well this fall.»
A: «Thank you, and too you, as well.»

Findings: Through this interview we found out that some work could be needed on Canvas regarding the functionality of finding what ones is looking for. «A multi-sorting button», as Eric pointed out, might have some usage at different places in Canvas. A way to remove tasks that are inactive for your class might help improving organizing. Overall it seems like Canvas have to many options for new Canvas users. Easy acces demo videos of what and how to do things at the front page might be an option. Overall the subject was pleased with Canvas, but most of their problems seems to be fixable trough organizing it one way or another. The idea of having a possibility to easily change their mindset from work to leisure seemed to be a factor in using Facebook and other such ways of communicating inviting when working on group prosjects – is this something we can help avoid trough means in Canvas? Mainly it seems from this interview that the organizing part of Canvas might be a good place for us to start looking for im provements.