This section of our blog will cover our findings in the correlation to testing of our prototypes done by both us and other participants. We have been doing thorough testing of the Adobe XD project and have tried to integrate as many functions we deemed necessary to give a glimpse into what our learning platform could provide both teachers and students alike. With a teacher controlled platform it enables them to monitor who’s actually doing things in class and to decide where the different game modules can be used. They also have the admin rights for the curriculum and are able to add what they want to the different modules and have an information mark that will let them know how to use it. For the students it will be more like Canvas in which you will see what tasks you have completed in what subject and what’s remaining.
Testing is our chance to learn what our users can get out of our solution. How will they benefit from it and what can we learn from their feedback? This is what the test mode is for. Getting constructive feedback on our prototype and learning what needs to be done to make our product better. This is our chance to make empathetic gains for the people we are designing products for. We want to know more about what different people think about the product, why do they think the way they do and how do they feel this product could be relevant to them. The reason for our extensive questioning is in order to help refine our prototypes and solutions. We will learn more about our users and through their opinions be granted a better view of what needs iterations.
With our prototype we want to test:
- User friendliness/user acceptance.
- Do interface testing for our graphical interface.
- The integration of such a platform in class.
- The core functionality of our game modes (in this case being the quiz).
The main focus we want to have is on the user friendliness and how the graphical interface feels for them to use. As the Adobe XD prototype is much more of a visual prototype than it is an actual prototype with proper functionalities, this is even the more reason observe and learn from our participants reactions and viewpoints. We’ve been testing it several times ourselves and have tried to spot the flaws in it, but our eyes are not the ones we’re supposed to please.
Before we initiated the testing process with people, we wanted to have a plan on how we were gonna make them test our prototype. We wanted to use the «Think aloud technique«, which is a method in which the participants will verbally be explaining what they are doing or thinking out loud during the testing of the product. This is a relatively easy way of gathering data and helps us understand the thought process of the tester. We will be on the sideline, only talking if talked to and making notes of what the person is saying. We ask them to play around with the Adobe XD prototype after telling them that it is designed to be a learning platform with integrated modules for different types of classroom activities such as quizzes. Without telling them much more, they’re able to figure it out themselves and get a less biased starting point. We will be there to help them if they got any questions about our prototype during and after their testing process. As well as alleviate any questions they might have about functionalities that are or aren’t there. We are there to show not tell. The information they get from us and the scenario down below is the only context they will have for testing process. We will be keeping this scenario relatively short as it is a reiteration of previously made scenario from 3.2.
Isabel is a 25 year old girl from Sarpsborg and she dreams of working within games and film/tv production. She wishes for more games and fun activities in class to make subjects more engaging for her as she tends to daydream a lot if she gets bored during lectures.
This is the role we want our participants to relate to, try to view things from Isabel’s point of view and we will be acting as the teacher, in a sense, as we have the answers to hopefully all the questions they might have about the prototype. Only one or two of our group members will be explaining and acting as the teacher, while the two others will be pure observers of the ongoing testing process. Our observers will be tracking the time of our participants and more focus on what kind of body language they have. We know what we have included and what he have not included in our prototype, as it is not a fully completed prototype certain functions will not be available. Adobe XD is rather limited with its functions and therefore some of them are just there visually but not available for showcasing through XD. For the testing process we made a list of what we wanted them to do with our test:
- Testing both the student and teacher interfaces seeing what they had to offer.
- Clicking on different buttons to see where it led and get a better understanding in how to look up different material.
- Try the quiz we integrated in the prototype.
- Tell us how they feel when it comes to games in learning
- Does games motivate you, or make you more engaged in what you’re learning?
We decided to gather information from three different test subjects for our evaluation report. We were rather selective with who we decided could take part in the tests as we wanted to get feedback from various sources, not just the classmates we have. The three test subjects we have therefore have both varying ages and relates to the prototype in different ways entirely (we hope). Their input can vary most likely, but with our the predefined goals of our test we’re hoping they’ll give us unique feedback when it comes to our «gamified» learning platform solution. Going by the list above and our scenario is how we’d introduce them to our prototype and our goal of their testing procedure. We asked of them after their testing to focus the feedback on the graphical interface setup and on how user friendly they thought this prototype was as these were our main functionality focuses for our prototype. We performed our first prototype testing in school for the most authentic experience possible for a learning platform such as ours. With the exception of two tests which was done via email/Discord. Our participants had this to say about our product after testing it with Adobe XD:
Person 1: Fellow student
The first person we asked to test out our prototype was a fellow student from our class. She is 31 years old. Our testing process was done as written above. We had one monitoring as a sideman/observer taking in all kinds of information while we had one active role in the form of a «teacher» showing her how it’s meant to be used if she had any questions. She started with the student login and had some slight issues understanding what she was supposed to do as she thought she had to create a user, but since it’s Adobe XD it’s all just fake so we had to show her how to move to the next panels. This was a reoccurring issue with this test subject. Our teacher role, being Martin, had to show her the intentions of the panel triggers on the XD prototype and show her what we wanted her to see. To me, the observer (Tim), she looked relatively confused and her mouse cursor was trying to activate things which weren’t available, as an example the different subjects (when only design methods was in use).
During her testing she was told to explain what she was doing, but it was more questions than anything else as she’s not too used to technology she claims. She was testing the prototype on a Mac so it doesn’t have the same functionality on the Adobe XD prototype as on Windows. When you left-click on Windows it will highlight the clickable items on the prototype if you’re in doubt. So Martin had to kinda take over at times to really show what was meant for the user to look through. After finishing the test she said that it looks well made and thought through, but the buttons and topics are quite big for a modern product, very 80s she claimed. She don’t really know if games will be a positive or negative thing when it comes to learning, but she does enjoy doing quizzes and getting a sense of accomplishment through doing well on said activities or games. Getting trophies or other rewards does make you feel good, a sense of accomplishment is always positive for motivation so it’s at least gonna have some positive outcome if the games are taken seriously and done as their supposed to.
Person 2: A friend
The second person we asked to test our prototype was a friend of one in the group. He’s completely unrelated to our classes at school and lives in Germany. We thought his input could provide some fresh new information or ways to look at the product. He’s 22 years old and is a biology student in Heidelberg. We did as explained above but here we only benefited from his written words, and can’t use the observation method. However during his testing we were available to answer any questions he might have when going about in the panels. He had some troubles with the login screens as well as he wanted to make a user but it wouldn’t let him, like the person above this was just an issue because it’s not really a website. It’s designed like one, so naturally he’d associate this with an actual forced user interaction. After we told him that he can just log in and it’s just there to show how it should look, he went ahead and tried out the prototype. He said it looked neat and he’s made something similar like this before, he thought of the design to be slick, but thought the text fonts was quite big and could be reduced in size. He jokingly asked if we had bad eyesight. Kinda looks like it was made for older people he said, but the detail in the profile panel and the amount of things we had taken into account for in our prototype was decent. When it came to his feelings relating to games in learning he was very positive, he’s a big gamer himself and for sure thought it could be interesting to see games more in learning. It makes him engaged and it feels rewarding when you perform well and you get rewarded with different types of achievements or whatever makes sense given the task/game. Proper reward structures alongside fun activities that relates to the subject in play can for sure be a good motivational factor.
Person 3: A mom
The third and last person we asked was a mother of one in the group. She is 63 years old and is a engineer and former marine. This was to try it on a different age group than our own. She did had some troubles with using it to begin with as she tried to open the prototype on her phone, but when we asked of her to use it on a PC or Mac it went better than expected. She looked through the different pages and saw what our platform had to offer. Like our other subjects we asked of her to keep three points in mind when looking at our prototype, being the ones above the Adobe prototype link in this blog post. She provided direct answers to the things we asked. During her testing we were available for answers if she needed them, but she only really had trouble trying to get it to work on her phone which we told her not to do. On this test subject we also tried to heavily tell her that it is just a prototype so she would not get confused by the prototype. She said that our prototype was user-friendly and that it was easy to navigate between the different panels for both student and teacher alike. She claimed that the headlines inside the subjects panel and such could be bigger though, as it’s more focusing towards the pictures instead of the buttons themselves which seem rather hidden.
The answers she provided was this:
- She tested them, both the student and teacher panel, and thinks the prototype is both user-friendly and orderly. It was easy to maneuver between the different menus.
- She clicked on the different buttons, menus and lesser topics and it seemed systematically built up. This could be a useful tool for student and teacher alike when it’s fully developed.
- She tested the quiz and managed to receive a trophy for her hard work!
- Her feelings when it comes to games in educations is rather positive, she mentioned that the completion of the quiz awarded her with a trophy which she said felt good. Having such a reward structure will surely be a positive when it comes to introducing games in learning.
- It will surely give some motivation in learning collecting a trophy or reward like I did after doing the quiz, and it can also be motivating to see if you did better than others in your class.
The feedback we have received shows the faults or flaws with our prototype providing us with useful information we can benefit of to make our prototype better for our users. There has been some common occurrences from our test subjects, mainly being from the «finished look» of our prototype. All of them have to some degree had issues with the Adobe XD prototype as they’ve perceived our product to be a fully functional website, but that’s quite far from what it really is. Going by this we should’ve been more clear on what it is you can do to our test subjects as we only really told the third test subject the importance of only having clickable items on the prototype, and not full-fledged login services and panels for each and every subject. Maybe we could have designed it more clear as our product looks finished but it isn’t really like that. We originally had «Placeholder» written in places before we added real subjects to the subject panels. This could maybe have been a part of the prototype we got them to test as to not cause confusion with things that aren’t prepared for testing yet.
It also was mentioned that our product is seemingly made for older people going by the big buttons and large fonts. This was more a choice made by us to make it a bit simpler to showcase what we could have there, not that it would necessarily have this really big button/text featured in a finished version. As most of our buttons, tabs, etc. was only to inform the test subjects of what we could have there. It could be more buttons, tabs, etc. that was smaller and filled more in that way, but it didn’t really seem necessary for the test. For a finished product this is great feedback, giving it a more modern look, maybe getting some inspiration from websites like Reddit which has a much more minimalist look (smaller buttons, fonts, tabs, etc.).
Our test subjects only really did take a proper look at the student side of the platform, and just really glimpsed over the teacher one so no big feedback came out of that. We should most likely have added another task for them to do that related to the teacher panel. Our prototype is mainly directed to students as our solution is meant to make the everyday life of a student more fun and engaging; giving them more ways to interact with school material other than just books and writing.
In hindsight we definitely should have made it more clear as to what was clickable in the prototype, as the finished look in coordination with only a select few of the features being available makes the prototype less easy to explore. Only one person for example saw on the teacher panel that you can both activate and disable timers and modules, but also the fact that we’ve added a description in the information button on how to activate said timers/modules. By having arrows or something of the sort this could’ve been easier for them to understand instead of having six of said information button and not knowing which is activated (if they don’t use the left-click trick in the teacher panel). We should’ve been more strict with our list above in mind, actually trying to get an answer of sorts to each of the tasks, instead of relying solely on oral feedback from our two first test subjects. Also briefly mentioned by our last test subject was the possibility of comparing to others which is a real good idea we should be taking into account with our product; as this can both make for a fun and competitive scene, as well as enable the option to create highscores and similar boards for people to aim for the top!
Going by all this feedback and reflection we’ve done, we have a more clear vision of what our product could and should be like. It needs a more modern look. The big buttons, fonts and such needs to be replaced by a more minimalist style. Most other modern websites generally take advantage of this style of webdesign, and we shouldn’t hesitate to follow the trend. For our finished product we also need all of our functions to be available, as to not cause confusion like our prototype did in some cases. It needs to be a functional fully-fledged website with all our different systems integrated. All the things that are shown visually, should be there as interactive functions. Due to our testing and prototype results we have gone back and restructured and fixed a lot of our previous work to fit in with the more finalized version of our idea. The whole process of testing is a way for us to iterate our design. Completing the full process of design thinking is what makes for good design as it establishes a good connection between the user and the product, and through their feedback enables the iteration process which is fundamental for great design.