Testing is done to receive feedback from end-users of what works and what doesn’t with a prototype. By testing the prototype, one can go back and modify it based on feedback.
We have conducted tests with 5 students with different studies like informatics, economics and student teachers. Testing duration with each and one of them was around 15 minutes, and the whole testing process was recorded vocally, with permission naturally. The first two testers were conducted in a group room, and the three others at Vrimle.
What functionality / form / concepts are important to test? How should these be tested?
- Concept testing – The need/importance of the solution.
- Usability testing – Functionality of the solution itself
- Location testing – Where the physical part of the solution (TV) should be placed
We started of by explaining our solution briefly to the testers, followed by showing them the video prototype from the previous phase. After this we asked the testers the following questions:
- Does your school offer a solution like this?
- Do you see any need or importance of this app?
They all replied with that their schools didn’t have anything similar to this app, but all of them saw a big need and the importance of this solution at campus because of various reasons. Some of these were:
- Because of how late you find out about events on facebook. Very easy to miss out on these
- Important for people who’re not social
- A lot of posters are just hanging around on campus without attracting attention
After receiving these answers on the need & importance of our solution, we asked the following:
- Would you be happy with this app if it existed on your campus?
All of the testers either nodded or replied with a clear yes. They thought the app would be great, as it would give a better ovewview of what’s going on than facebook. In addition to this, our app would help them plan better, make new friends as well as participate at more events.
After we were done with the concept testing, we wanted to let the testers actually play around and experience our app. This was done by letting the testers use our Adobe XD prototype on one of our PCs to test out all the functionality. We had prepared a small list of tasks for the different testers to try out, which looked like the following:
- Browse through the calendar (months/dates) and read the information about several events
- Sign up for an event in November
- Confirm sign up by finding the list for all events that the user has signed up for
- Create an event
- Confirm creation by finding the list for all events that the user has created
- Sign off from the event from task 2
- Log out
While the testers were doing all of these tasks, we asked them to talk out loud about what was going on inside their heads. Luckily for us, none of them found it difficult to share their thoughts and opinions at the same time as performing the different tasks. When they were talking, we remained quiet and took down notes. This is called the think aloud technique/method.
When the testers had completed the tasks, we asked the following three questions:
- How did it feel to use the app?
- What already existing functions could be improved?
- What functions do you miss?
Based on the think aloud technique as well as the three questions asked above, the testers found the app very easy to use, as it doesn’t include a lot of unnecessary things. The functions already baked into the app were very understandable and usable, however, there were some functions that could be added in a future version. Some of the suggestions were to add:
- List of names for all attendees for events, to see exactly who is attending an event
- Week numbers in calendar
- Different coloring for different types of events. For example sport events could be green, while parties could be yellow. Also add functionality for filters that lets the user only sort for one or two types of events, for example board games and sport if not interested in parties.
As our solution is not only an app, but also a TV showing the calendar for events in different months, we wanted to simulate this for the testers. Because of this we borrowed a TV-screen with wheels from HIOF, and planned to place this in Vrimle to attract attention. Unfortunately all testing was not conducted at Vrimle because of having to wait for approval from the reception to stand there. Because of this, the first two tests were conducted in a group room. After receiving approval from the reception, we moved the testing to Vrimle. This is how we attracted some of the testers.
We manually controlled the calendar switching screens/months every 10 seconds, as we wanted the testers to believe this was happening automatically. The screeen switching on the TV was naturally being operated by us secretly, which the testers couldn’t even tell. This technique is called Wizard of Oz testing.
After the usability testing of the app was complete, we wanted to test the TV screen as well. We asked the testers about where the TV/TVs for this solution could be placed, as well as if they could come up with ideas for improvement for the TV/s.
Most of the testers suggested that the testing area itself (Vrimle, outside the cafeteria) was the perfect location for the event calendar TV to be in, while the other testers suggested that the TV should be placed next to the information desk at the entrance of the university. All of them said that the TV should be at overhead height, as if it was any higher it would make the QR-code unscanable.
One of the testers also suggested that each event on the TV calendar should have its starting time next to it, so one wouldn’t need to open the app to check when a specific event is starting.