Testing Evaluation Report

Introduction

There are two thing that needs to be tested for this idea, where at this stage one is more important than the other. The two things that will be done is a Concept testing, which is the most important one, because i need to know if this is regarded as a solution to the problem by the users. The second thing to test, while important but not as important as the concept testiong is the user experience of the prototype. This will involve testing the UX but also the functionality that is currently in the prototype.

The goal for this testing is to see if there actually is an interest in this type of solution, or if the tester thinks that this can be a solution to the problem definition in the first place.

How these will be tested:

Concept testing:
The Concept testing will be conducted with one Norwegian speaker and Two Foreign Exchange students. These participants are the people that would be using the app. Even though the topic is specific for Adult refugees any person wanting to learn Norwegian can use this app so these are good people to test the concept on.
The testing will be done by showing the participant the Video Prototype, after they have watched it i will give them some prepared questions which are meant to get an insight in their thoughts about the app.
These questions are:

For both types of testers:

  • What are your initial impression of the app?
  • Would you use it? Why \ why not?
  • Is there some functionality would you like it to have that is not currently planned to be included?

Specific for the foreign exchange students:

  • Do you think this app would help your Norwegian learning?
  • Would you want guidelines for what you are supposed to talk about, or keep it completely up to the two people in the conversation?

Specific for the Norwegian speaker:

  • Would you want to use this app to help someone learn Norwegian?

Results:

There were some interest in the solution, the foreign exhange students thought this app would be great if they did not have anyone that spoke Norwegian to talk to. The Norwegian tester however felt that there was not enough of an incentive for Norwegian speakers to use this app. He thought that the Norwegian speakers could easily just be «used» and they maybe needed more of a reward to use this app.

The users all agreed that there should be some guidelines to what you should talk about, this was commented on as they felt that giving some rules to a first encounter would maybe make the apps purpose more clear. This would also discourage people from not suing the app for its intended purpose.

Recommendations:

One of the testers recommended that for a first encounter there should be a clear question, or something specific that the Non Norwegian speaker wants help with. They basically wanted the first encounter to have a purpose so that the app wouldnt just become another chatting service. They also clarified that when you have added eachother as friends on the app you should be aple to talk about whatever you want.

One of the testers thought that maybe after each first encounter you should get a question if you want to add the person you just talked to as a friend. This was so that there would be more of a mutual agreement to be friends between the two people.

UX testing:

The user experience (UX) will be tested on two IT students which are studying for different bachelor degrees. The test will be done with the prototype made in adobe XD, which has been uploaded uploaded to adobe cloud so that the test can be done on a smartphone instead of on a computer screen. This will make the test be more natural as the app this is the platform the app is designed for.

The tester will be asked to perform three tasks that is going to get them through the entire functionality of the prototype. I Will also instruct the tester to think out loud every action he or she is doing when trying to complete the task. When evaluating how well the tester did these tasks i considered time but most importantly where they clicked in the time used. It is important to see if the icons or the placement of these icons resonate with the user so they understand what page they will be redirected to when they click on it.

These tasks are as followed:

  1. Connect to a Text chat.
  2. Add a user as a friend.
  3. Check the friendslist and see who is online.

Results:

The testers flew throught the app with no problem, completing each task without pressing wrong once and in no time. Since there was no problem i asked them «What functionality do you think could be added?» since i believe that the small amount of pages on the app is what made them have no problem with it, which is a good thing. There where ofcourse some comments about the overall design. It was noted that the bottom and top bars where too bulky and should be slimmed down.

Reccomendation:

Fine tune the design to that the top and bottom bar is not too bulky and is more similar in size as other apps.

Add an option to see how many people are in que to talk, this was recommended as there might be times of the day where there are less users on. This is important so that a user can see if they have time to wait for someone or not.

Conclusion:

The concept itself is desired, and is thought to be a good addition to the language learning methods of the testers. However the Norwegian speakers might have a lower amount of users than the non-Norwegian speakers. As for this an incentive should be presented. One initial idea is to advertise this app both ways, so that there is an emphasis on the ability to talk to people from different cultures for the Norwegian speaker.

As for the design and functionality, there are some functionality that should be added. For example the option to see how many people are online and searching should definetly be added. Also the notes about the design should be fixed and fine tuned so that it looks more like how a user would expect from an app on their smartphone.

As for future plans for this project the prototype should be more fleshed out and fine tuned with the functionality and the design that where recommended from the testing. Then there should be a second round of testing which includes more test subjects to find out what more people think about the whole idea. This should be done as three people is not enough opinions for justifying the start of developing an app. However its enough to not completely dismiss this as a solution for the problem definition and continue the developing.

 

Prototype

The prototype created is a high fidelity look and feel prototype. My goal was to illustrate how the app were look, what pages i wanted to implement and how these pages would work together. I wanted really to have a minimalistic look with as little steps as possible to get from A to B. Early on i chose a color palette which is used throughout the entire prototype, including logos and icons for the app.

The functionality of the app is simply a way for people to connect with someone, either through voice chat or text chat. One tap of the buttons on the main screen searches for a person to talk to. There is also a possibility to add this person as a friend after or during the conversation, this makes it so you can call this person directly. There is also overview pages for all the chats you have had and the friends you have added on the app.

The app has these following pages:

  • A home screen with buttons to serach for a random person to have a conversation with.
  • A text chat screen.
  • A voice chat screen.
  • A screen with an overview of all text chats.
  • A screen with all the people the user has added to their friends list and their status (online, offline or busy).

I wanted the home screen to be a gateway straight to the main functionality of the app, this is where the user will search for a random person to connect with. It was important that there where no steps to get to these two buttons. Theres always buttons available for the user to go back, for example it can cancel the search for a conversation easily, i did not want to trap the user. The whole point of the app is to have someone to talk to so you can practice the Norwegian language, so simplicity in user experience was a must. I don’t need to reinvent the wheel, people with smartphones have used text chat and called someone before so i didn’t want to make anything too complicated.

Overall i am happy with my prototype, however the apps functionality is very much up to the user. This app in simple terms works as a middle man for connecting two people, one with the goal of learning Norwegian and one who already knows the language and want to meet new people from other backgrounds, get a new friend or simply help.

Video Prototype:

https://hiof.instructuremedia.com/embed/7ca86e21-663b-457d-b52e-5f0c540b0256

 

Personas, Scenarios and Storyboards

Persona – Naida:

Scenario:

Nadia is done with school for today and has been home for a couple of
hours. In class her teacher used a metaphor which she didn’t fully
understand, she understood the words but is not sure about what
situation it should be used in, or if it has a hidden meaning.
But she wants to find out.
She doesn’t have many friends who are Norwegian or Norwegian
speakers. So she uses the app which lets her connect to a random
Norwegian speaker. This is the first time she uses the app so she
chooses to chat over text rather than over voice chat. She connects to
Stian who is the same age as her. After they exchange greetings she
asks him about the metaphor. He explains what it means and what
contexts its usually used in. They chat for a bit and Nadia feels that the
conversation is flowing in a nice way and she feels comfortable talking
with Stian. She wants to add him on the app so she sends him a friend
request in hopes that He accepts, Which he does! Now when Nadia
wonders about something she thinks she will just ask her new «friend»
Stian. She is so happy she doesn’t have to introduce herself to lots of
new people and that she found a good tone with the first she connected
to.

Storyboard:

Persona – Ali:

Scenario:

Ali is coming home from work and is sitting in the back of a train cart, there is no one around him and seeing he has some free time he decides to use the app to try and connect with someone he can speak Norwegian with. Free time is rare for Ali. He opens up and presses the «connect with person» button and chooses voice chat without video. He connects with Martin who is about the same age as him. Ali introduces himself to Martin and Martin does the same. They end up just talking about their day. They work in different areas so there is many interesting topics to talk about. When the train is close to the station Ali says his goodbyes to Martin and closes the app. Ali now feels happy as he managed to find some time to practice Norwegian. When he comes home he boasts to his wife who is happy for his husband who is learning.

Storyboard:

 

Define your research questions

I chose to use the Affinity Diagram method when defining my research questions. The reason for choosing this method is because much of the information that was gathered in the empathy phase lead to similar issues or issues that may lead to the same solution. This method then lets me get an overview with clusters of information which could have a similar solution.

The process of setting up the Affinity diagram went like this. I started to write all the issues that the Students have with finding time outside of school to practice Norwegian. I then took the issues which are similar and put them under the a category with a fitting overarching issue, this makes a sort of hierarchy for the issues where you go more specific the deeper you go. Under is the result of the Affinity diagram.

As presented in the Affinity diagram there are two main issues which can be looked into. The first issue is time, and how little of it some people have. This means that a solution should be something you can do in a short amount of time if needed, so that if a user suddenly has 5 minutes of free time that person can now get some practice in that amount of time. The second issue is that people, for different reasons, may not have a social network of Norwegian speakers outside of school.  This means that the solution should help the user connect to Norwegian speakers.

The problem definition then becomes: Students not using Norwegian outside of school.

 

Generating Ideas

Problem definition: Students not using Norwegian outside of school.

How might we questions:

  • Inspire students to want to use a new language.
  • Engage students in Norwegian conversation outside of school.
  • Help students with little time outside of school use the language.
  • Reward those who exercsise the new language well / those who work hard

Ideate Process:

After we made some «how might we» questions we started just writing down what ideas we thought of. As we were only two members present during this brainstorming (Ludvig and Alexander) the ideas were few, but good. We only managed to think off three ideas. After we presented our ideas and talked about what the solution is meant to solve, we gave each idea points, maximum 5 points and minimum 1 point and we couldnt vote with the same points on multiple ideas.

Ideas:

  1. App that connects people who learn and know norwegian so they can talk to eachother either via text or voice.
  2. A type of registering with a leaderboard where you can register the books in norwegian you read where you can receive rewards.
  3. Norwegian Language learning app like duolingo which are for the languages the majority of refugees speak.
  4. Microphone that records when you speak and recognizes when you use Norwegian. Then it will analyze those parts so that you can quickly look at what you did good and what you could improve on.
  5. App that gives you different challenges to help you be more active with the leanguage you are learning and help you step out of your comfort zone.
  6. App that scans text you have written in norwegian and analyzes it for grammatical or sentence building erros and shows you how it should be written.

Idea we chose:

Number 1: App that connects strangers, one who is currently learning Norwegian,
and one who is fluent in Norwegian so that they can talk together. Either through voicechat or text chat.

The problem we believe this app will solve is students who might not have a social group outside of school where they mainly speak Norwegian, or that they have trouble getting to know fluent speakers. This app will act as a social media where you can connect to a random person and have a conversation with them. And if you find a person who you get a good connection with you can add them as a friend on the app if you want to continue the conversation.

 

 

Expert Interview

Interview Guide

  1. How does adult refugees experience learning Norwegian?
  2. How do different age groups experience the learning process?
  3. What are the biggest issues for young adult refugees while learning Norwegian?
  4. How does a pre-knowledge of the English language influence learning Norwegian?
  5. What are your thoughts on things that can be improved?
  6. What impact does social interaction in Norwegian outside of the class have?
  7. Do students generally succeed?
  8. Are you happy with the level of fluency that most of the students reach during the course?
  9. Have you heard of digital apps like duolingo etc, how do you think they supplement the course?
  10. Would you recommend more people use duolingo or a similar app, Why? Or Why not?

Who was interviewed?

The interview subject is Jon Bjerkan a Norwegian Teacher who has taught Adult Refugees.

Where was the interview conducted?

Both of the participants in the interview where in the comfort of their own home using digital video chat.

How was the interview Conducted

The semi structured interview was done using digital video chat. Omar from the research group was conducting the interview and the interview subject, Jon Bjerkan, was participating. A screen recording was made on Omar’s computer.

What was the theme (s) of the interview?

Understand how Adult refugees learn Norwegian.

Why was the theme(s) chosen?

The themes were chosen because we wanted to get an insight in what issues there currently are in the situation of our topic. The person that was interviewed has an overview of how multiple people experience this situation and could be able to find an answer or suggestions to improvements for more people as his experience does not differ as much from personal experience as the students might be. We also wanted to know his thoughts on supplements to a Norwegian Language course, how they work, how they could be improve and how much of an impact these have in motivating and activating the learning process outside of class.

Transcription

Interview transcription (attachment)

Analysis and summary of findings with quotes

Learning Norwegian can be quite challenging. But it really depends on the person, and their background. Whether they have had the experience of learning another language, or not. Students who have learned another language have an advantage because they have the experience but also knows what methods and learning techniques they can use when learning Norwegian.

However, the young students seems to have an advantage. Bjerkan states that he believes the younger students learn faster. And he also states that it is easier for them to get to use the new language. Some of the older students seems to have more trouble getting into the new language and ends up helping each other in class, using their mother tongue.

But regardless of age, students have different challenges. For some, writing is easier. And for other, speaking is easier. Many of the students are also struggling with understanding the language as a whole as well.

Those students who already have experiences with the English language seem to have an advantage. They are more used to a similar language to Norwegian. Both English and Norwegian have several similar words in the wocabulary, and the grammar between the two is pretty similar.

In order for the students to learn more efficiently, they have to use the new language in their spare time as well. Using the new language with friends and family will have a big impact on the learning process.

A good thing to know is that most students generally succeed learning the new language. Bjerkan states that if the students happen to get a job, where they are engaged in using the new language, they will get more motivated to keep using and learning more of it as time passes.

Bjerkan says he’s satisfyed with most of his students. As long as they do their best, and work hard to achieve their goals, he is more than satisfyed.

Tools are also an important tool in the learning process of a new language. Bjerkan claims that these tools are very handy to use on your spare time, when you don’t really have anything else on your schedule anyway. It is a good way to spend some time, in order to improve the learning process. He highly recommends them to any student who are trying to learn a new language.

 

 

Cultural Probe

  • What is your design goal?
    The cultural probe is designed so that the information gathered will be immediate thoughts after a Conversation in Norwegian, and in the evening after the day is done. This will make the person using the probe reflect of his or hers Norwegian usage throughout the day, but also the first thoughts after the language is used.
  • Who are your users?
    Adult refugees in Norway who are currently learning the Norwegian Language.
  • What is the probe made of (artifacts & tasks?
    • Artifacts
      • Notebook
      • Pencil
      • Stickers
    • Tasks
      • During the day
        • After a conversation in Norwegian fill out how you felt you mastered it on a scale from 1-5.
        • Comment about what went well in the conversation or what you want to improve on.
      • In the evening
        • Write down their use of newly learned words or phrases.
        • Write down your favorite word you learned or used today.
        • Put on a yellow star sticker for x amount newly learned words.
        • Circle around «emoji’s» that represent how they generally felt while speaking Norwegian that day.
        • Write down a goal for the next day. If the goal for the day was completed put up an extra sticker
          Goal examples: Better expression, Learn more phrases \ words, engage in conversation in Norwegian.

  • How do you plan to follow-up your probe?
    Collect the probe in person (if possible) and thank them for doing the tasks.
    Ask them if they want to be informed about the findings off the data in the probe.
    If yes ask them how they prefer to be contacted.

 

There is no information gathered from this probe as the group i was in didn’t hand them over to anyone after i made and gave another group member the notebooks before i left the group to start working in a one man group.

 

Literature Review Summary

Main elements/characteristics of my information flow situation

When attempting to teach Norwegian to young-adult refugees, a good teacher is what most people would consider. But what if there was better and faster ways to teach a new language?

Studies have proven to us multiple times, that being engaged in physical tasks helps learning. This is because when we are active, our brain is working much harder. It’s also memorizing the things we see and experience, much better. In other words, it’s recommended that you change the way you learn, with being more physically active when trying to learn something.

Some teachers are well experienced with teaching, and knows what works and what doesn’t. However, only the best teachers know how to engage students into making the learning process more fun. That way, it will be more exciting for them, and they will often feel the motivation to work harder. This is because we like experiencing getting better at something we are trying to achieve.

When learning a new language, it’s important to use it as much as possible. And if you don’t have too much time to use it in your daily life, there are several apps like Duolingo which provides push notifications and reminders on your phone, to make sure you do your daily language exercises. Using it every single day, is one of the most efficient ways to learn a new language.

Challenges encountered in the information flow

The drop-out numbers for adult refugees may have several reasons. These are refugee adults, who possibly have children witch gives them a financial aspect.

Academic challenges like, cultural dissonance, language barriers, fear or distrust of authority figures, fear of speaking in class and lack of academic support at home may be some of the biggest, and most obvious challenges for an adult refugee learning norwegian.

cultural dissonance causes lack of interest in learning the norwegian language, it causes discomfort or confusion, by people in the midst of a change in their cultural environment. This is relevant for the people who come to norway as a refugee, because they come from countries with a different culture and different expectations set to them. This makes people uncomfortable, and may cause them to not want to be a part of that culture if they feel rejected.

Language barriers can be underestimated. for example if you are a norwegian student learning english, you have a lot of references to use. The student can use movies and TV-shows as a learning tool, and when you watch people speak english you get a better idea of how to form your sentences in english, appose to just learning the words translated. Learning how to form a sentence in a different language like norwegian can be a difficult task, if the refugee don’t have a norwegianspeaking reference to follow in terms of what you see, when a person is learning english, and has an endless supply of movies and TV-series.

The learners have financial obligations leading them to take part-time jobs, witch can prevent the learners from doing school work, and resting for the next school day. These part-time jobs are uselie due to children at home, and they dominate the learners sparetime.

What needs to be improved

First of all, Norwegian teachers should understand that their students come from many different backgrounds and ages, and not all of them are of equal learning abilities or have the determination to learn. Some of them want to learn the language as fast as possible so they could get to work or to continue studying. Others, just don’t care at all about the whole learning process or the outcome of the Norwegian course. The result of this is that those students who don’t care, don’t wish, or simply don’t want to make an effort to learn the language as effective as the others, will eventually, negatively affect them.

There is one simple solution to what should be done here. Students have to be sorted into the classes they deserve. For example, a class dedicated to hardworking and serious students, and another class for students who like to have fun and play most of the time.

Second, It would have been really nice and more helpful if the teachers were multilingual. Sometimes no matter how hard you try to explain a word to someone, it never makes sense until it is translated into their mother language, or a language they know well. It is not necessary, but it would make learning the language easier.

An overview of available digital tools and what problem each of them addresses.

Currently the best digital tools for learning norwegian according to norskbloggen.no and lifeinnorway.net is Memrise, Babbel, Nemo and Duolingo. How you access the guides differ, they all have free and paid models. These are all apps which can be used both on apps on a smartphone or a website on a computer, which make them easy to use on the go.

Memrise is an app that uses memorization and visual techniques as its learning component. Memrise is made to fit into a busy day as a supplement to a language course. It is meant for being used whenever you have a free moment in your day so you stay fresh on the language you are learning. This app also has a goal to make the learning experience feel more local as opposed to make you learn the language in a strict formal way.

Babbel also aims at making your conversational skills in the new language better. They try to be up to date with the current way of speaking to each other, this makes the learner speak more naturally. They also want you to speak correctly. Babbel also offer review sessions of words and sentences you have already learned, but in new contexts so that you can reinforce and refresh your memory of your knowledge.

Duolingo is one of the most known language apps on the market, it’s a good app with a whole bunch of languages available for learning, but most importantly; its free. Duolingo claims themselves that using their app is the best new way to learn a language. Its designed to be a fun and addicting experience, it also gives you immediate grading which is good compared to waiting for a grade to know what you have to improve on, wasting valuable learning time.

 

Babbel costs money after the first course and memrise works with a subscription fee. The one drawback is that duolingo is inherently free, but you can only do so much of a course before you have to wait, unless you want to pay for duolingo plus which removes ads.

All of the apps mentioned are best used as a supplement for language courses, as they are great at refreshing your pre acquired knowledge. But one big issue that they have been critiqued for is the lack of grammar learning. Grammar is especially important if you want to also learn how to write in the desired language. For example french and Norwegian grammar is very different, and a french speaking person might not know this and get a bad learning experience. But that does not mean you can’t learn grammar using these apps, it’s just not going to be as quick and short as their other lessons as some grammar can be quite complicated and need exercises both verbally and written for you to fully understand it.

All in all these apps are designed to be used for short periods of time when you find the time in an otherwise busy day.

 

Documentation of the literature search

Our chosen topic is to look at adult refugees coming to Norway and their process in learning the Norwegian language. When we searched for literature that could help us gather more information on this topic we set some goals and questions that we would like to find the information necessary to achieve or answer.
These goals and questions are:

  • Improve the process of learning norwegian when coming to norway as an adult refugee
  • How does it work today?
  • How can we make it better
  • How can we support current solutions

The keywords we used to search for literature are the following:

  • Language
  • Learning language
  • Learning Norwegian
  • Adult refugee
  • New to norway

With these questions and goals in mind, we searched for relevant literature for our topic. We did not limit ourselves to only scientific books and research reports but also allowed ourselves to look at newspaper articles as interviews with individuals in this situation will give good insight into how they feel in this situation. Google and Google scholar were used when searching for literature with our keywords.

The sources we selected are:

The reason we chose these as our sources:

is that they provide insights into our information flow topic from different perspectives. It doesn’t only look at the learning situation but also the differences in cultures, which is also a huge part of integrating into the society. One big point is also the social aspect which is proven to be one of the most effective and important parts to learn a new language.

 

Learning Norwegian when coming to Norway as an adult refugee

We wish to explore the situation that many refugees find themselves in when they come to a new country,  learning the language. Specifically for young adults above the age of 16 in Norway.

These are the goals and questions we would like to be achieved and answered:

Improve the process of learning Norwegian when coming to Norway as an adult refugee.

How does it work today?

How can we make it better?

How can we support current solutions?