{"id":6336,"date":"2019-12-10t18:10:38","date_gmt":"2019-12-10t18:10:38","guid":{"rendered":"\/\/www.catharsisit.com\/?p=6336"},"modified":"2019-12-10t18:10:38","modified_gmt":"2019-12-10t18:10:38","slug":"conduct-a-good-user-interview","status":"publish","type":"post","link":"\/\/www.catharsisit.com\/blog\/conduct-a-good-user-interview\/","title":{"rendered":"no, you don\u2019t have to be a ux researcher to conduct a (good) user interview"},"content":{"rendered":"
small startup with no ux researcher on staff? here\u2019s why user interviews are worth doing anyway – and tips on how to get started.<\/em><\/p>\n spend enough time in startup land and you\u2019ll stumble upon one of the holy commandments of saas (software as a service) businesses: thou shalt talk to your users early and often<\/strong>. <\/p>\n that means not just doing usability testing (putting a mockup or a new feature in front of users to see how they interact with it), but also picking up the phone to learn about users and potential customers before you build, mock, or even brainstorm.<\/p>\n unfortunately, implementing a consistent user research practice is like building a regular fitness routine: we all know we should<\/em> do it because it\u2019ll be good for our short- and long-term (business) health, it\u2019ll give us iterative and actionable feedback that will help us improve more quickly, and it has a tendency to spread positive ripple effects to other areas of our lives (and our companies). <\/p>\n and yet, too many startups do the equivalent of driving past the gym every morning by putting user research on the back burner for an ever-distant \u201csomeday.\u201d if you\u2019re at a small startup like ours, the justification is often a lack of resources \u2014 with a tiny team (that already wears multiple hats)<\/em> and limited capital (that takes a lot of sweat equity to raise)<\/em>, user research can understandably feel like a \u201cnice to have\u201d luxury, one meant to be saved for the day when your company finally makes it and can afford to hire a snazzy ux researcher.<\/p>\n well, we\u2019re here to channel our best richard simmons-esque level of encouragement and let you know that the benefits of asking your customers for feedback are well worth the work, even and especially if you\u2019re on a team that lacks a dedicated ux researcher. here\u2019s how we\u2019ve approached user interviews at magoosh from the very start, how they\u2019ve helped our business grow, and tips and advice for getting started. <\/p>\n we\u2019ve conducted user interviews in some iteration for nearly every quarter over the past ten years. there are three main motivations\u2014and resulting benefits\u2014behind our prolific user research habit:<\/p>\n as a relatively young startup with a global student population, achieving product-market fit has been crucial to our growth and success. <\/p>\n it\u2019s allowed us to make it to profitability on just one seed round of funding, grow our product line from our staple gre offering<\/a> to products helping students prep for over nine standardized tests, and invest in expanding our team<\/a>, fostering great company culture along the way. <\/p>\n throughout our now decade-long journey, our growth has been catalyzed by product feedback from our users, from our recent addition of text-message reminders to our act and sat products, to our 2010 decision to switch from user-generated to expert-written content. today, though our products can always continue to be improved, we\u2019re stoked to see our net promoter (nps) scores overall on the rise, and our student numbers continuing to climb from eight million served. <\/p>\n we\u2019ll be the first to admit that, even amidst the fiercely proud-nerd culture of magoosh, there\u2019s still so much we don\u2019t know! <\/p>\n when we first entered the sat market, we assumed parents would be the louder voices we\u2019d need to listen to. a round of user interviews later, we realized the story was actually much more nuanced: many students were studying on their own, with little or no parent involvement.<\/p>\n it\u2019s in the spirit of one of our core company values<\/a>, learning > knowing, that we acknowledge our blind spots and seek to listen to our students in the hopes of learning what they can teach us or broaden our perspective around. <\/p>\n the world of education is rapidly changing, and even as lifelong learners, it can be incredibly hard to keep up. <\/p>\n this is especially true as the college admissions landscape becomes more complex and less accessible to students and, in line with our mission, user interviews have afforded us a personal, humanizing means of empathizing with our customers and their wildly diverse lived experiences. we would never have known what it\u2019s like to prepare for the toefl in iran if we didn\u2019t talk with people doing exactly that.<\/p>\n it\u2019s important to learn to differentiate between when a user interview is the best path for learning, and when it might actually be a distraction. generally, user interviews are better for focused learning over open exploration, such as talking through challenges and problems with your users. the more specific the info you\u2019re trying to get, the better a user interview can help you inform next steps.<\/p>\n some points to consider:<\/p>\n should it be a user call?<\/strong> define your purpose<\/strong> what does success look like?<\/strong> <\/p>\n so you\u2019ve gotten clear on the information you\u2019re trying to learn and decided that user interviews are indeed the best route. now, who in your customer pool should you reach out to?<\/p>\n <\/p>\n we\u2019ve made a lot of mistakes conducting user interviews over the past 5 years (and discovered a lot of useful habits along the way!). here are seven tips and tricks:<\/p>\n you\u2019ve checked off all the right things (defined a clear purpose and outcome, recruited the best users from your customers, set up a solid process and roles) but find that your questions are falling flat during the interview. what gives?<\/p>\n not convinced you\u2019re framing the right questions during your user interviews? avoid these common mistakes.<\/em><\/p>\n avoid asking leading questions<\/strong> priming is not the same as leading<\/strong> take answers to quantitative questions with a grain of salt <\/strong> covered all of your bases? here are a few bonus tips to be double sure you\u2019re not leaving insights (and honesty) on the table during your user interviews.<\/em><\/p>\n you won\u2019t hurt my feelings<\/strong> i know what you did last summer (or last week)<\/strong> silence is golden<\/strong> starting to conduct user interviews (and doing them well) can seem like a lot of hassle and upfront effort, especially when you\u2019re working with limited resources, but the barrier to entry is not as high as you might think. they\u2019re also a tremendous amount of fun! for us here at magoosh, we most appreciate them as a way to connect with and build even greater empathy towards our students.<\/p>\n even if you\u2019re not ready to lead a user interview project yourself, we\u2019d highly recommend you volunteer to be a notetaker for someone else at your company conducting user interviews, as it\u2019ll help you be better prepared when your time comes. <\/p>\n we hope you find these suggestions helpful, and welcome any thoughts or questions in the comments below! <\/p>\n if you\u2019d like even more resources on user research, here are a few of our favorites:<\/p>\n quick reads: tips for user calls<\/strong><\/p>\n user research interview tips<\/a> – brad dalrymple<\/p>\n the art of the user interview<\/a> – nick babich<\/p>\n how to conduct user interviews<\/a> – interaction design foundation<\/p>\n 6 tips for better user interviews<\/a> – veronica c\u00e1mara<\/p>\n starter questions for user research<\/a> – sarah doody <\/p>\n never ask what they want\u200a\u2014\u200a3 better questions to ask in user interviews<\/a> – chuck liu<\/p>\n longer, in-depth guides<\/strong><\/p>\n interviewing users: how to uncover compelling insights<\/a> – steve portigal<\/p>\nthree benefits we\u2019ve gained from conducting user interviews with our students<\/h2>\n
\nimage credit: thinglass (licensed via shutterstock)<\/em> <\/p>\nreason\/benefit #1 – offer better products<\/h3>\n
reason\/benefit #2 – check our assumptions<\/h3>\n
reason\/benefit #3 – deepen empathy for our customers<\/h3>\n
when – and how – to start conducting user interviews<\/h2>\n
\nimage credit: notto yeez (licensed via shutterstock) <\/em><\/p>\nwhen should you conduct user interviews?<\/h3>\n
\nwhether you are trying to identify a problem, get more information about a known problem, or see if a solution will work, you should consider whether user calls are the best tool to get what you\u2019re looking for. does the information you want already exist somewhere, or are there better methods (like a survey) to collect it? user calls may or may not be the route you want to go.<\/p>\n
\nare you conducting \u201cdiscovery calls\u201d to try and understand problem spaces, or have you defined a problem space and want to do a deeper dive into how users may be solving this problem already? this will additionally inform who you reach out to and the questions you ask.<\/p>\n
\nuser interviews are a means, not an end. if the sum of your user calls leads you to the next stage of your work, and that work is successful, then the calls were a success. <\/p>\nhow to recruit (the right) participants for user interviews<\/h2>\n
\nimage credit: ollyy (licensed via shutterstock)<\/em><\/p>\n\n
\nrecruit users who will provide the \u201cbest\u201d source of information for the problem you are tackling. take into account the various stages of your customer journey (for example, users already aware of your brand who have yet to purchase? trial users? power users? users whose engagement has dropped off recently? blog readers who never engage with your product?) to inform what subset of your users makes the most sense.<\/li>\n
\nworking on your core product? well, if 70% of your customers are domestic and about 30% are from india, then aim to recruit a cohort of users that matches this 70\/30 breakdown. if you can\u2019t get there, do more recruitment, and don\u2019t hesitate to turn some volunteers from over-represented groups down when you need to (kindly, of course!). self-selection is a real thing that can skew the makeup of your pool of users so you\u2019ll need to find ways to target your outreach so that your research is proportionate to your user base.<\/li>\n
\nstart with how many users you ultimately want to talk to. next, estimate the percent of users who you think will sign up and not show. then, the percent of users who will open your email and percent who click through to sign up. play around with this funnel (using historical data on the sign up and open rates of your customers, if available) until you understand how much outreach you\u2019ll need to reach your target number of interviewees.
\n
\nnote:<\/em> this initial outreach number might be bigger than your pool of potential candidates. if that\u2019s the case, you may need to sweeten the deal a bit with (bigger) incentives or build out a more robust campaign to increase open and sign-up rates.<\/li>\n
\nto find useful themes, you\u2019re going to need to talk to more than 5 customers. we\u2019ve found 10 works as a minimum for a lot of purposes, but shorter, less dense calls may warrant 20 or 30. for most interviews, recency matters. aim to recruit active customers from the past few weeks as opposed to the past few years. they are in the midst of using your product and not trying to remember what it was like. oh, and if you happen to be talking with low-intent or low-engagement users, get them on the phone immediately. the longer you wait after booking, the more likely it is that they won\u2019t show up for the call.<\/li>\n<\/ul>\ntips and tools to streamline your user interview process <\/h2>\n
\n
\nhaving a script will help you avoid leading questions, control your time, and stay focused on your goals. but be prepared to shuffle the order of your questions and ask follow up questions.<\/li>\n
\nhumans are actually really bad multitaskers. understandably, it’s hard to have a high-level picture of a conversation when you are in the middle of it. to make the most of each of your user interviews, it\u2019s best to have one person focused on asking questions, listening, and responding while someone else primarily takes notes.be clear about the type of notes each person should take\u2014verbatim, paraphrasing, or highlights. when you have two people, you can take two different approaches. it\u2019s inevitable that you\u2019ll forget some details, so if you have one person taking notes as close to verbatim as they can get, you\u2019ll be able to return to exact phrasing after the fact, which will help prevent false memories. but if there\u2019s only one person in the room, conducting the interview, taking verbatim notes, and seeing the big picture becomes impossible.<\/li>\n
\nwith multiple calls, and potentially multiple people making calls, having a spreadsheet with the users, relevant info, and recaps from the user calls is a great way to stay organized\u2014and know who needs to have a gift card or other reward sent to them.<\/li>\n
\ntechnical difficulties will happen, and it\u2019s helpful to plan for them in advance. when recruiting users, ask for their phone number (or other additional contact method) so you can call them instead of having them call you.<\/li>\n
\nschedule less engaged users as soon as possible in your interview window to mitigate against no-shows. in that same vein, overbook more users than you ultimately need to talk to because people will flake. to account for the faulty nature of human memories, text reminders to users an hour before the scheduled call to verify they\u2019re still available.<\/li>\n
\nideally, you discuss the call immediately after the call, so make sure to reserve time in your schedule before the next one. discuss what stood out and arrive at 2-4 takeaways. if you\u2019re pulling out more than 4 highlights, they\u2019re probably not equally important\u2014try to keep it to only the meaty points so you can focus on what really counts.<\/li>\n
\nsome people can rock through eight user calls in a day. others may find it challenging to do more than a few in a day. plan the interviews so that you don\u2019t burn out.<\/li>\n<\/ul>\ntroubleshooting common user interview challenges<\/h2>\n
\nimage credit: juergen priewe (licensed via shutterstock)<\/em><\/p>\n
\nasking leading questions is the best way to invalidate a response or even an entire call. if you ask a leading question (even one!), you may have to throw out the entire interview, so practice as much as you can to not ask leading questions. no yes\/no questions. ask who, what, where, when, why, and how questions. \u201ccan you tell me about x?\u201d and \u201ccan you tell me more about y?\u201d are good! also, practice might just mean conducting interviews where you make mistakes and the results are thrown out, assuming that you use those learnings to improve future interviews.<\/p>\n
\nit can be helpful to start broad and narrow into the questions you want to ask. start with something concrete and that will make it easier for them to answer questions later. instead of \u201chow did you use magoosh to study for the sat?\u201d, start with, \u201ctell me about the last time you studied a new topic.\u201d<\/p>\n
\npeople tend to search for the \u201cright\u201d answer or provide an answer that you want to hear. so you won\u2019t get an honest response if you are asking for something on a scale. the nps question (\u201chow likely are you to refer\u2026\u201d) when asked on a phone call is likely going to result in a higher score than the same question asked in another medium\u2014your interviewees will try to make you happy, and may not give the most accurate response. these kinds of questions can be helpful to lead into follow-up questions, but they\u2019re dangerous when used on their own.<\/p>\n
\nstart the call with why you are conducting the calls. and tell them you want them to be brutally honest, and that they can\u2019t say anything to offend you. people will unconsciously provide answers that they think you want to hear or that they think true of themselves. prime them with a statement at the beginning that hopefully dampens this.<\/p>\n
\nsimilar to setting the stage, let the user know you have their usage history in front of you, if you do. starting with a quick \u201ci see you spend a few hours using [our product] each week\u201d\u2014or whatever happens to be true for the particular interviewee\u2014can help avoid situations where people claim to have done much more or much less than they actually have. not everybody wants to admit how much time they spend on an entertainment service, and many of us will claim to have used that exercise app as much as we wished we had, not how much we really did.<\/p>\n
\nsometimes the best way to get someone to talk is with silence. let there be a space so people can talk, and think, and talk some more. the impulse to respond right away can be a difficult one to fight, but there\u2019s very often something more that will come out unprompted (with no danger of having led them to it, to boot!)<\/p>\nready? set? research!<\/h2>\n
\nimage credit: yummyphotos (licensed via shutterstock)<\/em><\/p>\nfurther reading on how to conduct user interviews<\/h2>\n