at startups, where time and money are often in short supply, it\u2019s hard to find the resources to focus on reducing hiring bias. it\u2019s really easy to think of bias and diversity as problems that can be solved later. <\/p>\n
the trap, of course, is that the larger your organization gets, the harder it can become to diversify and root out deeply ingrained biases. we\u2019ve struggled with how to approach diversity and bias<\/a> at magoosh and have found success in approaching this problem like we do other business problems: by taking small steps and iterating over time. <\/p>\n i lead our operations team at magoosh, so my team has been driving much of this change over the last few years. based on our experience, i want to share a list of low-effort measures we\u2019ve taken at magoosh<\/a> to reduce bias in our hiring process, as well as an estimate of how long these measures take. <\/strong><\/p>\n my hope is that by showing how a small, lean startup like magoosh has put these steps in place, we might help other companies take the first steps towards decreasing bias. we definitely don\u2019t have it all figured out, but if companies waited until they had it all figured out before taking steps, nothing would ever get done. <\/p>\n before i dive into the tips, it would be helpful to give some background on our overall hiring process, so let\u2019s start there. magoosh has a fairly robust and rigorous hiring process. we\u2019ve described it in detail<\/a> before, but to provide some context, here is a brief overview. (note: below i describe how we hire for full-time positions; our process is different for part-time remote work, although we still apply many of the same bias mitigation strategies.)<\/p>\n now that you have a sense of our process, here are four concrete steps you can take (or suggest to your company) to reduce bias in a lean startup environment. what is it? <\/strong> how long does it take?<\/strong> how does it work?<\/strong> during the questionnaire phase, for example, we ask candidates to not list any personally identifying information on their responses. our recruiter does a quick check before passing the anonymous questionnaires on to the hiring manager with a unique id number that only she knows. when we grade the questionnaires, we don\u2019t know who sent them. this takes a lot less time than blinding each individual resume would, and allows us at least one look at a candidate\u2019s materials without any bias about who they are.<\/p>\n why do it?<\/strong> what is it? <\/strong> how long does it take? <\/strong> how does it work?<\/strong> why do it?<\/strong> we also found that using a rubric not only made our decisions fairer, it also sped up questionnaire grading. we create rubrics for all our questionnaires now, and that small investment of time ends up reducing bias and speeding up grading.<\/p>\n bonus tip:<\/em> if this is a new position or an initial hiring round, it can help to have two people grade a few questionnaires just to get calibrated. what is it?<\/strong> how long does it take?<\/strong> how does it work?<\/strong> why do it?<\/strong> but there\u2019s no need to throw the baby out with the bath water. company culture is important! different people thrive or flounder in different types of organizations, and you want to make sure you\u2019re hiring people who are going to succeed in your company. if you can clearly define what types of behaviors align with your values, you can hire for culture fit in a way that is fair and equitable. what is it?<\/strong> how long does it take?<\/strong> how does it work?<\/strong> another method for uncovering your biases (h\/t to lifelabs<\/a> for this idea) is to make a list of people you know personally whom you admire and whom you might turn to for advice. next to each name, write down their gender, ethnicity, marital status, disability status, sexual orientation, religion, etc. as you look down the list, you might notice patterns about the types of people you tend to look up to, and that might surface some biases that you might not have been aware of.<\/p>\n
\n
\n <\/p>\nmagoosh hiring process<\/h2>\n
\n
\n <\/p>\ntip 1: blind materials when possible<\/h2>\n
\nremoving identifying information from application materials (resumes, exercises, work samples, etc.).<\/p>\n
\nsome tools can blind application materials automatically, but even doing it manually for some materials can take as little as 1-2 minutes per candidate. <\/p>\n
\nif you are working with an applicant tracking system that allows for blind resume reviews, that\u2019s wonderful\u2014unfortunately, greenhouse (the ats we use) doesn\u2019t have that function. however, we can still blind some steps in our process. <\/p>\n
\nresearch<\/a> shows that hiring managers will treat identical resumes and work samples differently depending upon the race or gender of the applicant. this means that, even if you don\u2019t intend to do it, you will likely evaluate materials differently if you know who submitted them. by removing personally identifying info from the resume and\/or work sample, you can reduce this bias.
\n
\n <\/p>\ntip 2: create rubrics<\/h2>\n
\na list of points that candidates earn or lose based on their responses to questions on questionnaires or exercises. here is a rubric<\/a> i used recently to grade questionnaire exercises for our business operations manager position. the rubric is far from perfect, but it\u2019s much better than subjectively grading the questionnaire<\/a>. <\/p>\n
\nit takes about 1 hour to create a quick and simple rubric, but you may spend additional time refining it as you grade more candidates\u2019 materials. the rubric does end up making the grading process more straightforward, so it can end up saving a few minutes per questionnaire. depending on applicant volume this may actually add up to a big time saver.<\/p>\n
\nusing your job description as a starting point, determine what success looks like on the questionnaire and assign points to different categories based on importance. you can get really granular, but that\u2019s not necessary to start. as your hiring managers gain more experience, they\u2019ll become better and better at creating rubrics. <\/p>\n
\nwe\u2019ve used questionnaires for hiring since the very beginning of magoosh, and for a while they were graded holistically without a rubric. however, early on when lucas and kevin, hiring managers for our first remote test prep experts, both graded a few questionnaires, we realized that they didn\u2019t always agree on which passed and which did not. depending on their own biases (lucas might put a heavier emphasis on grammar while kevin might give more weight to writing tone and style) they would come to different conclusions. when they worked together to create a rubric, they were able to quantitatively state how much weight each part of the questionnaire should have.<\/p>\n
\n <\/p>\ntip 3: structured culture fit<\/h2>\n
\na standard list of questions about work style and personality traits that you ask all candidates for a given position. <\/p>\n
\nabout 2-3 hours for an mvp (minimum viable product) version that you can iterate on over time as you learn more about which questions work best for your company.<\/p>\n
\nwhen conducting final round interviews for a position, we ask every candidate the same set of culture questions. these questions have changed over time but remain the same for all candidates for a particular position. over every iteration, our goal is to assess how well a candidate aligns with our core values. to implement this, think about your company\u2019s culture and about what traits tend to lead to success, and then ask questions to get at whether candidates share those traits. (for example, one of our values<\/a> is \u201cdone > perfect,\u201d so we\u2019ll ask a question where the candidate describes how they would respond in a situation where they have to make a tradeoff between speed and quality.) <\/p>\n
\nif you read about hiring bias, you\u2019ll come across countless recommendations to stop interviewing for culture fit<\/a>. in too many cases, \u201cculture fit\u201d comes down to \u201cdo i like this person?\u201d, and that obviously allows bias to sneak in.<\/p>\n
\n
\n <\/p>\ntip 4: acknowledge biases<\/h2>\n
\nall of these steps are helpful in mitigating unconscious bias, but they only address parts of the hiring process. ultimately you need to really confront your biases in order to make sure that you\u2019re making the best hiring decisions. this means thinking critically about yourself and admitting when you might be making decisions based on irrelevant factors (such as ethnicity or gender). <\/p>\n
\nyou can spend your life trying to understand your biases, but you can also make significant progress in just a few minutes. like everything else in this list, it\u2019s ok to start small and then do more over time. <\/p>\n
\nthere are lots of different ways to think about and diagnose some of your biases, including workshops<\/a>, classes<\/a>, or books<\/a>. there are also smaller scale things you can do as well. tools like project implicit<\/a>\u2019s bias tests can help surface biases you might be holding on to. these tests help identify what automatic preferences you might have about different types of people. and keep this in mind: if the results of one of the tests indicate you might have a bias, don\u2019t get defensive or make excuses! instead try to sit with these results and think about why you might have gotten them. <\/p>\n