{"id":4847,"date":"2017-08-02t09:00:14","date_gmt":"2017-08-02t09:00:14","guid":{"rendered":"\/\/www.catharsisit.com\/?p=4847"},"modified":"2017-08-02t09:00:14","modified_gmt":"2017-08-02t09:00:14","slug":"behind-the-scenes-the-magoosh-engineering-exercise","status":"publish","type":"post","link":"\/\/www.catharsisit.com\/blog\/behind-the-scenes-the-magoosh-engineering-exercise\/","title":{"rendered":"behind the scenes: the magoosh engineering exercise"},"content":{"rendered":"<p>stressful, inconsistent, and opaque. these are all words used to describe most engineering interviews. at magoosh, we want to provide a better experience to engineering candidates, so we\u2019re striving to create a more transparent process from start to finish. as part of that effort, i\u2019d like to share some behind-the-scene details about one important stage of our interview process: the engineering exercise.<\/p>\n<p>below is an overview our our general hiring process for any position. (you can also read about it in more detail in our full <a href=\"\/\/www.catharsisit.com\/blog\/magoosh-hiring-process\/\" target=\"_blank\" rel=\"noopener\">magoosh hiring process<\/a> blog post.)<\/p>\n<ul>\n<li><em><em>application review<\/em><\/em><\/li>\n<li><em><em>questionnaire<\/em><\/em><\/li>\n<li><em><em>phone interview<\/em><\/em><\/li>\n<li><em><em>reference and background checks<\/em><\/em><\/li>\n<li><em><em>in-person interview<\/em><\/em><\/li>\n<\/ul>\n<p>the engineering exercise comes at the <em>questionnaire<\/em> stage. we\u2019ve sought to create an exercise that\u2019s both representative of the skills a candidate would use at magoosh and helpful to us in assessing the candidate\u2019s experience.<\/p>\n<p>&nbsp;<\/p>\n<h2>the questions<\/h2>\n<p>we ask candidates to complete two questions: a programming question and an architecture\/design question. for the programming question, we give the candidate a file and ask them to write code that is ready for production and code review. for the architecture question, we describe a challenge that we\u2019ve faced and ask the candidate to share how they would approach that challenge.<\/p>\n<p>the first iteration of the exercise was far from perfect\u2014we received a number of questions, and we\u2019d see submissions we didn\u2019t expect. it didn\u2019t take us long to realize our instructions left something to be desired. over time, we\u2019ve taken candidate feedback and edited our instructions to make the exercise more clear. for example, we previously didn\u2019t explain that we wanted production-ready code, so some candidates solved the programming question by writing a quickly thrown-together script. we plan to continue improving the exercise as we get more feedback.<\/p>\n<p>&nbsp;<\/p>\n<h2>timing<\/h2>\n<p>originally, we estimated both questions would take a total of three hours to complete, so we asked candidates to set aside a three-hour window to complete the exercise. we received feedback both from candidates and from people we had hired that the short window created unrealistic stress, which is not what we wanted. based on that feedback, we now ask candidates to complete the exercise within 24 hours. that way, all candidates have the same amount of time but there isn\u2019t an unrealistic time pressure to complete the exercise exactly within three hours.<\/p>\n<p>&nbsp;<\/p>\n<h2>grading<\/h2>\n<p>our most senior engineer, <a href=\"\/\/www.catharsisit.com\/team\/zackm\/\" target=\"_blank\" rel=\"noopener\">zack mayeda<\/a>, grades the engineering exercises. with the help of others, he\u2019s created a rubric and grades against the rubric so that each exercise is graded on the same criteria. our hiring team also anonymizes as much of the candidate\u2019s information as possible before handing the exercise off to him, so he can evaluate the work with as little bias as possible.<\/p>\n<p>&nbsp;<\/p>\n<h2>testing our test<\/h2>\n<p>when designing our exercise, we knew it would be impossible to create a process that was 100% perfect, but we still wanted to run a test to see if it was at least directionally accurate (one of our core values is data &gt; intuition). so, i asked two talented senior engineers from another company to complete the exercise and then put their submissions (stripped of their names and information) into the queue along with other candidates\u2019 submissions. if those engineers scored poorly, then we could be fairly certain that there might be something wrong with our exercise or rubric. fortunately, they both scored very well, which gave us some confidence that our exercise was a useful competency test. (as an aside, zack was very excited about the candidates, until i told him that they weren\u2019t actually applying&#8230;sorry, zack!) all this said, we still acknowledge that our exercise isn\u2019t perfect and will probably generate some false negatives and false positives. again, we\u2019re dedicated to continuing to improve the process based on feedback we receive.<\/p>\n<p>&nbsp;<\/p>\n<h2>feedback<\/h2>\n<p>if a candidate does well enough on the exercise, we move them forward to the <em>phone<\/em> stage of the interview process. during the phone interview we may ask questions about why they made certain decisions. if a candidate doesn\u2019t do well on the exercise and doesn&#8217;t make it to the phone interview stage, we\u2019ll let them know we\u2019re pursuing other candidates and give them the option to receive feedback on their work. we know feedback makes us all better, so we offer it to all candidates\u2014not just those who make it all the way through to the end of the interview process.<\/p>\n<p>&nbsp;<\/p>\n<h2>tl;dr<\/h2>\n<ol>\n<li style=\"list-style-type: none\">\n<ul>\n<li>at magoosh, we believe your engineering exercise should be representative of the work you\u2019d do at magoosh<\/li>\n<li>our exercise assesses multiple types of engineering work: coding + architecture<\/li>\n<li>we give everyone the same time constraints and try not to make those time constraints too stressful<\/li>\n<li>when grading the exercise, we focus on the work not the person<\/li>\n<li>we give feedback to anyone who wants it<\/li>\n<li>we\u2019re constantly trying to improve our exercise<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n<p>&nbsp;<\/p>\n<p><strong><em>interested in working at magoosh? check out our <a href=\"\/\/www.catharsisit.com\/careers\/?utm_source=companyblog&amp;utm_medium=blog&amp;utm_campaign=behind-the-scenes-magoosh-engineering\" rel=\"noopener\" target=\"_blank\">open positions<\/a>.<\/em><\/strong><\/p>\n<p>if you don\u2019t see an open position that\u2019s perfect for you, please submit your resume via the <a href=\"http:\/\/grnh.se\/u2q87j1\" rel=\"noopener\" target=\"_blank\">general application<\/a>.<\/p>\n<p>learn more about our philosophy and culture <a href=\"\/\/www.catharsisit.com\/careers\/\" rel=\"noopener\" target=\"_blank\">here<\/a>.<\/p>\n<p>&nbsp;<br \/>\n&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>stressful, inconsistent, and opaque. these are all words used to describe most engineering interviews. at magoosh, we want to provide a better experience to engineering candidates, so we\u2019re striving to create a more transparent process from start to finish. as part of that effort, i\u2019d like to share some behind-the-scene details about one important stage [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[7],"tags":[],"ppma_author":[496],"acf":[],"yoast_head":"<!-- this site is optimized with the yoast seo premium plugin v21.7 (yoast seo v21.7) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>behind the scenes: the magoosh engineering exercise - magoosh 2022年足球世界杯举办地<\/title>\n<meta name=\"description\" content=\"at magoosh, we want to provide a better experience to engineering candidates, so we\u2019re striving to create a more transparent process from start to finish.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"\/\/www.catharsisit.com\/blog\/behind-the-scenes-the-magoosh-engineering-exercise\/\" \/>\n<meta property=\"og:locale\" content=\"en_us\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"behind the scenes: the magoosh engineering exercise\" \/>\n<meta property=\"og:description\" content=\"at magoosh, we want to provide a better experience to engineering candidates, so we\u2019re striving to create a more transparent process from start to finish.\" \/>\n<meta property=\"og:url\" content=\"\/\/www.catharsisit.com\/blog\/behind-the-scenes-the-magoosh-engineering-exercise\/\" \/>\n<meta property=\"og:site_name\" content=\"magoosh 2022年足球世界杯举办地\" \/>\n<meta property=\"article:published_time\" content=\"2017-08-02t09:00:14+00:00\" \/>\n<meta property=\"og:image\" content=\"\/\/www.catharsisit.com\/blog\/files\/2017\/08\/rsz_shutterstock_292225463.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1944\" \/>\n\t<meta property=\"og:image:height\" content=\"1296\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"bhavin parikh\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:title\" content=\"behind the scenes: the magoosh engineering exercise\" \/>\n<meta name=\"twitter:description\" content=\"at magoosh, we want to provide a better experience to engineering candidates, so we\u2019re striving to create a more transparent process from start to finish.\" \/>\n<meta name=\"twitter:label1\" content=\"written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"bhavin parikh\" \/>\n\t<meta name=\"twitter:label2\" content=\"est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"webpage\",\"@id\":\"\/\/www.catharsisit.com\/blog\/behind-the-scenes-the-magoosh-engineering-exercise\/\",\"url\":\"\/\/www.catharsisit.com\/blog\/behind-the-scenes-the-magoosh-engineering-exercise\/\",\"name\":\"behind the scenes: the magoosh engineering exercise - magoosh 2022年足球世界杯举办地\",\"ispartof\":{\"@id\":\"\/\/www.catharsisit.com\/blog\/#website\"},\"datepublished\":\"2017-08-02t09:00:14+00:00\",\"datemodified\":\"2017-08-02t09:00:14+00:00\",\"author\":{\"@id\":\"\/\/www.catharsisit.com\/blog\/#\/schema\/person\/a39368a4ec92c14292a2159b802bd6ff\"},\"description\":\"at magoosh, we want to provide a better experience to engineering candidates, so we\u2019re striving to create a more transparent process from start to finish.\",\"breadcrumb\":{\"@id\":\"\/\/www.catharsisit.com\/blog\/behind-the-scenes-the-magoosh-engineering-exercise\/#breadcrumb\"},\"inlanguage\":\"en-us\",\"potentialaction\":[{\"@type\":\"readaction\",\"target\":[\"\/\/www.catharsisit.com\/blog\/behind-the-scenes-the-magoosh-engineering-exercise\/\"]}]},{\"@type\":\"breadcrumblist\",\"@id\":\"\/\/www.catharsisit.com\/blog\/behind-the-scenes-the-magoosh-engineering-exercise\/#breadcrumb\",\"itemlistelement\":[{\"@type\":\"listitem\",\"position\":1,\"name\":\"home\",\"item\":\"\/\/www.catharsisit.com\/blog\/\"},{\"@type\":\"listitem\",\"position\":2,\"name\":\"behind the scenes: the magoosh engineering exercise\"}]},{\"@type\":\"website\",\"@id\":\"\/\/www.catharsisit.com\/blog\/#website\",\"url\":\"\/\/www.catharsisit.com\/blog\/\",\"name\":\"magoosh 2022年足球世界杯举办地\",\"description\":\"\",\"potentialaction\":[{\"@type\":\"searchaction\",\"target\":{\"@type\":\"entrypoint\",\"urltemplate\":\"\/\/www.catharsisit.com\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inlanguage\":\"en-us\"},{\"@type\":\"person\",\"@id\":\"\/\/www.catharsisit.com\/blog\/#\/schema\/person\/a39368a4ec92c14292a2159b802bd6ff\",\"name\":\"bhavin parikh\",\"image\":{\"@type\":\"imageobject\",\"inlanguage\":\"en-us\",\"@id\":\"\/\/www.catharsisit.com\/blog\/#\/schema\/person\/image\/cc5d6102e586a34b45fcebac62755922\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/daf99fb3286cfe961afb930eadd2298c?s=96&d=mm&r=g\",\"contenturl\":\"https:\/\/secure.gravatar.com\/avatar\/daf99fb3286cfe961afb930eadd2298c?s=96&d=mm&r=g\",\"caption\":\"bhavin parikh\"},\"description\":\"bhavin sets the vision and strategy for magoosh, along with whatever else needs to be done. with a bs\/ba in economics and computer science from duke university and an mba from the haas school of business at uc berkeley, he\u2019s on a mission to change the way people learn and how they think about learning. years ago, bhavin played on several nationals-level ultimate frisbee teams. today, he\u2019s our resident gelato connoisseur.\",\"sameas\":[\"adeney\"],\"knowsabout\":[\"test prep\",\"gre exam\",\"executive leadership\"],\"knowslanguage\":[\"english\"],\"jobtitle\":\"ceo\",\"worksfor\":\"magoosh\",\"url\":\"\/\/www.catharsisit.com\/blog\/author\/bhavin\/\"}]}<\/script>\n<!-- \/ yoast seo premium plugin. -->","yoast_head_json":{"title":"behind the scenes: the magoosh engineering exercise - magoosh 2022年足球世界杯举办地","description":"at magoosh, we want to provide a better experience to engineering candidates, so we\u2019re striving to create a more transparent process from start to finish.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"\/\/www.catharsisit.com\/blog\/behind-the-scenes-the-magoosh-engineering-exercise\/","og_locale":"en_us","og_type":"article","og_title":"behind the scenes: the magoosh engineering exercise","og_description":"at magoosh, we want to provide a better experience to engineering candidates, so we\u2019re striving to create a more transparent process from start to finish.","og_url":"\/\/www.catharsisit.com\/blog\/behind-the-scenes-the-magoosh-engineering-exercise\/","og_site_name":"magoosh 2022年足球世界杯举办地","article_published_time":"2017-08-02t09:00:14+00:00","og_image":[{"width":1944,"height":1296,"url":"\/\/www.catharsisit.com\/blog\/files\/2017\/08\/rsz_shutterstock_292225463.jpg","type":"image\/jpeg"}],"author":"bhavin parikh","twitter_card":"summary_large_image","twitter_title":"behind the scenes: the magoosh engineering exercise","twitter_description":"at magoosh, we want to provide a better experience to engineering candidates, so we\u2019re striving to create a more transparent process from start to finish.","twitter_misc":{"written by":"bhavin parikh","est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"webpage","@id":"\/\/www.catharsisit.com\/blog\/behind-the-scenes-the-magoosh-engineering-exercise\/","url":"\/\/www.catharsisit.com\/blog\/behind-the-scenes-the-magoosh-engineering-exercise\/","name":"behind the scenes: the magoosh engineering exercise - magoosh 2022年足球世界杯举办地","ispartof":{"@id":"\/\/www.catharsisit.com\/blog\/#website"},"datepublished":"2017-08-02t09:00:14+00:00","datemodified":"2017-08-02t09:00:14+00:00","author":{"@id":"\/\/www.catharsisit.com\/blog\/#\/schema\/person\/a39368a4ec92c14292a2159b802bd6ff"},"description":"at magoosh, we want to provide a better experience to engineering candidates, so we\u2019re striving to create a more transparent process from start to finish.","breadcrumb":{"@id":"\/\/www.catharsisit.com\/blog\/behind-the-scenes-the-magoosh-engineering-exercise\/#breadcrumb"},"inlanguage":"en-us","potentialaction":[{"@type":"readaction","target":["\/\/www.catharsisit.com\/blog\/behind-the-scenes-the-magoosh-engineering-exercise\/"]}]},{"@type":"breadcrumblist","@id":"\/\/www.catharsisit.com\/blog\/behind-the-scenes-the-magoosh-engineering-exercise\/#breadcrumb","itemlistelement":[{"@type":"listitem","position":1,"name":"home","item":"\/\/www.catharsisit.com\/blog\/"},{"@type":"listitem","position":2,"name":"behind the scenes: the magoosh engineering exercise"}]},{"@type":"website","@id":"\/\/www.catharsisit.com\/blog\/#website","url":"\/\/www.catharsisit.com\/blog\/","name":"magoosh 2022年足球世界杯举办地","description":"","potentialaction":[{"@type":"searchaction","target":{"@type":"entrypoint","urltemplate":"\/\/www.catharsisit.com\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inlanguage":"en-us"},{"@type":"person","@id":"\/\/www.catharsisit.com\/blog\/#\/schema\/person\/a39368a4ec92c14292a2159b802bd6ff","name":"bhavin parikh","image":{"@type":"imageobject","inlanguage":"en-us","@id":"\/\/www.catharsisit.com\/blog\/#\/schema\/person\/image\/cc5d6102e586a34b45fcebac62755922","url":"https:\/\/secure.gravatar.com\/avatar\/daf99fb3286cfe961afb930eadd2298c?s=96&d=mm&r=g","contenturl":"https:\/\/secure.gravatar.com\/avatar\/daf99fb3286cfe961afb930eadd2298c?s=96&d=mm&r=g","caption":"bhavin parikh"},"description":"bhavin sets the vision and strategy for magoosh, along with whatever else needs to be done. with a bs\/ba in economics and computer science from duke university and an mba from the haas school of business at uc berkeley, he\u2019s on a mission to change the way people learn and how they think about learning. years ago, bhavin played on several nationals-level ultimate frisbee teams. today, he\u2019s our resident gelato connoisseur.","sameas":["adeney"],"knowsabout":["test prep","gre exam","executive leadership"],"knowslanguage":["english"],"jobtitle":"ceo","worksfor":"magoosh","url":"\/\/www.catharsisit.com\/blog\/author\/bhavin\/"}]}},"authors":[{"term_id":496,"user_id":2,"is_guest":0,"slug":"bhavin","display_name":"bhavin parikh","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/daf99fb3286cfe961afb930eadd2298c?s=96&d=mm&r=g","user_url":"","last_name":"parikh","first_name":"bhavin","description":"bhavin sets the vision and strategy for magoosh, along with whatever else needs to be done. with a bs\/ba in economics and computer science from duke university and an mba from the haas school of business at uc berkeley, he\u2019s on a mission to change the way people learn and how they think about learning. years ago, bhavin played on several nationals-level ultimate frisbee teams. today, he\u2019s our resident gelato connoisseur."}],"_links":{"self":[{"href":"\/\/www.catharsisit.com\/blog\/wp-json\/wp\/v2\/posts\/4847"}],"collection":[{"href":"\/\/www.catharsisit.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"\/\/www.catharsisit.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"\/\/www.catharsisit.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"\/\/www.catharsisit.com\/blog\/wp-json\/wp\/v2\/comments?post=4847"}],"version-history":[{"count":0,"href":"\/\/www.catharsisit.com\/blog\/wp-json\/wp\/v2\/posts\/4847\/revisions"}],"wp:attachment":[{"href":"\/\/www.catharsisit.com\/blog\/wp-json\/wp\/v2\/media?parent=4847"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"\/\/www.catharsisit.com\/blog\/wp-json\/wp\/v2\/categories?post=4847"},{"taxonomy":"post_tag","embeddable":true,"href":"\/\/www.catharsisit.com\/blog\/wp-json\/wp\/v2\/tags?post=4847"},{"taxonomy":"author","embeddable":true,"href":"\/\/www.catharsisit.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=4847"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}