first, why is diversity at work important for us? our mission is to help level the playing field in education, and we’ll only be able to achieve that mission if our products serve all students equitably. that’s why we believe it’s important to ensure our company and workplace culture reflect our students and their lived experiences.

this journey has been an incredible learning process for our people teams here at magoosh, and we’ve made a ton of progress. looking back, it’s clear that we did not get everything right. we hope that by publicly sharing our successes and our failures, we can hold ourselves accountable (and help other companies) to create a diverse and equitable workplace for all. together, we can work collectively on diversity and inclusion in tech and higher education.
 

vision of diversity at magoosh

magoosh students are a broadly diverse group. in order for magoosh to best serve its students and to grow as a business, our team must reflect the diversity of our students. our long term goal is to have a magoosh team that reflects the diversity of our us-based students.

here’s what we know about our students:

race/ethnicity: 46% white, 17% hispanic or latino, 16% black or african american, 13% asian, 6% two or more races, 1% american indian or alaskan native, 1% prefer not to say
 

first-generation college student: 23.2% yes, 75.8% no, 1% prefer not to say
 

first language: 34% not english, 65% english, 1% prefer not to say
 

gender: 63% female, 35% male, 1% other/nonconforming

the benefits of diverse teams have been widely researched: diverse teams tend to have better outcomes. we want magoosh to be a more diverse company, not only because doing so is aligned with our mission, but also because a diverse team will set us up for continued growth and success. at magoosh, we are treating diversity at work like any other area of our business and setting specific, measurable goals to ensure that we achieve our vision.
 

our 2020 diversity at work goals

to reach the longer-term vision of magoosh reflecting the diversity of its students, we set benchmark goals for 2020.

we recognize that there are many ways for groups to be diverse and that no set of diversity goals will ever fully encompass all of the aspects of our students. furthermore, different dimensions of diversity are not discrete line items; they intersect in important ways that have a significant impact both at the systemic and the individual levels. we also know from experience that we are most likely to see change when we set specific, measurable goals and consistently track our progress towards them.

to build a team that reflects the diversity of our students, we set the following goals for 2020 as part of our three-year plan:

1. 20% of magoosh employees will identify as a member of an underrepresented minority (urm) ethnic/racial group.
2. 16% of magoosh employees will identify as the first in their family to go to college.
3. 23% of magoosh employees will have grown up in a home where the primary language spoken was not english.
4. on every team made up of seven or more people, there will be at least two people of non-majority gender identities.

we’ll be sharing our process for setting these goals in a future blog post.

these goals are benchmarks, not goalposts. we’re measuring ourselves against these goals in 2020, but also looking ahead to future years where we will continue to push ourselves to better represent our students.
 

where we started

we started the process of setting diversity at work goals in late 2017, and our first step was to get accurate data on our current team. here is a snapshot of how we stacked up against our diversity goals at that time:

• 7% of employees identified as a member of an underrepresented minority group.
• 10% of employees identified as the first in their family to go to college.
• 28% of employees identified as growing up in a home where the primary language spoken was not english.
• half of our teams with more than seven employees had at least two people of non-majority gender identities.

where we are now

on our recent survey of employees, here were the results:

*additionally, for our gender goal, 100% of teams of size seven or more have at least two members of the non-majority gender identity.

looking at these results, we can see we’ve made substantial progress towards our 2020 goals, hitting them in two areas and coming quite close in the other two. however, we still have a long way to go to achieve our vision of reflecting the diversity of our students. we’ll be pushing forward that work into 2021 and beyond to ensure we achieve that vision.
 

diversity at work: addressing inclusion, equity, and belonging

of course, hiring a diverse team is not enough to ensure that magoosh and our students benefit from the results. members of historically underrepresented communities need to:

• be included and engaged in their work
• feel a sense of belonging
• be treated equitably
• be given equal opportunities for promotion and advancement.

as we grow (and grow more diverse), we are challenging ourselves to think more critically about diversity, equity, inclusion, and belonging in the workplace, and we’ll be sharing more about our progress in future blog posts.
 

the post magoosh’s diversity goals appeared first on magoosh 2022年足球世界杯举办地.

(note: this next paragraph contains a reference to self-harm; if that’s something you struggle with, you might want to skip it.)

i have a vivid memory of walking to the school i taught at one morning and thinking “hmm, that car isn’t going too fast. i bet if i jumped out in front of it i’d get hurt enough to get some time off. it’d be bad, but not too bad.” at the time i remember laughing at how preposterous that thought was and (thankfully!) did not jump in front of moving traffic.

with the perspective of time, i can now recognize that this was a completely unhealthy thought that should have indicated i was in a seriously rough state.

i was experiencing burnout.

image credit: nickvector (licensed via shutterstock)

i ended up leaving the teaching profession, going to business school, and getting a job at magoosh. but i never forgot that feeling of burnout, and i try to be aware if i see it in others.

fortunately, magoosh is a company that prioritizes the whole lives of its employees. we have a flexible vacation policy, generous benefits and parental leave, and a culture that specifically eschews perfectionism. but we’re still a startup, and that means there’s always more to do than there is time to do it. work is often a sprint, and you just can’t sprint forever. i want to make sure no one who works at magoosh ever feels that same sense of burnout i felt in dc.

that’s why we’re introducing sabbaticals at magoosh.

magoosh now offers six-week fully-paid sabbaticals for qualifying full-time and part-time employees who have worked at magoosh for five years or longer, averaging at least 24 hours per week.

you can take a look at our sabbatical policy (the document we shared with all employees when we rolled this benefit out back in october) here if you’re curious about the specifics.

why offer sabbatical to our employees (especially as a small company)?

the goal of magoosh’s sabbatical benefit is to encourage longevity with the company, prevent and alleviate burnout, and help longer-tenured employees get a fresh perspective on work.

in addition to this being good for the individuals who take it, there is some compelling research that a sabbatical policy can be good for the company overall.

magoosh employees have also shown an interest in having a sabbatical policy. we have an “ideas and brainstorming” section of our company asana, and offering a sabbatical benefit has been a proposed idea for quite some time.

so, for a company wanting to attract and retain great employees — and help them do their best work — offering a sabbatical benefit seemed like a great next step.

but why now? well, magoosh grew substantially in 2013 and 2014, adding many new employees in that time (including me!). a lot of us are still working at magoosh, and some of us are in need of a break. after having conversations with a few long-tenured employees, we decided that 2019 was the right time to prioritize creating a sabbatical policy.

how we structured our sabbatical program (and challenges we’ve faced so far)

when we first set out to make a sabbatical policy, we thought that the biggest issues would be hammering out the specific rules (length of service, length of sabbatical, etc.). it turned out those questions were pretty easy to answer, but then others were surprisingly tricky!

let’s start with the easy stuff.

length of service. we pretty quickly settled on five years as a good length of time. when we looked at other companies that offered sabbatical, five years was the most common requirement. i’ll admit, as a linguistic purist, i really wanted to keep to the traditional seven year duration (“sabbatical” comes from the biblical “sabbath”, as in “on the seventh day, he rested”). but five years is far more common in corporate sabbatical, and is also a nice round number.

length of sabbatical. we chose six weeks as the length of time for a couple of reasons. again, this length is comparable to length of sabbaticals at several other companies that offer them. additionally, it feels long enough to be meaningful, but not so long that it would be particularly difficult for our teams to plan around.

eligibility criteria. everyone at magoosh was aligned on the idea that we wanted to be able to provide sabbatical benefits to both our part-time and full-time employees. we have several part-time employees who have worked at magoosh for many years, helping literally thousands of students individually. we modeled our part-time sabbatical requirements after adobe’s, offering the benefit to folks who work the majority of their week at magoosh (three out of five days, or 24 hours per week).

these points came together pretty quickly. but, as i said, it was the other questions that came up that really made us think. “can i take a sabbatical in smaller chunks?” “can i save up sabbaticals and take twelve weeks after ten years?” “can i combine sabbatical with other pto, unpaid time, or leave?”

we really struggled with some of these questions! on the one hand, we want employees to do what is best for them and don’t want to put up arbitrary restraints or rules. on the other hand, sabbatical really only benefits the company when employees take the time in a way that is restorative.

we decided to align our approach to these trickier questions around a guiding principle: the purpose of sabbatical is to give employees lengthy, uninterrupted time away from work so that they come back to magoosh with a fresh perspective and renewed energy. if an employee wants to use their sabbatical time in a way that is in alignment with that principle, then we’re generally ok with it.

image credit: alena ozerova (licensed via shutterstock)

our first employee to take sabbatical will go out in december 2019, and we have four more already scheduled for 2020 (including our ceo’s in late summer — have fun bhavin, we may or may not fold our bagels in half while you’re away!). while our sabbatical is optional, we do hope that everyone who is eligible takes it.

i’m sure we’ll have more challenges and bumps along the way as we prepare for and go through our first employee sabbaticals. if you’ve taken a sabbatical, let us know in the comments what challenges you and your organization faced. and if you’ve never worked at a company with a sabbatical policy, maybe now’s the time. check out our careers page for open roles and join our team!

the post away they go: introducing paid sabbaticals at magoosh appeared first on magoosh 2022年足球世界杯举办地.

what would happen if all salaries at your company became public today — everyone found out what everyone else was paid? would an employee be upset to learn that they were being paid less than their peers of equal experience? would one team be shocked at how much higher other teams were being paid? would manager biases become immediately apparent? would it fuel feelings of jealousy or of fairness?

if that happened at magoosh today, my guess is most people would simply shrug and say “yeah, that sounds about right,” and carry on with their day. and that’s because, without exception, any two people at magoosh at the same job level in the same role are paid equally.

employee salaries and benefits are the largest expense, by far, for most tech companies. and yet, most don’t talk about, much less share, how much people are paid. not only is that a shame, it’s a short-sighted stance and a missed opportunity to improve hiring practices and employee retention.

sadly, tech companies aren’t alone in the struggle for pay equity. five years ago, beyoncé famously spoke out on the myth of gender equality, noting that “equality will be achieved when men and women are granted equal pay and equal respect.” a forbes piece on the rampant nature of gender pay gaps across a number of industries cites that, in the past few years, the 10 top-paid actresses in hollywood earned anywhere from 24 to 35 cents on the dollar compared to their male counterparts. on netflix drama the crown, even the queen herself, played by actress claire foy, found out she was being paid less than her co-star matt smith, who played prince phillip. the reality is even more sobering for the pay gaps faced by women of color.

a 2017 paper on the hollywood wage gap from economists at lancaster university suggested salary transparency could help reverse this phenomenon, noting that “making contracts not blinded in the film industry and thus providing social information about what other co-stars earn can reduce the negotiation gap and therefore the residual wage gap.” ethical clothing brand nisolo recently began publicly disclosing the lowest wages in its supply chain as a means of encouraging other fashion manufacturers to commit to ensuring that all workers are paid at least a living wage. this summer, museum workers banded together to create a public spreadsheet of salaries across museum positions, to expose the rampant pay disparities in the art world. the initiative, led by art + museum transparency, went on to write an opinion piece calling for salary and benefits to be included in all culture sector job descriptions.

so, fellow tech workers, what’s our excuse? why are we still behind the times in treating salaries as a shameful secret or a game to be hacked by the most privileged of us?

it doesn’t have to be this way. you can be a fun, profitable tech company with transparent and fair salary practices, all in the same breath. we know this because magoosh is living proof, ten years since our founding, that transparency is good for business. here’s what we’ve learned from including salaries in our each of job descriptions:

transparent salaries help you hire the right people

my guess is that most startups don’t make salaries for their open positions public for a simple reason: they don’t know what the salary ranges are. managers have identified a need on their team, and they want to find someone to fill that need. they likely haven’t figured out exactly how much filling that need is going to cost. unfortunately, this means they don’t know who they’re actually looking for.

publishing salary ranges in your job descriptions forces you to figure out what problem or opportunity you’re solving for by hiring, and how much you think it will cost to find someone who can do it. this planning prevents you from pursuing under- or over-qualified candidates, and helps you hone in on exactly the right people for your role.

this planning also saves you money. when you’re clear on the level and role you need to fill, you prevent yourself from 1) spending too much hiring someone “great” who is overqualified for the job you need and who will likely become disengaged and quit because there isn’t enough exciting work for them to take on, or 2) spending too little hiring someone “good enough” who is unable to grow into the role long-term and will need to be replaced. getting the right person in the role now saves you money in the long-term.

this planning is critical, but often not a step that happens at fast-growing startups where things are moving quickly. by putting salaries in your job descriptions, you force hiring managers to think critically about their needs, and to budget accordingly.

transparent salaries save you and your candidates from wasted time

a few weeks ago, i received an outreach email from a recruiter at another ed-tech startup. i took the call and spent about 35 minutes on the phone with a recruiter. at the very end of the call, they shared the salary range. as it turns out, it was substantially lower than what i currently make at magoosh (shout out to our compensation!), and if i were seriously considering the opportunity, this would have been a waste of both my and the recruiter’s time. if the initial job description had included salary information, i could have self-selected out of the process and saved us both the time.

(side note: i don’t have plans to leave magoosh, but it’s good to know what’s out there! and since the recruiting function at magoosh is on my team, i like to understand how other companies approach outreach.)

i’m not alone in wanting salary details in job descriptions. according to a study linkedin published on what candidates want to hear from recruiters, 72% said they wanted to learn about salary range in the initial conversation. that’s more than the percent who wanted to hear about job title, company culture, or mission (54%, 40%, and 27%, respectively).

by keeping salaries secret, you will lose out on experienced talent who don’t want to waste time going through a hiring process before finding out what the compensation will be.

transparent salaries help close the wage gap for women and minorities

at magoosh, we put a lot of time and effort into making sure that our salaries are equitable and fair for all employees. we have a no-negotiation policy, and a structured salary framework for every role. any two people in the same role at the same level are being paid the same amount — equal pay for equal work. publishing our salary ranges is a natural outcome of that focus on equity.

by keeping salary information secret, you create an information imbalance that favors your company during salary negotiations. but here’s the problem: studies consistently show that women and minorities are less likely to negotiate salary, and less likely to win larger concessions when they do negotiate. this economic benefit to the company is coming at the direct expense of historically marginalized people.

moreover, unequal salaries will hurt your company. the kapor center’s research identified unfairness as the top reason that people of all backgrounds leave tech companies. saving a few thousand dollars in a salary negotiation isn’t really a win if it later leads to talented individuals leaving your company for competitors who choose to pay them what they’re worth.

“yes, but…” – three common objections to publishing salary ranges in job descriptions

while researching other perspectives on salary transparency, i came across a number of surprising (to me) rebuttals from hiring managers and recruiters. to be quite frank, i believe much of what people frame as disadvantages to salary transparency are honestly just excuses to perpetuate terrible hiring practices, and by extension, foster unhealthy company cultures.

one of the most common objections to publishing a salary range is that hiring managers are worried that great overqualified candidates won’t apply. the idea goes that you might be looking for someone who can do x, but if someone truly great comes along who is capable of doing x+y, you would offer them more money. it’s a really tempting idea, but it’s also totally backwards. do you need x+y? if so, then budget and pay for it. if not, then don’t overspend; you’ll end up hiring someone who is overqualified and will likely become disengaged.

another common objection is that managers don’t want to hire people who are motivated solely by money. in fairness, i, too, don’t want to work with people motivated solely by money. but this is a bit of a straw man. most people have lots of other motivations, but they also want to be paid what they’re worth — and that’s ok! people with families, dependent parents, or other responsibilities have mouths to feed and little time to waste. saying that they shouldn’t have information about salary to guide their life decisions is foolhardy.

then there’s the objection, worth mentioning again, that sharing salary ranges in job descriptions takes away the company’s leverage in a negotiation. and you know what? that’s true. but shifting the inequitable power dynamic between employers and candidates is also a benefit. being transparent about salary in your job descriptions encourages equity and fairness, and helps those who have historically not been able to advocate for themselves (or have been dismissed as overly assertive or demanding when they do).

will you take a step towards equal pay?

salary transparency is a great practice to institute in your company, one that can take many forms. although it can be a robust and (sometimes) complicated process, one of the least burdensome steps you can take is to make the salary ranges for your open positions public in job descriptions. this simple change will reap benefits multiple times over, helping you plan, find qualified candidates, and promote equity within your team. in the long run, this will strengthen your business.

creating a salary structure, an equity-based performance management framework, and an unbiased compensation review process is a lot of work (i know, because we’ve put these all in place and still have a lot to learn!). but publishing salary ranges in job descriptions takes almost no work at all. it’s a great first step towards building a compensation system that is fair, equitable, and transparent.

at magoosh, one of our core values is done is greater than perfect. you don’t have to have all things pay equity figured out to make salary ranges a consistent presence in your job descriptions. after all, when has beyoncé ever been wrong?

the post want to hire the best candidates? add salary ranges to your job descriptions appeared first on magoosh 2022年足球世界杯举办地.

the trap, of course, is that the larger your organization gets, the harder it can become to diversify and root out deeply ingrained biases. we’ve struggled with how to approach diversity and bias at magoosh and have found success in approaching this problem like we do other business problems: by taking small steps and iterating over time.

i lead our operations team at magoosh, so my team has been driving much of this change over the last few years. based on our experience, i want to share a list of low-effort measures we’ve taken at magoosh to reduce bias in our hiring process, as well as an estimate of how long these measures take.

my hope is that by showing how a small, lean startup like magoosh has put these steps in place, we might help other companies take the first steps towards decreasing bias. we definitely don’t have it all figured out, but if companies waited until they had it all figured out before taking steps, nothing would ever get done.

before i dive into the tips, it would be helpful to give some background on our overall hiring process, so let’s start there.
 
 

magoosh hiring process

magoosh has a fairly robust and rigorous hiring process. we’ve described it in detail before, but to provide some context, here is a brief overview. (note: below i describe how we hire for full-time positions; our process is different for part-time remote work, although we still apply many of the same bias mitigation strategies.)

now that you have a sense of our process, here are four concrete steps you can take (or suggest to your company) to reduce bias in a lean startup environment.
 
 

tip 1: blind materials when possible

what is it?
removing identifying information from application materials (resumes, exercises, work samples, etc.).

how long does it take?
some tools can blind application materials automatically, but even doing it manually for some materials can take as little as 1-2 minutes per candidate.

how does it work?
if you are working with an applicant tracking system that allows for blind resume reviews, that’s wonderful—unfortunately, greenhouse (the ats we use) doesn’t have that function. however, we can still blind some steps in our process.

during the questionnaire phase, for example, we ask candidates to not list any personally identifying information on their responses. our recruiter does a quick check before passing the anonymous questionnaires on to the hiring manager with a unique id number that only she knows. when we grade the questionnaires, we don’t know who sent them. this takes a lot less time than blinding each individual resume would, and allows us at least one look at a candidate’s materials without any bias about who they are.

why do it?
research shows that hiring managers will treat identical resumes and work samples differently depending upon the race or gender of the applicant. this means that, even if you don’t intend to do it, you will likely evaluate materials differently if you know who submitted them. by removing personally identifying info from the resume and/or work sample, you can reduce this bias.
 
 

tip 2: create rubrics

what is it?
a list of points that candidates earn or lose based on their responses to questions on questionnaires or exercises. here is a rubric i used recently to grade questionnaire exercises for our business operations manager position. the rubric is far from perfect, but it’s much better than subjectively grading the questionnaire.

how long does it take?
it takes about 1 hour to create a quick and simple rubric, but you may spend additional time refining it as you grade more candidates’ materials. the rubric does end up making the grading process more straightforward, so it can end up saving a few minutes per questionnaire. depending on applicant volume this may actually add up to a big time saver.

how does it work?
using your job description as a starting point, determine what success looks like on the questionnaire and assign points to different categories based on importance. you can get really granular, but that’s not necessary to start. as your hiring managers gain more experience, they’ll become better and better at creating rubrics.

why do it?
we’ve used questionnaires for hiring since the very beginning of magoosh, and for a while they were graded holistically without a rubric. however, early on when lucas and kevin, hiring managers for our first remote test prep experts, both graded a few questionnaires, we realized that they didn’t always agree on which passed and which did not. depending on their own biases (lucas might put a heavier emphasis on grammar while kevin might give more weight to writing tone and style) they would come to different conclusions. when they worked together to create a rubric, they were able to quantitatively state how much weight each part of the questionnaire should have.

we also found that using a rubric not only made our decisions fairer, it also sped up questionnaire grading. we create rubrics for all our questionnaires now, and that small investment of time ends up reducing bias and speeding up grading.

bonus tip: if this is a new position or an initial hiring round, it can help to have two people grade a few questionnaires just to get calibrated.
 

tip 3: structured culture fit

what is it?
a standard list of questions about work style and personality traits that you ask all candidates for a given position.

how long does it take?
about 2-3 hours for an mvp (minimum viable product) version that you can iterate on over time as you learn more about which questions work best for your company.

how does it work?
when conducting final round interviews for a position, we ask every candidate the same set of culture questions. these questions have changed over time but remain the same for all candidates for a particular position. over every iteration, our goal is to assess how well a candidate aligns with our core values. to implement this, think about your company’s culture and about what traits tend to lead to success, and then ask questions to get at whether candidates share those traits. (for example, one of our values is “done > perfect,” so we’ll ask a question where the candidate describes how they would respond in a situation where they have to make a tradeoff between speed and quality.)

why do it?
if you read about hiring bias, you’ll come across countless recommendations to stop interviewing for culture fit. in too many cases, “culture fit” comes down to “do i like this person?”, and that obviously allows bias to sneak in.

but there’s no need to throw the baby out with the bath water. company culture is important! different people thrive or flounder in different types of organizations, and you want to make sure you’re hiring people who are going to succeed in your company. if you can clearly define what types of behaviors align with your values, you can hire for culture fit in a way that is fair and equitable.
 
 

tip 4: acknowledge biases

what is it?
all of these steps are helpful in mitigating unconscious bias, but they only address parts of the hiring process. ultimately you need to really confront your biases in order to make sure that you’re making the best hiring decisions. this means thinking critically about yourself and admitting when you might be making decisions based on irrelevant factors (such as ethnicity or gender).

how long does it take?
you can spend your life trying to understand your biases, but you can also make significant progress in just a few minutes. like everything else in this list, it’s ok to start small and then do more over time.

how does it work?
there are lots of different ways to think about and diagnose some of your biases, including workshops, classes, or books. there are also smaller scale things you can do as well. tools like project implicit’s bias tests can help surface biases you might be holding on to. these tests help identify what automatic preferences you might have about different types of people. and keep this in mind: if the results of one of the tests indicate you might have a bias, don’t get defensive or make excuses! instead try to sit with these results and think about why you might have gotten them.

another method for uncovering your biases (h/t to lifelabs for this idea) is to make a list of people you know personally whom you admire and whom you might turn to for advice. next to each name, write down their gender, ethnicity, marital status, disability status, sexual orientation, religion, etc. as you look down the list, you might notice patterns about the types of people you tend to look up to, and that might surface some biases that you might not have been aware of.

why do it?
it is uncomfortable to admit that you might be harboring unconscious biases against people of different gender identities, ethnicities, or backgrounds. but the fact is that you probably are.

diverse teams get better results, and unconscious biases are likely preventing startups from having diverse teams. by taking these small first steps, you can begin to reduce bias in your hiring processes and iterate over time.

the post how a lean, scrappy startup reduced hiring bias appeared first on magoosh 2022年足球世界杯举办地.

prep4gmat is a mobile-first test prep app that is designed to get you studying for the gmat on the bus, train, elevator, or wherever you happen to have free time.  it’s created by a small team coming out of mit who’s hoping to bring test prep to where the people are.  read on to find out more!

usability: b-

the prep4gmat app works well when it works.  unfortunately, the first few times i tried to access it, it crashed.  restarted, re-downloaded, and still nothing.  i had to wait for the app to update in the store to access it, and i lost my data.  hopefully the problem is fixed, but if you’re trying to cram for a last minute test then this frustration might be a big issue.  however, once it started working, it worked well.

design: b

prep4gmat is pretty and intuitive.  however, there are a couple of nuisances in the app that prevent it from getting an a.  first, there appears to be some system by which you can unlock more practice questions.  but after looking at the app for quite a while, i couldn’t figure out how.  i ran into a few situations where the app presented a box for “practice questions” that was greyed out and unavailable, but i couldn’t figure out how to make it available.  second, you can’t move on from one module until you’ve finished the previous one — but if you can’t access the questions, you can’t pass the module!  i’m sure there are workarounds, but they aren’t obvious and intuitive.

effectiveness: a-

prep4gmat’s tutorials are written.  if you like to read to learn, this is very effective.  if you don’t, there aren’t really other options.  the practice questions are well written and approximate what you’ll actually see on the gmat, which is tough to do well.  prep4gmat can provide you with reasonably good instruction for the gmat, although it’s mobile-only format and text-dependent instruction does mean that it doesn’t cover some things as in depth as some students might need.

addictiveness: a-

prep4gmat asks some initial questions and gives a quick diagnostic exam to students.  after this, they give a “personalized” gmat prep roadmap, and send push notifications to remind you to study.  it’s not candy crush, but it doesn’t need to be!  it’s reasonably good at getting you back in the app to continue studying.

ad distraction: a

prep4gmat is a free app with no ads.  they monetize by connecting students with local tutors who can provide additional gmat support for various hourly / package fees.  this monetization model is not distracting and rarely gets in the way of the app.

motivation: a

because prep4gmat gives you a clear roadmap to success on the gmat, it’s easy to see where you are in relation to your goals.  it updates its score estimates based on your recent performance, and even gives you a percent complete estimate at the top.  this helps you focus on why you’re studying, which is very motivating.

support: a-

prep4gmat offers some traditional help; you can submit errors and bugs within the app very easily.  it also offers some non-traditional help by connecting you with tutors if you want additional support.  if you have an academic question about the content and don’t want to pay for a tutor, though, you’re out of luck.

overall gpa: 3.5

overall grade: a-

the post education app review: a- for prep4gmat appeared first on magoosh 2022年足球世界杯举办地.

we’ve written before about our support for the partnership, but it bears repeating:

magoosh is committed to making education more accessible, effective, and enjoyable, and we welcome anything that aligns with that mission. students everywhere —  regardless of economics, geography, or background — deserve access to high quality education and test prep.  but beyond a shared commitment to leveling the education playing field, there are several other reasons that magoosh welcomes khan academy’s new sat prep tools.

a shifting paradigm helps online education

when magoosh was founded, the idea of sitting down in front of a computer to study for a test was still pretty foreign.  most students who could afford test prep paid big bucks to go to test centers and take scheduled classes, while many more who couldn’t afford it bought or borrowed guidebooks and did their best alone.  having so much of the college board’s official material available online can only help bring more students into the (far cheaper) world of digital education.  for an online test prep company, more students studying online is a good thing!

khan academy’s offering will also help to erode the misconception that, when it comes to test prep, quality is correlated with cost.  at magoosh, we’ve known for a long time that it’s possible to help students achieve great score improvements without charging them through the nose, but we still encounter that perception.  khan academy’s free, high quality prep is yet another counterexample we can point to.

better official material makes for better products

anyone who works in test prep can tell you, when it comes to making the best possible practice questions, no one does it better than the test-makers themselves.  at magoosh, when we’re writing questions for our test prep products, we rely heavily on the official guides to inform how we work.  for the first time, the official stuff is available from the college board, en masse, for free, to everyone.

we love this for two reasons.

first, we have more inspiration to draw from, and a better roadmap for how to make our own test-like questions and passages.

but second, and more importantly, students can now have great examples of what the test actually looks like, and will be able to more easily evaluate test prep options against the college board’s standard. more examples of what great prep looks like means better informed students, which will help make our products shine against the competition.

raising expectations for what’s possible in test prep

perhaps the best reason we’re excited for the official study material from khan academy is that it forces everyone in the test prep game — including us! — to step it up.  having good lessons and questions simply won’t be enough to justify charging students, and all test prep providers are going to have to figure out how to create more value in order to ask students to pay.

magoosh’s offerings already go beyond questions and lessons — when students pay for our service, they also get access to our team of test-prep experts.  students can send in questions about anything in our product — lessons, questions, general study tips, or whatever else they need — and a tutor will get back to them as quickly as possible.  our team works hard, 24/7 to make sure that students have what they need.  what’s more, our product team is hard at work building new features and products to help students succeed.

in the end, the only way a business can survive is if its customers find its products valuable.  our students tell us that they love what we do. we’re continually humbled by their overwhelming support and we hope to keep earning that love.  this new free sat prep helps everyone — both students and test prep companies like magoosh — continue to push for bigger and better things.

the post why a test prep company supports khan academy’s free sat prep appeared first on magoosh 2022年足球世界杯举办地.

so this is the story of how we improved student happiness by a/b testing changes to our product with the goal not of optimizing clicks or conversions or revenues, but of maximizing student happiness.  to start, though, i’ll introduce the metric at hand: net promoter score.

nps: our reliable referral indicator

net promoter score is a metric that tells you, on the whole, how willing your customers are to promote your product.  customers are asked on a scale of 1-10 how likely they would be to recommend your product; 9s and 10s are considered “promoters”, 7s and 8s are neutral, and anything below 6 is a “detractor.”  your net promoter score is calculated by subtracting the number of detractors from the number of promoters and dividing by the total number of respondents.  as a result, nps is a percentage somewhere in the range of -100% (all detractors) to 100% (all promoters).  not to brag, but our nps is high.  really high.  apple high.

at magoosh, nps is one of the most important metrics we track — it helps us determine not only whether students like our customer service and user interface, but also how well our products prepare students for their exams.  and most importantly it has been a reliable leading indicator of growth in word-of-mouth referrals — our largest marketing channel.  when nps is high, students talk about magoosh and more people buy it!

historically, we’ve asked students the nps question after they’ve taken their exams (and, importantly, seen their final scores).  we do this because our products prepare students for tests, and, really, the proof is in the pudding.  you can’t fully decide if you’re willing to recommend magoosh for gre prep until you’ve taken the real gre.  the downside is that it can take a while for us to see nps change in response to product changes.  since we’re waiting until after students are done studying to survey them, it can take months between when a student sees a new feature and when she rates our product.

our nps issue: mismatched expectations

because nps is such an important metric to our company, we take changes very seriously.  earlier this year we saw nps for our gmat product dip fairly significantly.  looking into why, we discovered that several passive and detractor students were complaining that they were getting lower scores on their real gmat than they did on their magoosh practice tests.

our algorithm was telling students to expect one score, but, for some, their official reports were coming back lower — obviously a frustrating experience.  these students were still improving their scores significantly, but once you’ve got a 750 in your mind, a 700 seems disappointing!  we determined that we needed to fix our score prediction algorithm to be more accurate, but we were left with a major concern: would an improved algorithm that displayed a lower predicted score be demoralizing for students?  which was worse for customer satisfaction — a lower predicted score while studying, or a disappointing final score after the exam?

the challenge: could we optimize quickly for nps?

normally when we have questions about what works best for conversion or marketing, we run a quick a/b test to determine what works best.  but nps was different — we’d never a/b tested for nps optimization before, and our nps collection survey only went to students after their exams.  it would be months before students who saw the changed algorithm took their exams and we got back nps data.  making a significant change without knowing how it would affect our word of mouth marketing was a big risk.

our solution:  bring nps inside our product

we determined that in order to a/b test the algorithm change, we needed a method for collecting nps data while students were still studying — not just waiting til the end of their exam.  we began using a third-party tool called wootric, which allows us to ask the nps question in our product and analyze the data in real-time.  we then deployed the changed algorithm to half of our gmat students, and we could then match the “likely-to-refer” rating to students in the treatment and control groups.  suddenly nps had a new use case for us — as a powerful, agile product tool.

it turned out that the improved algorithm did not affect student satisfaction while studying with magoosh — nps from both student groups was identical.  knowing this allowed us to roll the change out to all students more quickly.  we were also able to track the students in the a/b test over time, and have seen that post-exam nps for students in the treatment group is a full nine points higher than for the control.

takeaways from a/b testing for nps

1)  include current customers in your optimizable funnel

our goal is always to provide our students with the best possible test prep experience.  but since we’re not able to read minds, it’s not always easy to know if what we’re doing is actually providing a great experience.  it’s easy to think of customer acquisition as a funnel, and to wrap our brains around how to a/b test to optimize that funnel.  but what doesn’t come easily (at least for most startups — and definitely not for magoosh, at first) is to think of current customers as part of an optimizable funnel too.

2)  optimize your products for referrals

if your business is built on recommendations and word-of-mouth, then you really can’t afford not to optimize your products for referrals.  this process has helped us make sure that what we’re doing is making a meaningful difference for students, and has provided us with a useful and repeatable framework for testing future features and products.

3)  focus on agility

shift your thinking on nps from a one-time transactional model to an ongoing and contextual model.  in-product nps tools available today like wootric can help you do this easily, as well as keep track of your a/b test groups.  you can speed up decision making and keep your pulse on customer happiness.

read about it on wootric’s blog too! a/b testing to optimize net promoter score at magoosh

the post how we optimized for student happiness using a/b testing appeared first on magoosh 2022年足球世界杯举办地.

last time we reviewed studyblue (an app for storing and accessing class notes, flashcards, and lectures), but today we turn our attention to mathway!

mathway is a tool that takes the guesswork out of solving math problems.  mathway can solve equations, simplify expressions, and evaluate limits.  it can do everything from reduce a fraction to integrate a function.  it’s a powerful app that can help you get answers on math problems from pre-algebra to college calculus.  so how did it fare in our review?  let’s find out!

usability: a

this app has a lot of features, and is easy and intuitive to use.  the mathway app can do as much as most graphing calculators, all on your phone.  it’s stable and fast, and makes finding answers to your math problems super quick.  you can even save problems for later if you’re doing work on the go and want to save the problems for later.

design: b+

mathway is very well-designed.  everything works just as you think it would — no small feat for something with no buttons!  it’s only downfall: it isn’t pretty.  only you can decide if this really matters, but when solving a problem i’d prefer something that works well to something that looks nice.

effectiveness: b-

so this is where mathway falters — not because of what it does or how it works, but what it signifies.  mathway can solve all of your math problems for you.  it can show you how to do it step by step.  it can make sure that all your problem sets are done, and done correctly.  but it can’t teach.

now, that’s not to say that you can’t use mathway to learn math effectively.  you can use this app to check your work or to give you the blueprint for how to solve future problems.  but if you just plug every question into your app and copy down the answers, you aren’t going to learn anything.  you should be learning how to do these things without mathway so that you can apply them at a higher level.  this app is powerful, but beware.  you don’t want to become addicted.

addictiveness: b

didn’t i just say “don’t become addicted?!”  ok, so this app isn’t addictive to use in the sense that it is fun or compelling.  you’re never going to say “oh, let me go back to mathway to see how my equations are doing.”  it’s not a tamagotchi.

what makes it addictive is how useful it is.  the right answer (with explanations, if you pay for them) to nearly every math problem in your textbook.  now that’s something to get addicted to!

but this is where it’s addictiveness can be dangerous — imagine you use mathway to get a’s on every homework assignment, but come test time you have no idea what you’re doing.  it would be easy to use this app to get all the answers to your homework and move on, but that would be a mistake.  mathway can be a great tool, but it shouldn’t be your first tool in solving a problem.

ad distraction: b+

mathway manages to be very useful, even if you don’t pay anything.  mathway gives you the answers to all your problems, but if you want to see the step by step explanation then you need to pay! a subscription costs $19.99 a month, or $99.99 a year.  see the images below for what the solutions look like without a subscription.

if you’re using mathway to check your answers or to get quick solutions, then the free version is great.  if you need more step-by-step help, then you’ll probably need a subscription.

motivation: b+

there is nothing in the mathway app that is specifically motivating.  and it’s possible that using it and becoming dependent on it could be extremely demotivating.  but, if you’re the type of person who is prone to making small mistakes, this app could be a lifesaver.  you can go about all the right steps, but forget to carry the 1, and wind up with the wrong answer.  using mathway to check your work, you would catch that — hopefully helping you get better marks on your homework and teaching you which mistakes you’re most prone to.  but again, beware: too much of a good thing can be a problem!

support: b

mathway doesn’t provide tutors or a help center.  it doesn’t really have any support to speak of.  but i don’t know that it needs it — you’re paying for a answers and steps, not a math tutor.  the app works well, so customer support isn’t going to be necessary.  still, this does highlight an important point: don’t download this app thinking you’re going to learn math from it.  there’s no one on the other end to the line to teach you.  download it thinking you can learn math with it.

overall gpa: 3.2
overall grade: b+

the post education app review: b+ for mathway appeared first on magoosh 2022年足球世界杯举办地.

first, a little about me (and why you should pay any attention to my reviews). i’m the content team lead at magoosh, meaning that i support our teachers and test prep experts to make sure that our students are getting the best possible lessons, questions, and explanations. before magoosh, i was a high school teacher and instructional coach, meaning that i’ve spent most of my adult life teaching students and working with teachers.

here’s how the reviews work: i’ll grade different aspects of each app on a scale from, you guessed it, a+ to f-. at the end i’ll tell you the app’s overall gpa and grade. new reviews will be posted every two weeks (or so — we’re busy here!). so without further ado…

the first app: studyblue!

studyblue is a platform for finding and sharing study materials — everything from class lecture notes to flashcards to quizzes. much of the content is user generated, although you can also join specific classes at your college or university and see professor-uploaded content as well.

usability: c

the app is stable and fast, moving quickly from class lists to content and back. you can bounce around a lot, and the app keeps up with little wait time. the app does have two main usability problems, though. first, there are no reviews of different content, so in order to determine whether one set of, say, gmat flashcards is better than another, you actually have to look at them all. further, some user-generated content is password protected, and some is not — but it’s impossible tell which is which when scanning through the different search results. so to find a flashcard set that will work for you, you have to try to look at them all — but sometimes you can’t.

design: b

it’s a well-designed app that’s easy on the eyes and simple to use. some small design flaws prevent it from getting an a, though. the inbox messaging system sends messages that run off the screen and contain links that take you out of the app. other icons and logos aren’t clear, and you can spend some time going down a rabbit hole looking for what you want.

effectiveness: b-

well, this is a tough one. studyblue is a platform, and users, professors, and fellow students can upload content to study. so depending on what you’re looking for (and whether or not you can find the best resources) studyblue might be really effective — or really not. if you’ve got a specific course or piece of content that you’re looking for, it can be great. but for content discovery, it’s not the greatest.

addictiveness: c-

the point of studyblue is to keep all your notes and study materials in one place. this app is a work horse, but it’s not fun to use. if you need access to lots of study materials or notes, then this is your app. but if you need games and badges to help keep you motivated to actually look at those materials, then you should look elsewhere.

ad distraction: c

there are no traditional ads, but lots of content is unavailable unless you upgrade to pro. pro membership costs vary depending the length of subscription ($7 per month for a full year, or $18 for one month).

motivation: b-

studyblue is only as good as the content you access through it. some of that content is going to be engaging and motivational, and some will not. the tone of studyblue is not aspirational — they emphasize helping you get the material you want to accomplish your assignments. it’s certainly not de-motivating, but the focus is on doing what you have to do, not what you want to do.

support: b

the app contains several helpful tutorials and explanations, complete with screen grabs and images. there isn’t however any way to send in a question to a help team. so if you can’t figure it out on your own, tough nuggets!

overall gpa: 2.4
overall grade: c+

the post education app review: c+ for studyblue appeared first on magoosh 2022年足球世界杯举办地.

today, we’ll talk about magoosh content — every practice test, vocab quiz, and blog post that thousands of students rely on every day to study for exams. when it comes to content, here’s how we break it down:

what do we do?

create consistently excellent academic content.

why do we do it?

because students all over the world depend on our product to get into the colleges and grad schools of their dreams. and because we’re a group of passionate educators determined to make magoosh the best online education experience possible.

how do we create this maddeningly-good content?

by using our values.

by focusing on values, we don’t limit ourselves to one philosophy of education or another. on the content team, we gather ideas and inspiration from everywhere, and we weave our values into every decision we make and every test question we write. you’ll find magoosh values at every step of the content creation process, from our teaching styles to specific practices. here’s how.

teaching & our values

as content creators we are also teachers — real people interacting with and influencing real students. that means before we dive into any kind of content creation we must first use our values to shape the way we communicate information. in the end, we should be able to look back and say three things:

1. we are great teachers
great teachers make the difference between an amazing experience and a mediocre one. our students like our teachers. much of online learning is devoid of humanity: a robotic voice; a disembodied and personality-less narrator; no fun, no joy, no encouragement, and no help. our products feature teachers and tutors who care, encourage, smile, and assist.

our students enjoy learning from us because we are skilled educators who understand that motivation can be as important as knowledge and skills.

supporting values:

friendly > formal

accessible > exclusive

passion > balance

wow > profit

2. we do what works for students
despite our excellent teachers, we don’t rely solely on their expertise. we use data to challenge traditional notions of what works, and we make sure that everything we do actually improves student learning.

supporting values:

challenge > comfort

data > intuition

wow > profit

communication > efficiency

3. we are students too
while the magoosh team includes excellent teachers, engineers, and tutors, we don’t presume we know everything. we push outside our comfort zone to learn about new tests, new materials, and new methods. we develop interesting ideas and test them scientifically. other education products are created, shipped, and finished. but we believe that our students have as much to teach us about our products as we have to teach them about their exams.

supporting values:

done > perfect

learning > knowing

change > status quo

data > intuition

the roadmap to content creation

when creating products for our students, we never make a move without some indication that it’s what students want and need. we validate each decision at every step, particularly seeking out ways to conduct fast, small-scale tests whenever possible.

at magoosh, we follow the practice of creating “minimum viable products” — products that are as small and easy-to-build as possible while still being useful and valuable to our students. we then follow an iterative product development process to refine and improve our product, making it increasingly valuable to students. this cycle applies when we’re doing something as substantial as a mobile app or as tiny as an email message.

for this process to work, we have to collect and analyze tons of data. and we do!

we collect data on every lesson and question in magoosh, as well as on all the features and products. using item discrimination, we can tell if a question is problematic and fix it. by studying student feedback and the amount of times they “like” lessons, we can improve our courses so they have the biggest impact. and by analyzing the behaviors of students who have the largest score increases, we can help push students towards features that will grow their scores the fastest.

every time you log into your magoosh account, you should feel confident that you’re studying with the highest quality content out there, carefully written and selected to raise your scores. we work hard on our end to make sure the studying process is easy and enjoyable on yours. and any time you tell us we’ve eased your path to college or grad school, it reminds us why we do what we do.

—

photo at top courtesy of wikimedia commons user bcds. image licensed under creative commons attribution-sharealike 3.0.

the post using values to create better online learning appeared first on magoosh 2022年足球世界杯举办地.