The Tech-job Interview: Secrets of Making It Fair and Effective

BY Emily McCrary-Ruiz-Esparza | November 12, 2021

Everyone has had the experience of interviewing for a job and being turned down. The answer you got was likely something like this one I received once: “While we were very impressed with your qualifications, we were faced with a difficult decision; we finally selected another candidate who we believe more closely fits the position’s requirements at this time.”

What if, instead of giving interviewees a yes or no answer about the job, companies were more candid about missing skills that might have changed the outcome? What if promising candidates were given the opportunity to try the interview again? And what if employers focused on building relationships with talented candidates before they’re ready to hire?

“It was crazy to us at Karat that candidates just get rejected for jobs and companies never track their aptitude or how they’re progressing,” said Mohit Bhende, cofounder and CEO of Karat, a company that is pioneering the Interviewing Cloud to conduct technical interviews for enterprise-scale hiring of software engineers. “I would say that forward-looking clients, they've stopped rejecting candidates. Instead, they say, ‘Hey, you're not ready.’ They use the term ‘not ready,’ rather than a binary yes or no. And when you lead with that, it changes the equation.”

This simple change has led Karat’s client companies to form relationships with job candidates, grooming larger, more diverse and richer talent pools. It has helped them improve their reputation as an employer and create networks of potential future hires. “That is a total mindset shift on what the interview can be, to not think about the interview as a one-stop shop or one point in time, but to think about it as a way to build a relationship with a candidate,” said Bhende, who I interviewed in a recent From Day One webinar titled “The Inclusive Job Interview: How It's Making Tech Recruiting Effective, Efficient, and Equitable.” We discussed how employers can reimagine the interview process to increase efficiency, improve employee retention, and support goals toward diversity, equity, and inclusion (DEI).

What Happens When You Rethink the Interview Process

Karat has seen first-hand what happens when the interview process is thoughtfully planned, carefully executed, studiously measured, and constantly improved. “The most immediate benefit,” Bhende said, “will be maximizing your current workforce so that you just get more throughput, happiness, morale, and retention.”

The very idea for Karat was born out of this need. Bhende needed to hire hundreds of engineers for his team at Xbox, where Bhende led global product strategy in 2010-13. “I started one day just counting the sheer number of hours that my engineers were spending interviewing, versus coding. None of our engineers were formally trained in how to be good interviewers, and so the net result was countless hours going into a process that was oftentimes inconsistent.”

In creating Karat, Bhende and his cofounder Jeffrey Spector took it upon themselves to solve the efficiency problem, building a network of engineers, working outside of a given organization, who are trained to interview for consistent and equitable results.

Beyond creating immediate efficiency, the effects of a well-conceived interview structure are long-lasting. It brings in talented people who add value to an organization–Bhende favors the notion of “cultural add” over “cultural fit”–and it helps companies prepare for the future in a consistent manner.

A conversation on job interviews: Mo Bhende, CEO of Karat, and journalist Emily McCrary-Ruiz-Esparza (Image by From Day One)

Hiring practices that invest in candidates beyond a single interview address the growing skill gap. Forty-three percent of executives and managers say their organizations currently have a skill gap, according to a survey by McKinsey & Company, and an additional 44% say that a gap will open in the next five years. The same survey indicated that executives and managers who prioritize skill-building among current employees are more likely to consider themselves prepared to address role disruptions caused by the skill gap than are those preparing for the skill gap through other methods, like hiring contract workers.

Interviews in which the evaluator interacts closely with the person being evaluated are better at testing for aptitude and potential than traditional interviews, Bhende said. “Most interviews today are really focused on ‘Can you do the job today?’ But what you should really be thinking about is, ‘If I give you learning and kind of guidance in the interview, can you learn and demonstrate the ability to learn the job I want you to do?’”

How Interviewing Practices Affect Black Engineers’ Access to Tech Jobs

The role of well-designed interview programs in furthering DEI in the workplace, especially in tech environments like the ones Karat supports, shouldn’t be underestimated. “There is a sheer processing and expansion of opportunity that a well-run tech program will do to drive better and more equitable outcomes,” Bhende said.

Karat has studied this effect. In September, the company released “The Interview Access Gap for Black Engineers,” a report created in partnership with Howard University, which examines how exposure to computer science education, tech industry networks, and interview practice opportunities–or lack thereof–affect the ability of Black engineers to land tech jobs. The survey found that access to these three factors significantly increases candidate confidence, reduces feelings of imposter syndrome, and ultimately influences career trajectory.

Bhende said the companies that achieve and maintain diversity goals will do so because they prioritize candidate-centric interviews. “What do all of the diverse candidates that we brought into the company seek and want? They want learning, they want growth, they want fulfillment, they want compensation. It's such a simple metric flip. If you start with the customer and work back, it'll yield a better outcome than starting with the business objective.”

How to Design an Effective and Equitable Interview Program

Though Karat specializes in conducting technical interviews for companies hiring software engineers, the principles can apply to interviews more broadly. Bhende encouraged employers to think of it as skills-based interviewing. “What Karat is really focused on is demonstration of craft, demonstration of skills, demonstration of expertise. And so the nature of the interviews are inherently technical in that they are evaluating your ability to learn the job or do the job.”

        1.) Name the Who and How: The first step is considering who will conduct the interviews, and how. Bhende was clear that it shouldn’t necessarily be the first person to volunteer. “Just because you raise your hand or you’re a good engineer doesn't necessarily make you a good interviewing engineer,” he said. “I think our data has been very clear that those are actually fundamentally different skills.”

All Karat interview engineers go through training that includes how to use technology to mitigate bias. Bhende said that in their process, the interviewer “never scores the candidate, all the interviewer does is say, ‘Here's what happened in the interview.’ Our tech scores the candidate.” Karat interviewers are also equipped with “battle-tested” questions–the company tests their questions across broad cohorts for bias–and the methodology for asking them. The Karat program even cues the interviewer about the appropriate times to give hints, in order to reduce the effect of subjectivity on the part of the interviewer.

     2.) Align It With Your Identity as an Employer: Identify the goals of the interview and the impression it will leave on the prospective employee. “How is that interview program going to reflect the brand experience and the employee experience that comes downstream? The employee experience starts at the point of hire.” For example, many professionals now work from anywhere, anytime–and yet the interview process to get one of those jobs often follows the old 9-to-5 model. Bhende believes the two should be more closely aligned. “How you get hired and how you work, ideally, are reflective of each other. And so the hiring process itself should be inclusive, it should be flexible, it should be candidate-responsive.”

That means offering interviews at times convenient to candidates–when they perform at their best, on days that work best for them. “I think the future of work is quickly evolving and interviewing is evolving from candidates fitting into a company model to one that is much more flexible and ultimately accommodating.”

Flexibility also includes rethinking how companies relate to candidates. Don’t underestimate the power of a do-over, Bhende said. “It’s ridiculous that people are coming to interviews, not doing well, and they will forever never get the job,” he said. Candidates often perform better on their second tries, he said, with “convincingly high” improvement ratios. “I would just encourage the industry as a whole to adopt [this], because I think it's just a higher-empathy way to lead.”

      3.) Plan for Internal Communications: The program should clearly identify how communication between the interviewer and hiring team will work. “How are the handoffs going to work between the recruiter to the engineer, back to the recruiter? Having a guideline of what that's going to look like is really critical,” Bhende said.

      4.) Keep Improving the Process: “I would encourage any company that is thinking about designing a program to really think hard about where you are going to get signals to improve that interview over time. It's really critical,” he said. “Even our own interviews, they’re constantly adapting and constantly changing for companies as we get more data on what's predictive, what really matters. The interview can evolve and change.”

Bhende said Karat asks interviewees to score as much of the interview process as possible. “They score the company, they score the question, they score everything.” All of this is in the service of treating the candidate like a customer, a practice Bhende repeatedly recommended during our conversation. As corporate America moves toward providing a consumer-grade employee experience, feedback is indispensable.

Editor's note: From Day One thanks our partner who sponsored this webinar, Karat.

Emily McCrary-Ruiz-Esparza is a writer, editor, and content strategist based in Richmond, Va.


RELATED STORIES

Sharing Their Truths: Working Parents Reveal the Benefits That Matter Most

Each year, HR leaders ask themselves: What benefits do my employees want? And what will provide me the most ROI? But many are left without answers.In a recent survey of 2,000 working parents conducted by Ovia Health, 62% said that their employers are not family friendly enough.The need for family friendly benefits is clear. Additionally, 94% said family benefits are a top priority and 73% said they would consider making a lateral move to another organization that offered better benefits and a family-friendly culture.In a From Day One webinar, Corrinne Hobbs, general manager and vice president, employer market organization at Ovia Health, discussed the results of the survey. Hobbs offered insight on current benefits offerings, where more support is needed, and what matters most to employees. Family Benefits That Match Today’s Culture“Women’s health benefits are one of the fastest growing segments within healthcare,” Hobbs said.  This is due to changing circumstances during and post-pandemic as more and more workers experienced shifting work-life balance due to hybrid schedules. It’s also due to the increasing range of types of families that need to be accounted for as lifestyles become more diverse. In this current marketplace, “employees have more control and more power than they have had in the past,” said moderator Siobhan O’Connor, chief content officer at Atria Institute. Therefore, it’s even more critical that employers make sure these specific needs are being served.While most companies do offer some family benefits, Hobbs says, there is often a disconnect between perceived needs and actual needs of employees. “There’s a strong push for employees to have better fertility benefits in their workplace. And 38% of respondents said that they’re looking for their employer to provide alternate family planning support,” Hobbs said. This is especially true with more and more single by choice or LGBTQIA+ parents in the workforce, and an overall trend of people waiting until later in life to have children. Unfortunately, many workplaces do not offer benefits to cover the costs of these services, which can be exorbitant.Siobhan O'Connor of Atria Institute interviewed Corrinne Hobbs of Ovia Health during the webinar on family-friendly benefits (photo by From Day One)Incorporating these benefits helps build an overall inclusive corporate culture and can be a way to help retain senior level female employees. Additionally, 83% of respondents said that perimenopausal or menopausal symptoms affect their ability to work, but only 1% receive benefits to help with those symptoms, says Hobbs. In order to “make sure that whatever you’re providing is equitable and inclusive all around,” a diverse range of age and gender must also be factors incorporated into a comprehensive benefits plan.Providing Better Family BenefitsWith family benefits top of mind for employees, Hobbs says there is a clear way forward for organizations looking to provide better care. The most important, according to respondents, is family leave. Hobbs advises: “Make sure that it’s paid, that it’s for at least four months, that it’s inclusive to both parents and that you don’t have to dip into your sick leave or your PTO before taking leave. That is a stress factor for many.” And employers must account for alternate pathways to parenthood, such as adoption, which might entail different costs or timeframes, she says.Hobbs says employers should not only plan for parental leave, but also for parental return. One way to do this is by setting up a return-to-work program to make it easier for parents to re-enter the workforce, noting that it’s a smarter investment than having to endure the cost of hiring someone new. Gradual part-time schedules can ease the burden on stressed parents, as can accommodating PTO policies, flex time, and hybrid or work from home options.Additionally, managers need to be prepped on how to work with returning parents. “A manager training program to ensure a family friendly workplace and ensure that people are able to bring their full selves to work without fear of repercussions is critical,” Hobbs said. ERG support groups can also provide a sense of community support within the workplace.Incorporating Digital Healthcare and AdvocacyOvia Health uses predictive analytics to power millions of members’ care and engagement with their health. Such apps can help provide crucial education about health symptoms, Hobbs says. For example, 85% of respondents said they don’t know much about menopause and how it may affect their performance. Ovia can help fill that gap through online resources, and also provide peer support groups. “We have a community wall where people with uteruses can talk about symptoms together and really feel a sense of community and commonality with others who are going through some of the [same] things,” Hobbs said. Finally, Ovia can also match employees with proper treatment.Using health assessments and surveys, Ovia gets to know its users and can provide highly personalized information to current, expecting, or potential parents. Health alerts will pop up based on users’ reported symptoms, and the app even provides proactive healthcare outreach to guide users through any bumps on their fertility journey.“Digital solutions offer round the clock access, education, and opportunities to really delve deeper into topics,” Hobbs said. “And they also come with advocacy, helping you navigate and understand these complex situations.” The app accounts for a wide variety of families and lifestyles, helping employers provide better care to a diverse workforce. “We have 50+ personalized clinical pathways and programs to support women and families, and then we personalize the experience for each member based on the dynamic health assessments and digital symptom report,” Hobbs said, describing the data-driven service as “person-centered care.”Hobbs says that while women have increasingly reached the upper echelons of the corporate world in recent years, women’s participation in the labor market is currently at a 33-year low. Having a family-friendly workplace can help ensure talented women stay on. “It costs upwards of $75,000 to replace an employee,” Hobbs said. By offering a diverse suite of benefits companies can retain top talent, encourage a more diverse workforce, and save money in the process.Editor's note: From Day One thanks our partner, Ovia Health, for sponsoring this webinar. Katie Chambers is a freelance writer and award-winning communications executive with a lifelong commitment to supporting artists and advocating for inclusion. Her work has been seen in HuffPost and several printed essay collections, among others, and she has appeared on Cheddar News, iWomanTV, and CBS New York.

Katie Chambers | April 10, 2024

How to Measure Employee Engagement and Spot Disengagement

When we think about engagement, we think about all of the different ways that we track engagement consciously and subconsciously. In some ways, we track engagement by just realizing things, like who’s on camera during meetings online, who has a green dot next to their name, and who has a yellow dot next to their name. These are all of the different ways to subconsciously track engagement, but there are biases in each observation because context is key. Just because a person is off camera doesn’t mean they’re less engaged. They might be in a crowded spot or have a background that’s distracting, so they’ve elected to be off-camera. Or maybe their WiFi just isn’t as strong as it needs to be on that particular day.Regardless of the industry or nature of business, maintaining a high level of team productivity is crucial, and disengagement can be a significant obstacle. Learning to recognize the signs of employee disengagement early is key to preventing its negative impact. In a recent From Day One webinar led by ActivTrak colleagues, Gabriella Mauch, VP of Productivity Lab, and Javier Aldrete, SVP of product, the speakers discussed how boosting self-awareness and manager coaching can help address disengagement before employees check out.Gabriela Mauch, pictured, led the webinar alongside colleague Javier Aldrete (company photo)We’re making all these subconscious assumptions about engagement because we know that engagement leads to great results, says Mauch. But disengagement, on the flip side, leads to harmful attrition. As such, it’s important that we find better ways to track engagement so that we can drive to a healthy work environment. Mauch shares that only 23% of employees are fully engaged in their work, leaving over 75% of employees at risk of disengagement. This can cost organizations a significant amount of money, both from an attrition standpoint, a knowledge management standpoint, and the productivity they’re not necessarily getting out of their business. The benefit of addressing employee disengagement is the ability to get a better return on workforce investments. Organizations can see up to 40% improvement in employee churn and burnout rates, plus an opportunity to gain 15% to 25% in productivity when disengagement is addressed effectively, says Mauch. “So often, disengagement and quiet quitting is a function of that individual not being properly aligned to their work, not being properly coached by their manager, or not being properly guided by their leadership team,” said Mauch. It’s  important to learn how to use insights to better inform leaders, managers, and individuals to be more thoughtful about productivity and more engaged in the work being done. As such, it’s important to have measurable indicators into our work environment. This means understanding when we have individuals performing with low focus, low working hours, and perhaps very passive participation. It doesn’t necessarily mean that the individual doesn't want to be working. Instead, there’s an opportunity to coach and guide the employee to work the right way, on the right things, at the right time. Mauch encourages employers to be thoughtful about employee behavior as a helpful indicator of engagement. This means observing things like people coming into the office, badging in, and leaving two hours later merely to show their faces. This could be because while they are expected to be in office, they might actually be more productive at home. The final thing to note is whether or not employees are making the impact you expect them to be making. Here are some questions to ask: Are they putting in the productivity that you would expect? Are you getting the output that you expect to earn, and are you ultimately getting the revenue that you would expect? By collecting insights on an ongoing basis, you can gain a level of understanding of engagement on an ongoing basis. Additionally, leaders need to identify the factors that are contributing to employee disengagement and quiet quitting in their particular context, as well as invest in measures to improve them.Editor's note: From Day One thanks our partner, ActivTrak, for sponsoring this webinar. Keren's love for words saw her transition from a corporate employee into a freelance writer during the pandemic. When she is not at her desk whipping up compelling narratives and sipping on endless cups of coffee, you can find her curled up with a book, playing with her dog, or pottering about in the garden.

Keren Dinkin | April 09, 2024

The Gender Penalty: Addressing Workplace Inequity

Studies show that despite recent movements for equal pay, no significant gender pay gap has been made in the last two decades. Women are still earning less than men, with some variance as high as 22%.But the discrimination extends far beyond just the pay gap: from childbirth to menopause, women are also discriminated against for their life choices and in some cases, life stages, with  42% of working women reporting facing gender discrimination at their workplace.In a From Day One webinar, Lydia Dishman, senior editor of growth and engagement at Fast Company, moderated a discussion among women in roles of leadership on how to achieve equality in the workplace.Studies show that women are 41% more likely to experience toxic workplace culture than men, underlining the need for a culture revamp in companies.According to recent research, one in three working parents stated they lacked access to a reliable workplace lactation location. The disparity shows that offering solutions is far more than checking off boxes, Teresa Hopke, CEO of Talking Talent said.“Having a pumping room is a checkbox. So even if we check the box and we get the right rooms and accommodations for people, that’s not going to move the needle in the way that we need to in terms of the systemic change that needs to take place,” Hopke said.For change, both workers and leaders need to be actively working to create the shift that they need, Hopke says.Speakers from Talking Talent and KPMG joined moderated Lydia Dishman in a discussion about the role of gender in the workplace (photo by From Day One)“There is some hard work that organizations need to do to create the right culture with the right mindsets, behaviors, conditions, and structures that will support women as they advance through their careers,” Hopke said. “There is also work that women need to do to articulate their needs and not suffer in silence when the load gets too hard.”When asked about allyship, seventy-seven percent of white employees consider themselves allies to women of color. However, far fewer replied to actively participating in allyship, with only 39 percent stating they confront discrimination when they see it, and 21 percent stating they advocate for new opportunities for women of color.“If you are not taking any of those ally actions regularly, you’re not moving things forward in a positive way,” Marcee Harris Schwartz, director of diversity, equity, and inclusion at KPMG, said. “We have to think about how we activate allyship whether that’s taking someone under your wing who comes from a different background or experiences so that it has an impact.”When asked about biases at work, 83% of employees stated that the biases they experienced were subtle and indirect. In one work case, Renu Sachdeva, head of client solutions at Talking Talent, found this to be true.“We asked leaders to pick people to actively sponsor who belonged to these identity groups. And when the results came back in, we found a majority of them had selected white women, the next most selected group was men of color, and the least selected group was women of color,” Sachdeva said.The findings weren’t surprising, Sachdeva says. Research has found that white people demonstrate a clear bias for other white people, affecting workplace processes from hiring to promotion. Challenging biases is key to moving allyship in the right direction, Sachdeva said.“If you’re talking about the majority, corporate America is usually white men in most organizations, so the highest level of comfort tends to be with white women because there’s a relational aspect to it,” Sachdeva said. “But with intentionality, we need to consciously choose to connect with [different] people to mentor, sponsor or be an ally to because that’s usually the group that gets the most overlooked and left behind.”Editor’s note: From Day One thanks our partner, Talking Talent, for sponsoring this webinar.Wanly Chen is a writer and poet based in New York City.

Wanly Chen | April 08, 2024