Taking Interviews in the time of ChatGPT

The age of AI and LLMs are upon us already. And it’s changing many aspects of life. Making most of the things easy, while making some things obsolate. But one thing is sure, things are not going to be same moving forward.

In the age of AI and large language models like ChatGPT, the way we conduct interviews is changing. Traditional theoretical knowledge-based questions may not be as effective in assessing a candidate’s true potential(I always believe it solves a little purpose). Instead, interviewers should focus on practical problems, staying up-to-date with industry developments, and cultural fit to determine if a candidate is the right fit for the organization. In this post, I will discuss the importance of asking practical questions, assessing cultural fit, and evaluating a candidate’s ability to keep up with the latest developments during interviews.

The Problem with Theoretical Questions

Theoretical questions often make up a significant portion of interviews, but they may not be the best way to assess a candidate’s abilities. Most of the times these questions are easy to predict and easy to memorise. In many cases, these questions fail to provide insight into how a candidate will perform in their day-to-day job. Instead, interviewers should focus on practical problems that reflect the challenges a candidate will face in their role.

Practical Problem Solving

Asking practical questions during interviews can help assess a candidate’s ability to understand instructions and requests, and how they would translate them into code or architecture. This approach also allows interviewers to evaluate how candidates keep track of the latest developments in their field and how they would solve specific problems faced by the team.

In my experience, I have seen interviewers assessing candidates based on pure theory, like how timers and timeouts work, and one or two practical questions where some parallelism is used while printing a sequence. In more than a decade of my experience, I am yet to see a practical module being developed in a team where such functionalities are crucial and should be developed internally without using any library or framework. We always have libraries, framework and enough time of experimentation to solve such problems no matter the language or framework we are using.

Instead, I prefer to ask more peculiar questions related to the framework being used. Like when I am interviewing for Node.js. I ask candidates to replace commas with dashes in a string or differentiate between types of for loops. These questions help me assess whether a candidate has faced any issues in their work, which reflects such topics and how aware they are of these intricacies. Focusing on practical problem-solving during interviews can provide a more accurate assessment of a candidate’s abilities.Most of the time I generate a real world scenario from the things I have developed. And ask questions that reveal how the candidate solves the problem and what concept they use while bringing up the solution. This aspect of the questions help interviewers to understand how a candidate will perform in their day-to-day job and help making more informed hiring decisions.

Staying Up-to-Date with Industry Developments

It’s essential to assess a candidate’s ability to stay current with the latest developments in their field. The time is arrow, it never goes backwards, and it isn’t useful when it stays in quiver. The candidate must be able to know the latest developments happening. And we can assess it by asking questions that require candidates to demonstrate their knowledge of recent advancements and trends. Evaluating a candidate’s ability to keep up with industry developments can provide insight into their adaptability and commitment to continuous learning.

In one of the interviews I conducted, I wanted to assess the candidate’s ability to stay current with the latest industry developments. I asked them about any recent advancements or trends in their field that they found particularly interesting or relevant to their work. The candidate shared their enthusiasm for a new technology they had been exploring and explained how it could potentially improve their team’s workflow and productivity. While this was the very open ended question, the candidate’s response told me where their interest lies and how they are keeping up with the latest developments in their field. In this particular scenario, candidate told me what tools they are using to advance their knowledge, but also mentioned that they are not using it in their work and told me the reason why. Their response also demonstrated not only their commitment to continuous learning but also their ability to think critically about how new developments could be applied to their work. bility and dedication to staying at the forefront of their field.

Cultural Fit

In addition to technical skills, it’s essential to assess a candidate’s cultural fit with the organization. This can be achieved by asking questions that reveal a candidate’s values, beliefs, and attitudes, and how they align with the company’s culture. A strong cultural fit can lead to higher retention rates and better performance.

During interviews, I like to ask open-ended questions to assess a candidate’s cultural fit with the organization. One of the very first questions I ask is about the work they are most proud of. This question can reveal a lot about a candidate’s values, how they respond to challenges, and their approach to problem-solving. For example, one candidate told me that their proudest work was implementing a response in Redis within a week, which significantly reduced downtime for their company. This response demonstrated their ability to tackle challenges and prioritize important problems.

Another question I often ask is how a candidate would spend a week’s worth of free time without any tasks. This question helps me understand their priorities and goals for the immediate and mid-term future. By asking these open-ended questions, I can gain valuable insights into a candidate’s cultural fit with the organization, which is crucial for building a cohesive and high-performing team.

I often give importance to the cultural fit because it’s always easy to learn a skill, and the LLM makes it even easier.
But it’s hard to change the attitude and cultural fit of a person. And as a long time, senior member of the team, we don’t want to be on a position where we have to bakfill same position again and again.

Take-Home Coding Tasks and AI-Powered Coding Assistants

Take-home coding tasks have been a popular method for assessing a candidate’s coding abilities. However, some teams have found success without using take-home tasks in their interview process. With the emergence of AI-powered coding assistants like Copilot or Codewhisperer, the landscape of coding assessments may change, as these tools can assist candidates in completing tasks more efficiently. And we may not know how they are using it, we only know the final product. So it’s better to ask them to explain the code they have written and ask them to explain the approach they have taken to solve the problem. Or if it’s possible, ask them to write or mention prompts. While this is not foolproof because of the nature of models like GPT4, it’s better than nothing.

Offline Tasks and ChatGPT

Sometimes, interviewers may ask questions that require candidates to skim through documentation to solve an issue.¹ In such cases, it’s acceptable for the task to be taken offline, especially if it’s a make-or-break situation. Allowing candidates to use ChatGPT in these situations can be beneficial, but interviewers should ask candidates to share their prompts. This helps assess how candidates work with AI tools and their communication skills.

At this point, I won’t blame you if you think this the post is a clickbait. While putting ChatGPT in the title, I didn’t share any tips or tricks or prompts I passed to them. I am using most of the technique before ChatGPT came. So it’s important to note that the improvements in interview techniques discussed in this post can be achieved without involving ChatGPT or similar tools. By focusing on practical problem-solving, staying up-to-date with industry developments, and assessing cultural fit, interviewers can make better hiring decisions and build stronger teams, even without the assistance of AI tools like ChatGPT.

Conclusion

In the era of ChatGPT, interviewers should focus on practical problem-solving, staying up-to-date with industry developments, and cultural fit to find the best candidates for their organization. By asking questions that assess a candidate’s ability to handle real-world challenges, their alignment with the company’s culture, and their commitment to continuous learning, interviewers can make more informed hiring decisions and build stronger teams, with or without the help of AI tools.

May the force be with you.