Detecting Cheating Candidates - Sandbox

While most Karat candidates are honest, and it is very important to enter every interview without bias or assumptions, it is important for an IVE to know how to detect when a candidate is cheating.

Cheating candidates can expose their cheating in many ways. The following section is a guideline, though it is important to pay close attention during the interview before making accusations. All cheating accusations will be reviewed and validated by QC, so please leave as specific notes as possible, including relevant timestamps, in the interview results. Details for reporting this in the results can be found here.

Important Reminder: An IVE must never discuss cheating with a candidate or call-out a cheating candidate. The interview must be conducted as if the candidate was not cheating.

Possible observations for cheating candidates

 There are many things that can be noted in an interview to demonstrate a candidate is cheating. They range from Blatant to more subtle.

Blatant Cheating

• Someone else can be heard giving the answers or talking to the candidate.
  • Also look for significant inconsistencies between the candidate's movements and their speech, ones that video lag cannot account for.
• The candidate pastes in a solution or a major section of a solution
• Candidate quickly writes a perfect solution line-by-line but provides a glaringly incorrect explanation of approach or code
• Code matches online solution or another solution exactly
  • Also provide a link to this solution in the interview results.
• Candidate writes code for the wrong question from the same set, for example, Q2 when Q1 was presented.

Suspected/Less Obvious Cheating

These are more subtle, and are not always signs of cheating, so it is important to closely observe the candidate before suggesting they are cheating.

• Candidate doesn’t explain their approach and instead jumps immediately into a completely new and optimal solution
• Candidate writes code top-down without jumping around to add if-cases later or move something into a function, etc
• Candidate appears to look down at other papers or a laptop or switches windows when explaining the approach or optimality, or when writing code.
  • Especially when they appear to look back and forth frequently
• Candidate says code works and cannot come up with any edge cases for a perfectly optimal solution
• Candidate cannot say anything meaningful about complexity for an optimal solution they had no trouble coding
• Solving Q1 with an extremely complicated solution useful for later questions in the set

Resources

Check out two lessons in the learning library related to cheating: Recognizing Suspicious Candidate Behaviors and Evolution of Candidate Cheating.

Measures from Karat to Address Candidate Cheating

Karat Studio: Candidate Copy/Paste Toggle [Launching February 18, 2025]

In the ‘Markdown’ mode, candidates will not be able to copy and paste text by default. 

You will notice a new toggle at the top of Karat Studio and to the left of the language selector called ‘Candidate Copy/Paste.’ This toggle controls the candidate’s ability to copy/paste in Karat Studio. 

[Default Setting] Copy/Paste functionality is disabled for candidates in Karat Studio when the toggle is OFF. 

IVEs can enable the Copy/Paste functionality for the candidate by clicking the toggle to purple to turn it ON. 

What changes when the Candidate Copy/Paste toggle is off?

• What changes: Candidates can’t copy/paste text in Karat Studio on either the left (input) or right (output) side.
• What stays the same:
  • IVE can still copy/paste text, regardless of the toggle setting.
  • Both IVE and the candidate can still type text as usual in Karat Studio.

The controls are in the IVE's hands. This toggle is only visible to IVEs, not the candidate. You should enable this functionality for candidates in types of exercises like:

• Object Oriented Design Code Reviews
• Review and Update Code
• API Design Exercise

This is not an all-encompassing list; IVEs are trusted to judge when a candidate needs this functionality enabled during an interview.

Candidate Copy/Paste Toggle FAQ

What if a candidate asks about copy/paste functionality? If a candidate asks why copy/paste isn’t working for a particular exercise, you can inform them that they are expected to answer the questions verbally and do not need to write anything at that time. 

Will the script state when I should toggle candidate copy/paste on/off? The scripts for exercises where candidates are expected to write in Karat Studio in ‘Markdown’ mode will not be changed. Some exercise guides will have specific notes when this functionality is more critical to the question such as API Design.

Can candidate copy/paste be enabled for more than what is mentioned in this article, a guide, or a script? IVEs have the discretion to enable copy/paste for the candidate when necessary. The listed exercises (e.g., OOD Code Reviews, API Design) are just a few examples, not strict limitations. If you believe a candidate needs to copy/paste in Markdown mode for another reason, you can enable the toggle. 

How will the use of this feature be evaluated in QC? If the improper use of the toggle causes a negative impact on the candidate, such as not being able to copy the template for the API Design Exercise then a markdown in QC can occur.

Updated Cheating Options in the Results Form

Two additional options have been added to the cheating section of the results form. These changes aim to enhance the accuracy of our data, enabling Karat to better adapt to and learn from common cheating techniques:

• “Candidate highlights the text of every problem in the coding environment.”
• “Other” – This option now includes a required free-form text field to provide additional context

Updated Candidate Emails

Emails sent to candidates before interviews now include a more apparent reminder about cheating and a warning to discourage such behavior.

Universal Script Update

The universal introduction script for all interviews has been updated to discourage cheating further:

• “All answers you provide during the interview must be your own original work. If any behavior appears suspicious, I am obligated to flag it for review by the hiring team. I’ll conduct the interview the same regardless of anything flagged.”

If candidates ask for examples of suspicious behavior, here are a few you could share:

• Other voices in the background
• Highlighting large chunks of text
• Significant unexplained typing outside the Karat studio