ChatGPT and Github Copilot being able to quickly answer interview-style questions and leetcode exercises. Sites like Hackerrank or Codility can only offer comparison of the solutions that have been submitted. Already today, they flag duplicate and similar solutions as ways to combat leaked questions and copy-paste submissions. It seems that the effectiveness of those platforms is lower with time for verifying whether the candidate knows how to code.<p>There is a difference between the tool increasing the productivity of a developer vs. using the tool to pass an interview while not having the know-how. ChatGPT is especially good at knowledge questions and such that compare approaches X with Y. This means that it's necessary to approach interviewing differently in order to extract more signal from the interview session.<p>How are you planning to adjust the ways you interview candidates for Software Engineers?<p>One way is to test for experience and dive deeper into past projects of a candidate and let them present how they solved problems, why and what alternatives they considered. This puts those under NDA at a disadvantage as they can't speak about their past projects.
The tools can _answer_, but they tend to hallucinate functions and arguments that seem plausible but are wrong upon closer inspection.<p>This output is an example: the generator made up an option for Macro.escape and completely misunderstood what Macro.to_string does:<p><a href="https://elixirforum.com/t/chatgpt-generated-code-for-eval-string-alternative/52544" rel="nofollow">https://elixirforum.com/t/chatgpt-generated-code-for-eval-st...</a>