> "I decided to test GPT-4 as follows: I’d take a personality test. Then, I’d give GPT-4 some information about me and ask it to take the same personality test—but pretend to be me. I’d tell it to attempt to predict the ways I’d answer the questions based on what it knows. Then I’d measure the difference between what GPT-4 predicted my responses would be and my actual responses to see how accurate it was. I’d also ask my girlfriend, Julia, who is unusually perceptive about people and knows me quite well, to do the same task. Then I’d compare GPT-4’s accuracy against hers."<p>I wonder if he gave his girlfriend the information that he "give GPT-4 some information about me". Maybe if the girlfriend saw what he was emphasizing in that information, she could have better guessed what he would have put on the test. Another possibility is that he asked his girlfriend what she thought he would <i>actually</i> do in those situations, whereas he asked the bot to guess what it thought he would put as answers to hypothetical questions (which wouldn't necessarily match what he would actually do in those situations).