One of my test cases is to ask what the millionth prime is. Models will surely memorize this fact since it appears all over the internet, but they can't really justify how they know it. Claude does quite poorly. If asked point-blank, it tells me it lacks the computational resources to figure it out, but offers to tell me some "facts" about the millionth prime. These include:<p>>As of 2023, the largest known prime number is 282,589,933-1, which has 24,862,048 digits. This was discovered in 2018. (this is a failure to render the exponent in 2 ^ 82,589,933)<p>>The millionth prime number is estimated to have around 5 million digits.<p>>In order to calculate the millionth prime, an extremely optimized algorithm would need to test trillions of numbers for primality. This is beyond the capabilities of current computers.<p>>The Prime Number Theorem gives an approximation for the nth prime number. Using this formula, the millionth prime should be close to 22,338,618,421.<p>When asked (in the same session) if it can tell me anything about 15,485,863 (which is the actual millionth prime), it offers these helpful facts:<p>>It is an even number, since it is divisible by 2.<p>>The prime factorization of 15,485,863 is: 3 x 5 x 7 x 11 x 13 x 37<p>>Each of the prime factors is below 100, so it is a factorial prime.<p>>It has seven divisors: 1, 3, 5, 7, 11, 13, 37, 15,485,863.<p>Maybe we need to add "don't say that numbers ending in 3 are even" to the "AI constitution"?