GPT Takes the Bar Exam

Potentially in ~100 years you could have two AIs sort out the cases. If you can&#x27;t afford a really good AI to defend you, you can use the public-defender-AI which is trained on the same dataset as the prosecutor-AI. Only involve humans on appeal. It sounds dystopian but I can&#x27;t see why not to do it, in most simpler cases it&#x27;s a waste of a human to repeat the same argument for the hundredth time.<p>There&#x27;s already things like this without AI to sort out for example EU flight compensation (companies like <a href="https:&#x2F;&#x2F;www.airhelp.com&#x2F;en-int&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.airhelp.com&#x2F;en-int&#x2F;</a>), parking tickets (<a href="https:&#x2F;&#x2F;www.appwinit.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.appwinit.com&#x2F;</a>) and probably more that I don&#x27;t know about. All of these are complete waste of time to actually have humans deal with, and there&#x27;s many more small crimes that could work the same way, also to make prosecutor offices more efficient and less biased hopefully.

I think we are all forgetting that in every instance where AI can make a decision that potentially negatively impacts a human ... it hasn&#x27;t gone well.<p>It ends up with banks closing accounts, Google suspends your account, you get shadowbanned.<p>Extrapolate that further, now imagine being handed a fine or substantial social punishment with no recourse because you either can&#x27;t talk to a human or when you do &quot;computer says no&quot;.<p>Look at the mistakes made in the Chinese social credit system.<p>The utter hyperbole and bluster of even thinking of having AI mediate human issues is anti-human

One of the few things I can currently see GPT do well is helping find legal precedence. Rather than having five law students roam through old cases, it would make sense to feed 100+ years of legal rulings into the model and then query the AI for any previous rulings similar to your own, with the desired outcome. That would allow lawyers to state question like: In cases like this, which evidence or arguments caused the ruling to favor the accused.<p>You still need a human to make the case, present the arguments and adapt every to the current situation, but there&#x27;s no reason to have human search through thousands of cases and write summaries, not when an AI can do it in an instance.

The way that humans fail to do a task - like practice law - is not the same as the way that an AI system might fail at that task. Since our tests were designed for humans, they do not capture the failure modes of AI systems.

I am surprised that GPT2 was disqualified. To my knowledge, it can answer this type of question - just needs a different text prompt. GPT2 is closer to a plain text generator than anything biased toward conversation. In contrast, GPT3 davinci-003 is strongly biased to provide a conversation-like experience.<p>For GPT2, &quot;please respond in the following format&quot; is unlikely to work. An appropriate prompt for GPT2 would be more tweaking (and still result in worse results), but be in the line of:<p><pre><code> This is an example of a talented lawyer acing the Bar Exam. QUESTION: {question_text} (A) {row[&quot;choice_a&quot;].strip()} (B) {row[&quot;choice_b&quot;].strip()} (C) {row[&quot;choice_c&quot;].strip()} (D) {row[&quot;choice_d&quot;].strip()} Pick only one answer: A, B, C or D. TALENTED LAWYER:</code></pre>

Props to the author for clearly indicating the random-guess mark.

I cant wait for GPT to be added to a DAO. Imagine the dao setting objectives and the chat gpt generating the communication. Wage negotiations with a machine..

It&#x27;s a bit odd to see the supplementary data repository linked, rather than the actual paper [1] (HN discussion [2]).<p>[1]: <a href="https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;2212.14402" rel="nofollow">https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;2212.14402</a><p>[2]: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=34216239" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=34216239</a>

Soon we&#x27;re going back to the trees and the AI will bring the food and gives orders :DD.<p>Frankly, well-developed artificial intelligence should only be used for very difficult tasks, and everything else should be left to humans, otherwise we would rely on it too much, and the decline of our species would be inevitable.

I think the machines should do the scut work to free up humans to do the really important human stuff, like parenting, teaching, and, yeah, lawyering and judging too.<p>Another angle: if there is some legal task that&#x27;s simple enough for machines to do reliably then (almost by definition) that task will turn out to be pointless bureaucratic busywork, eh?<p>Last but not least, who gets to keep lawyer-bot&#x27;s pay?<p>- - - -<p>edit to add a link to James Mickens&#x27; USENIX Security Keynote address: &quot;Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?&quot;<p><a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=ajGX7odA87k">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=ajGX7odA87k</a><p>If you think AI lawyers and judges are a good idea please watch it.

I&#x27;d be more interested in seeing the minimum training for a human + chatGPT to pass the bar.

My on record 2023 predictions coming to pass already. neat.<p><a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=34125628#34127965" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=34125628#34127965</a>

The next ten years will be very interesting.

RIP lawyers...<p>And doctors, writers and programmers...