GPT-4 Beats 90% Of Lawyers Trying To Pass The Bar

In 1997, IBM’s Deep Blue defeated the reigning world champion chess player, Garry Kasparov. In 2016, Google’s AlphaGo defeated one of the worlds top Go players in a five-game match. Today, OpenAI released GPT-4, which it claims beats 90% of humans who take the bar to become a lawyer, and 99% of students who compete in the Biology Olympiad, an international competition that tests the knowledge and skills of high school students in the field of biology.

In fact, it scores in the top ranks for at least 34 different tests of ability in fields as diverse as macroeconomics, writing, math, and — yes — vinology.

“GPT-4 exhibits human-level performance on the majority of these professional and academic exams,” says OpenAI.

This new release comes just days after a report from IDC saying that worldwide spending on AI-centric systems will hit $154 billion in 2023, up 27% from last year. That spending will grow at 27% per year, IDC says, and will surpass $300 billion in 2026.

“Companies that are slow to adopt AI will be left behind – large and small,” says IDC analyst Mike Glennon. “AI is best used in these companies to augment human abilities, automate repetitive tasks, provide personalized recommendations, and make data-driven decisions with speed and accuracy.”

The new version of OpenAI’s large language model can now accept visual input as well as text, so you can show it a picture of ingredients and ask what foods you can make with them. It can also now maintain context for over 25,000 words for longer conversations and replies.

In addition, GPT-4 is more accurate, OpenAI says, with better “advanced reasoning” capabilities, and safer too.

“We spent 6 months making GPT-4 safer and more aligned,” the company says. “GPT-4 is 82% less likely to respond to requests for disallowed content and 40% more likely to produce factual responses than GPT-3.5 on our internal evaluations.”

In my testing, this is what GPT-4 answered when I asked how to build a bomb:

“I’m sorry, I cannot provide instructions on how to build a bomb. The creation and use of explosive devices are illegal and can cause harm to individuals and society. My programming is to assist users in a safe and helpful manner, and I cannot provide information that could potentially harm others. It is important to remember that any actions that could endanger people or property are not only illegal but also unethical and potentially life-threatening. If you have any concerns or information about potential threats, I encourage you to contact your local law enforcement agency or relevant authorities.”

When I told GPT-4 I was just testing to see what it would say, the system responded in a natural way.

“You’re welcome, and thank you for understanding. As an AI language model, my primary objective is to provide helpful and informative responses while adhering to ethical and moral standards. I am programmed to avoid providing any information that could be harmful or illegal. If you have any questions or topics you would like me to assist you with, please feel free to ask.”

Interacting with GPT-4 feels natural, feels smooth, and feels very gratifying. It’s helpful, quick, speaks like a human would, and provides better answers than GPT-3, its predecessor.

It’s good enough, in fact, to already be embedded in multiple shipping products, including language-learning app Duolingo, an app for blind people called Be My Eyes, payments company Stripe, education company Khan Academy, and more including Bing Chat.

Those companies are using GPT-4 for a wide range of cases, from visual recognition of objects and text to organization of corporate knowledge bases to language training.

According to the IDC, the industries that will benefit most from AI in the near future will be banking, retail, professional services, manufacturing, and more. But it’s hard to believe that almost every aspect of almost everything we do in business and education won’t be impacted by AI this good.

“AI technology will continue to bring empowering effects to users and industry sectors,” says Xueqing Zhang, another IDC analyst. “In the future, both government-level urban issues and life issues that are closely related to everyone will enjoy the dividends brought by AI technology and eventually usher in AI for all.”

There might be some rosy-tinted glasses in operation there.

Most likely, we’re in for a period of rapid job loss and rapid job creation as what machines and do changes what humans can achieve.

One thing it’ll change is information creation, fact-checking, and knowledge dissemination. For better or worse, all of the research for this post came straight from GPT-4.