A new chatbot from OpenAI seems to be exceeding expectations.
ChatGPT (Chat Generative Pre-trained Transformer) is a chatbot launched by OpenAI this past November. The New York Times hailed it as “the best artificial intelligence chatbot ever released to the general public.”
It was built by OpenAI, the San Francisco A.I. company that is also responsible for tools like GPT-3 and DALL-E 2, the breakthrough image generator that came out this year.
Like those tools, ChatGPT — which stands for “generative pre-trained transformer” — landed with a splash. In five days, more than a million people signed up to test it, according to Greg Brockman, OpenAI’s president. Hundreds of screenshots of ChatGPT conversations went viral on Twitter, and many of its early fans speak of it in astonished, grandiose terms, as if it were some mix of software and sorcery.
For most of the past decade, A.I. chatbots have been terrible — impressive only if you cherry-pick the bot’s best responses and throw out the rest. In recent years, a few A.I. tools have gotten good at doing narrow and well-defined tasks, like writing marketing copy, but they still tend to flail when taken outside their comfort zones.
…But ChatGPT feels different. Smarter. Weirder. More flexible. It can write jokes (some of which are actually funny), working computer code and college-level essays. It can also guess at medical diagnoses, create text-based Harry Potter games and explain scientific concepts at multiple levels of difficulty.
ChatGPT has now passed the Wharton Business School Test.
The new artificial intelligence system ChatGPT has passed an exam at the Wharton Business School, according to a new research paper, signaling the potential of the controversial chatbot.
Research from Wharton professor Christian Terwiesch found that the AI system “has shown a remarkable ability to automate some of the skills of highly compensated knowledge workers in general and specifically the knowledge workers in the jobs held by MBA graduates including analysts, managers, and consultants.”
On the final exam of Operations Management, a core course in the Wharton MBA program, ChatGPT did “an amazing job” and gave answers that were correct and “excellent” in their explanations.
“ChatGPT3 is remarkably good at modifying its answers in response to human hints. In other words, in the instances where it initially failed to match the problem with the right solution method, Chat GPT3 was able to correct itself after receiving an appropriate hint from a human expert. Considering this performance, Chat GPT3 would have received a B to B- grade on the exam,” the research concluded.
A research team also notes that ChatGPT has passed the US Medical Licensing Examination.
We evaluated the performance of a large language model called ChatGPT on the United States Medical Licensing Exam (USMLE), which consists of three exams: Step 1, Step 2CK, and Step 3. ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement.
Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations. These results suggest that large language models may have the potential to assist with medical education, and potentially, clinical decision-making.
These developments have significant potential ramifications, especially in education.
“Goodbye, homework,” tweeted Elon Musk after the launch of ChatGPT, a bot that writes plausible answers and even rhyming poetry. This kind of generative artificial intelligence sparks fear, loathing and awe in equal measure. But it is the world of education which is most spooked.
Since OpenAI launched the ChatGPT language-generation model before Christmas, New York’s public schools have banned pupils from using it. In Australia, universities are planning a return to supervised pen and paper examinations to evade the chatbot fakes. Teachers are rightly concerned that they won’t be able to help pupils who are falling behind if they can’t spot faked assignments. But one reason these bots pose such a threat is that so much of our education remains fixated on being able to elegantly regurgitate information.
The news about the chatbot passing the medical licensing exam especially highlights the need to reassess how medical education is approached.
One of the biggest ongoing complaints in medicine is that the step exams don’t correlate with actual clinical skills. A lot of the tests are rote memorization of standardized scenarios. Step 1 has a lot of obscure facts which is why students study for so long.
— Roxana Daneshjou MD/PhD (@RoxanaDaneshjou) January 23, 2023
However, there could be real opportunities for effective use in medicine.
And this is just the beginning. Short term a reliable AI could act as a co-pilot for general MDs in areas that they are not well versed.
e.g. how a patient’s nutrition, sleep and movement affect a specific problem they have instead of quickly resorting to prescribing drugs.
— Leo Rezaei (@theLionary) January 22, 2023
Of course, there are still some things that a doctor can’t do.
But does it have illegible handwriting? That’s the true prerequisite for a physician
— dave stanton (@WhosDaveStanton) January 22, 2023
Donations tax deductible
to the full extent allowed by law.