OpenAI's New ‘Deep Research’ AI: A Promising Tool with Limitations

OpenAI has introduced a new feature called “deep research” within its ChatGPT platform, aiming to revolutionize how we conduct research. This artificial intelligence (AI) tool claims to perform in mere minutes what would typically take a human researcher hours to accomplish.

Marketed as a research assistant that could compete with trained analysts, deep research can autonomously browse the internet, compile sources, and generate structured reports. It even achieved a score of 26.6% on Humanity’s Last Exam (HLE), a challenging benchmark for AI performance, surpassing many existing models.

However, the reality of deep research doesn't quite align with the hype. Although it produces well-formatted reports, there are notable shortcomings. Journalists and users have reported that the tool sometimes overlooks essential details, struggles with the latest information, and even fabricates facts.

OpenAI has acknowledged these limitations, stating that the AI “can sometimes hallucinate facts in responses or make incorrect inferences, although at a notably lower rate than existing ChatGPT models, based on internal assessments.”

It's important to remember that AI models, unlike humans, don’t possess true understanding. The idea of an AI acting as a “research analyst” raises significant questions: Can a machine truly replace a trained expert? What would this mean for professional knowledge roles? And is AI genuinely enhancing our thinking abilities or merely encouraging us to stop thinking critically?

Understanding ‘Deep Research’ and Its Target Audience

Deep research is aimed at professionals across various fields, including finance, science, policy, law, and engineering, as well as academics, journalists, and business strategists. It represents a recent “agentic experience” launched by OpenAI in ChatGPT, claiming to handle comprehensive research tasks swiftly.

Currently, this feature is available only to ChatGPT Pro users in the United States at a subscription cost of $200 per month. OpenAI plans to extend availability to Plus, Team, and Enterprise users soon, with a more budget-friendly version expected later.

In contrast to a typical chatbot that offers quick responses, deep research operates through a multi-step process to create detailed reports:

1. The user submits a research request, such as a market analysis or a legal summary.
2. The AI clarifies the task, potentially asking follow-up questions to refine the research parameters.
3. The agent conducts a web search, exploring numerous sources, including news articles, academic papers, and databases.
4. It synthesizes its findings, extracting key points, organizing them into a structured report, and citing sources.
5. The final report is presented to the user within five to thirty minutes, spanning multiple pages and possibly resembling a PhD-level thesis.

While this may sound like an invaluable tool for knowledge workers, potential drawbacks become apparent upon closer examination.

Early tests have revealed significant limitations:

• Lack of context. AI can summarize data but often misses the crux of what's significant.
• Ignoring new developments. It fails to account for recent legal rulings and scientific advancements.
• Generating misinformation. Like other AI models, it may confidently fabricate details.
• Difficulty distinguishing fact from fiction. It struggles to discern credible sources from unreliable ones.

While OpenAI asserts that its tool competes with human analysts, AI inevitably lacks the judgment, depth of understanding, and critical perspective that are crucial for high-quality research.

What AI Cannot Replace

OpenAI's ChatGPT is not alone in its ability to search online and produce reports; shortly after its launch, alternative solutions, such as Hugging Face's free open-source version, emerged, closely mirroring its capabilities.

A major concern about deep research and similar AI solutions is the misleading impression that they can substitute for human cognitive processes. While AI can condense information, it cannot challenge its assumptions, identify knowledge gaps, engage in creative thought, or appreciate varied viewpoints.

The depth of analysis provided by AI-generated summaries falls short of what an accomplished human researcher can deliver.

Nonetheless, AI remains a tool, and it shouldn't be seen as a replacement for human intelligence. As knowledge workers navigate these advancements, it's essential to hone skills that AI cannot imitate, including critical thinking, fact-checking, specialized knowledge, and creativity.

If you choose to incorporate AI in your research endeavors, doing so thoughtfully can enhance effectiveness without compromising accuracy or thoroughness. You might leverage AI for tasks that improve efficiency, such as summarizing lengthy documents, while preserving human judgment in decision-making processes.

Always verify sources since AI-generated citations can sometimes be unreliable. Avoid accepting conclusions without question; apply critical thinking and cross-reference information with trustworthy sources. In critical areas—like health or justice—it's vital to complement AI findings with expert insights.

Despite aggressive marketing suggesting otherwise, generative AI is still bound by many limitations. Those who can creatively synthesize details, challenge assumptions, and engage in critical analysis will continue to be indispensable, as AI cannot supplant them just yet.

AI, Research, Analysis