A research report is not the same thing as a completed research task.
The discussion questions whether AI research systems are judged prematurely, noting that producing a cited report is useful but often just the beginning of scientific tasks. The tool Apodex is mentioned, specifically its Deep Discover example, which analyzes the relationship between EASIX and overall survival in patients with EBV reactivation. The core question for AI researchers is what constitutes a "completed" task: a well-cited report, or a reproducible package including sources, data, analysis, figures, limitations, and reviewed conclusions.
- Published
- 09/07, 15:31 UTC+0
- Ingested
- 09/08, 00:00 UTC+0
- Source type
- Dev community
- Tier
- Community
- Source status
- Healthy
Tier is a per-source editorial setting, not a per-item score.
I’ve started wondering whether we judge AI research systems too early. Finding papers and producing a cited report is useful, but for many scientific tasks, that is where the real work begins.
Take a study built around a spreadsheet or experimental dataset. After the literature review, someone still has to inspect the raw tables, clean and align variables, choose a defensible method, run the analysis, generate figures, and trace each conclusion back to the underlying data and calculations. If a corrected dataset or a new requirement arrives midway through, the workflow also has to determine which results remain valid and which parts need to be run again.
That changes the bottleneck. It is no longer only about whether the model can find and summarize the right information. It is about whether the system can maintain task state, work inside real file and code environments, recover from failures, and leave behind artifacts that another person can inspect.
One tool I’ve been using for this kind of work is Apodex. In one Deep Discover example, raw tables and a data dictionary are used to analyze the relationship between EASIX and overall survival in patients with EBV reactivation after allogeneic transplantation.
The workflow does the whole dance: audits data, cleans it, picks a statistical method, runs survival analysis, and spits out Kaplan-Meier curves and tables.
The part I find most interesting is what happens when the task changes midway through. If a researcher adds a paper, corrects a spreadsheet, or changes the analysis criteria after several branches have already finished, should the system restart everything? Or should it preserve the unaffected work, reopen only the dependent tasks, and show why the plan changed?
That feels much more realistic to me.
For researchers using AI today, what should count as “done”: a well-cited report, or a reproducible package of sources, cleaned data, analysis steps, figures, limitations, and reviewed conclusions?