Skip to content
SPECIFICTION
Get listed

RobotReviewer

robotreviewer.net

Free plan Open source WebWindowsmacOSLinuxAPISelf-hosted
Visit robotreviewer.net
Spec score 8.4 #6 of 38 in AI Literature Review Software

What it is

RobotReviewer analyzes clinical trial reports by annotating PDFs with PICO characteristics and likely bias assessments using the Cochrane Risk of Bias tool. Uploading PDFs on the website extracts key data and presents a summary table. Its REST API can process groups of clinical trial reports to synthesize results across trials. The project is open source under GNU GPL version 3.0 and is free to use. It can run locally as Docker services, with a web app and API server exposed on local ports. The self-hosted processing setup uses Grobid, RabbitMQ, Celery, and SciBERT through bert-as-a-service. Docker instructions cover GPU use, and the worker can also run CPU-only. Manual setup calls for Python 3.6, TensorFlow 1.12.0, and the associated processing services. Installation is documented as tested on Ubuntu and Mac OS; Windows works with substantial installation difficulty. Maintainers ask users to cite the 2017 ACL paper and invite questions by email, including requests for help integrating RobotReviewer into other software.

Spec sheet

What it does
RobotReviewer analyzes clinical-trial PDFs and automatically extracts trial information and risk-of-bias assessments.Read 15 Oct 2026 · github.com
Purpose
RobotReviewer automatically annotates clinical trial PDFs with PICO characteristics and likely bias assessments using the Cochrane Risk of Bias tool.Read 7 Oct 2026 · github.com
Deployment
The project can run locally as Docker services, with a web app and API server exposed on local ports.Read 7 Oct 2026 · github.com
API
A REST API exposes the underlying models for integration into systematic-review software.Read 15 Oct 2026 · pmc.ncbi.nlm.nih.gov
Integrations
The self-hosted setup uses Grobid for document processing, RabbitMQ and Celery for worker processing, and SciBERT through bert-as-a-service.Read 7 Oct 2026 · github.com
Support
The project directs users to an email address for questions and integration assistance.Read 15 Oct 2026 · github.com
License
RobotReviewer is open source under the GNU GPL version 3.0 license.Read 15 Oct 2026 · github.com
API extensibility
The API endpoint implementation is available in the repository's robotreviewer/app.py file.Read 15 Oct 2026 · github.com
Batch synthesis
The newer API works with groups of trial reports so results can be synthesized across multiple studies.Read 15 Oct 2026 · github.com
Citation
The maintainers ask users to cite the 2017 ACL paper when using RobotReviewer.Read 7 Oct 2026 · github.com
Citation year
The cited RobotReviewer evaluation was published in the Journal of the American Medical Informatics Association in 2015.Read 15 Oct 2026 · github.com
Dependencies
Installation requires Grobid, Java, RabbitMQ, Celery, spaCy, NLTK, TensorFlow, and SciBERT model files.Read 15 Oct 2026 · github.com
Docker deployment
Docker Compose files are provided for building and running the web, API, and supporting services.Read 15 Oct 2026 · github.com
Documentation depth
The README includes Docker, manual installation, configuration, startup, API, testing, and troubleshooting instructions.Read 15 Oct 2026 · github.com
Funding
The project acknowledges NIH National Library of Medicine and UK Medical Research Council grant support.Read 15 Oct 2026 · github.com
GPU option
The project provides Docker instructions for GPU use and also documents running the worker CPU-only.Read 7 Oct 2026 · github.com
GPU support
A separate Docker Compose configuration supports NVIDIA GPU execution.Read 15 Oct 2026 · github.com
Integration
The research paper identifies Covidence and similar evidence-synthesis platforms as potential integration targets.Read 15 Oct 2026 · pmc.ncbi.nlm.nih.gov
Limit: human oversight
Researchers in the real-time study still considered human input and discussion superior to semi-automated assessment.Read 15 Oct 2026 · pubmed.ncbi.nlm.nih.gov
Limit: incomplete automation
A 2026 scoping review describes RobotReviewer as time-saving but still requiring human supervision and reports uncertain reliability in some studies.Read 15 Oct 2026 · link.springer.com
Limit: lower accuracy than humans
The original evaluation reported overall risk-of-bias classification accuracy five to ten percentage points below published human systematic reviews.Read 15 Oct 2026 · pmc.ncbi.nlm.nih.gov
Limit: scope
The 2026 review reports that RobotReviewer is primarily evaluated on randomized trials and has limited evidence for other study designs.Read 15 Oct 2026 · link.springer.com
Multiple reports
The API supports groups of clinical trial reports to synthesize results across trials.Read 7 Oct 2026 · github.com
Operating systems
The documentation reports successful installation testing on Ubuntu and macOS, with Windows possible but difficult.Read 15 Oct 2026 · github.com
PDF input
The public service accepts clinical-trial documents in PDF form for automatic annotation.Read 15 Oct 2026 · github.com
PDF summaries
Users can upload PDFs through the website to extract key data and view a summary table.Read 7 Oct 2026 · github.com
Performance range
The 2026 scoping review reports published RobotReviewer accuracy estimates ranging from 71% to 78%.Read 15 Oct 2026 · link.springer.com
PICO extraction
The system extracts Population, Intervention or Comparator, and Outcome information from trial reports.Read 15 Oct 2026 · github.com
Praise: extraction quality
Expert reviewers scored automatically extracted rationale sentences comparably to human-extracted sentences in the published evaluation.Read 15 Oct 2026 · pmc.ncbi.nlm.nih.gov
Praise: human agreement
A real-time study found reviewers were about as likely to accept RobotReviewer's judgments as another human reviewer's judgments.Read 15 Oct 2026 · pubmed.ncbi.nlm.nih.gov
Praise: practical usefulness
The real-time study reported that less-experienced reviewers viewed the tool more positively and used it more flexibly.Read 15 Oct 2026 · pubmed.ncbi.nlm.nih.gov
Project authors
The repository credits Iain Marshall, Joël Kuiper, and Byron Wallace as the authors of the 2015 RobotReviewer work.Read 15 Oct 2026 · github.com
Python requirement
The documented installation requires Python 3.6 for the main repository version.Read 15 Oct 2026 · github.com
Rationale extraction
The models identify sentences that support their risk-of-bias classifications.Read 15 Oct 2026 · pmc.ncbi.nlm.nih.gov
RCT classification
An ensemble of convolutional neural networks, support-vector machines, and PubMed metadata classifies randomized controlled trials.Read 15 Oct 2026 · pmc.ncbi.nlm.nih.gov
Risk-of-bias assessment
It applies machine-learning models based on the Cochrane Risk of Bias framework.Read 15 Oct 2026 · pmc.ncbi.nlm.nih.gov
Self-hosting
Users can run the web interface and API locally from the source repository.Read 15 Oct 2026 · github.com
Setup requirements
The documented manual setup calls for Python 3.6, TensorFlow 1.12.0, Grobid, RabbitMQ, Celery, and SciBERT services.Read 7 Oct 2026 · github.com
Study similarity
RobotReviewer creates two-dimensional visualizations of study similarities using learned abstract representations.Read 15 Oct 2026 · pmc.ncbi.nlm.nih.gov
Swagger API
The repository documents starting a Swagger-enabled API server with the REST_API environment flag.Read 15 Oct 2026 · github.com

Plans and prices

Free Free Free of charge · open source under GPL-3.0 Read 7 Oct 2026 · github.com
Free Free Free of charge · citation requested when used · attribution link required for integrations Read 15 Oct 2026 · pypi.org

Every RobotReviewer plan and price →

Strong points and limits

Strong points

  • Free to use under GNU GPL version 3.0.
  • Website uploads produce key-data summaries.
  • API can synthesize groups of trial reports.
  • Docker instructions cover GPU and CPU-only workers.

Limits

  • Manual setup requires Python 3.6 and TensorFlow 1.12.0.
  • Windows installation involves substantial difficulty.
  • Self-hosting requires several processing services.

Who it suits

RobotReviewer suits users analyzing clinical trial reports who need PICO extraction, likely-bias assessments, or synthesis across reports. Its local deployment and API may suit teams incorporating these functions into other software.

Verdict

RobotReviewer provides free clinical-trial PDF analysis through its website and API, with local Docker deployment also available. Manual installation has specific dependencies, and Windows setup is notably difficult.

Where Specifiction places it

Alternatives to RobotReviewerAll alternatives →