paper2code: Best AI Coding Agents for ML Engineers Reproducing Papers in 2026

What Is paper2code?

paper2code is an open-source AI coding agent skill that converts arXiv paper URLs into fully structured Python code repositories with citation references to specific sections and equations. Developed by PrathamLearnsToCode, it targets ML engineers facing vague paper descriptions by generating verifiable implementations including model.py (§3.2), loss.py (§3.4), and train.py (§4.1). paper2code is one of the best AI Coding Agents tools for ML engineers reproducing research papers, boasting 559 GitHub stars as of October 2024 and MIT licensing for unrestricted use.

It outputs a complete directory with README.md, REPRODUCTION_NOTES.md, requirements.txt, src/ folder, configs/base.yaml, and notebooks/walkthrough.ipynb, ensuring every code line traces back to the paper or flags uncertainties.

Quick Overview

Who Should Use paper2code?

• ML reproducibility teams validating 10+ papers quarterly who need citation-traced code to audit deviations from original results.
• Academic researchers implementing baselines for new models, requiring [UNSPECIFIED] flags on hyperparameters like learning rates omitted in main text.
• Indie ML hackers prototyping from arXiv without hunting appendices, saving 4-6 hours per paper on detective work.
• Conference reviewers quickly generating runnable code from submissions to test claims in under 10 minutes.

Not ideal for:

• Production deployment engineers needing optimized, battle-tested inference code rather than research prototypes.
• Non-ML developers lacking Python familiarity, as outputs assume proficiency in PyTorch or JAX.
• Teams with proprietary papers, since paper2code relies on public arXiv parsing.

Key Features of paper2code

• Citation anchoring — each generated file like src/model.py includes inline comments linking to paper sections (e.g., # §3.2 cited), enabling direct verification against Eq. 4 or Fig. 2.
• Ambiguity auditing — pre-generation scan classifies every parameter as SPECIFIED (from text/eq), PARTIALLY_SPECIFIED (appendix hints), or UNSPECIFIED, logging choices in REPRODUCTION_NOTES.md.
• Appendix mining — parses footnotes, captions, and supplements as primary sources, extracting buried details like batch size=256 from Table A1.
• Framework specification — supports PyTorch default or --framework jax flag, adapting tensor ops and autodiff accordingly without hallucinated conversions.
• Full pipeline output — generates data.py, evaluate.py, utils.py, base.yaml with all params, and walkthrough.ipynb for end-to-end reproduction.
• Honest uncertainty handling — inserts [UNSPECIFIED] comments with alternatives (e.g., "optimizer: Adam (common) vs SGD (rare)" ) at decision points.
• Symlink installation — npx skills add integrates as global skill for agents like Claude Code, preserving updates via Git.

paper2code vs Alternatives

Hugging Face Replicate suits quick demos of published models but lacks custom paper-to-code generation, forcing manual adaptation. Papers with Code aggregates community repos yet often misses unpublished tweaks, requiring paper2code for fresh arXiv drops. Claude Code Canvas excels in iterative sketching but skips automated parsing—pair it with paper2code outputs for refinement.

How paper2code Works

paper2code pipelines arXiv URLs through LLM-driven parsing: fetch PDF/text, extract sections/equations/appendices, audit ambiguities, then generate anchored code. Core abstraction is a citation graph mapping paper spans (§3.2 Eq.4) to code lines, enforced during prompting to prevent hallucinations. Node.js CLI orchestrates agent skills, outputting a {paper_slug}/ directory with Python files traceable via comments.

Ambiguity classification uses regex + LLM: SPECIFIED (direct quote), PARTIALLY_SPECIFIED (inferred range), UNSPECIFIED (flagged). Configs/base.yaml dumps all params with sources, e.g., lr: 1e-4 # §4.1, Table 1. This design prioritizes verifiability over completeness, averaging 85% SPECIFIED coverage on NeurIPS papers.

Run in Claude Code:

/paper2code https://arxiv.org/abs/1706.03762

This fetches the paper, audits (e.g., optimizer UNSPECIFIED), generates src/, and opens walkthrough.ipynb showing train.py execution with reproduced metrics (e.g., 92% acc on CIFAR-10). Expect 2-5 minutes runtime, with REPRODUCTION_NOTES.md summarizing gaps like "dataset split: 80/20 assumed".

For JAX:

/paper2code https://arxiv.org/abs/2006.11239 --framework jax

Adapts to jax.numpy, generating haiku-style models if specified.

Pros and Cons of paper2code

Pros:

• Verifiable outputs cut debugging time by 70%, as every line cites §X.Y or flags issues.
• Handles appendices/figures ignored by naive LLMs, boosting SPECIFIED param coverage to 85%.
• Modular src/ structure mirrors paper (§3 model, §4 training), easing extensions.
• Zero-config install via npx skills, works offline post-symlink.
• Framework flexibility (PyTorch/JAX) matches 90% arXiv codebases.
• Notebook integration visualizes train/eval loops with paper metrics.

Cons:

• Limited to arXiv; proprietary PDFs require manual upload hacks.
• Early stage (5 commits), may hallucinate on math-heavy proofs.
• Python-only outputs; no JS/Rust generation yet.
• Agent-dependent: shines in Claude Code, weaker in basic GPT CLI.
• Full mode omits custom datasets, assuming paper-standard loaders.

Getting Started with paper2code

Install globally for your coding agent:

npx skills add PrathamLearnsToCode/paper2code/skills/paper2code

Prompt selects Claude Code (recommended), global scope, symlink method. This links ~/skills/paper2code for updates.

In agent terminal:

/paper2code https://arxiv.org/abs/1706.03762 --framework pytorch

Generates ./1706.03762/ with src/, configs/, notebooks/. cd in, pip install -r requirements.txt, then python src/train.py --config configs/base.yaml. First run trains model in 10-20 epochs (GPU recommended), logging losses matching §4.1 curves. Edit base.yaml for sweeps, rerun evaluate.py for paper metrics.

Verdict

paper2code is the strongest option for ML engineers reproducing arXiv papers when verifiability trumps speed. Its citation anchoring and ambiguity audits deliver trustworthy codebases unmatched by naive generation. Use it for baselines, but supplement with manual tweaks for edge cases—install today for your next paper impl.