The short answer
- For getting the words out, pdftomd and PyMuPDF4LLM are level. Both kept 99% of the body text on average across the 15 scored documents. PyMuPDF4LLM kept reading order slightly better on papers and dense tables.
- pdftomd leaves the least clean-up. It removed running headers, footers and page numbers far more consistently, and left no words broken by end-of-line hyphens. It is also the only one of the three that runs without installing anything, and your file stays on your device.
- MarkItDown is built for many formats, and PDFs are not its strength. On PDFs it returns plain text without headings, keeps the PDF's line breaks and hyphens, and lost much of the text of the research papers in our set. In our run it had no OCR for PDFs, so the scans came out empty.
- For scans, both pdftomd and PyMuPDF4LLM read our scanned financial statement almost perfectly, tables included, once PyMuPDF has an OCR engine installed.
If you write Python and the AGPL licence suits your project, PyMuPDF4LLM is an excellent library. If you want the result now, in a browser, without uploading the file, use the converter on our home page.
Results
| Document | pdftomd | PyMuPDF4LLMPyMuPDF 2.1 + RapidOCR 3.10 | MarkItDown0.1.9 |
|---|---|---|---|
| US Supreme Court opinion12 pages | 9,940% | 10,000% | 10,000% |
| NIST AI framework14 pages | 9,980% | 9,970% | 6,240% |
| Research paper, one column15 pages | 9,790% | 9,810% | 2,290% |
| Research paper, two columns12 pages | 9,930% | 9,920% | 5,060% |
| Paper with formulas15 pages | 9,160% | 9,330% | 4,550% |
| Financial statements4 pages | 10,000% | 9,820% | 10,000% |
| IRS tax tables8 pages | 9,980% | 9,810% | 9,840% |
| IRS form W-96 pages | 9,980% | 9,940% | 9,990% |
| Federal Reserve projections table5 pages | 9,970% | 9,950% | 9,050% |
| Scanned financial table2 pages | 9,930% | 9,960% | 0% |
| Chinese text7 pages | 10,000% | 10,000% | 10,000% |
| Japanese report10 pages | 10,000% | 10,000% | 10,000% |
| Korean text7 pages | 10,000% | 10,000% | 10,000% |
| Product brief with images7 pages | 9,990% | 10,000% | 10,000% |
| Lecture slides12 pages | 9,850% | 9,950% | 10,000% |
| Average | 9,900% | 9,897% | 7,801% |
The 1969 typewritten scan isn't in the table: its only reference text is the scanner's own OCR, which is too poor to score against. Both pdftomd and PyMuPDF4LLM produced readable text from it, PyMuPDF4LLM with fewer stray punctuation marks; MarkItDown produced none.
What we tested
| Converter | Version | How we ran it |
|---|---|---|
| pdftomd | this site, 2026-11-13 | Default settings in Chromium, all 16 files dropped at once, Download all |
| PyMuPDF4LLM | PyMuPDF 2.1 with RapidOCR 3.10 | pymupdf4llm.to_markdown(path) |
| MarkItDown | 0.1.9 | MarkItDown().convert(path).markdown |
The documents are public and chosen to cover what people actually convert:
| Document | Pages | What makes it hard |
|---|---|---|
| US Supreme Court opinion | 12 | Footnotes, running headers |
| NIST AI Risk Management Framework | 14 | Cover pages, headings, links |
| Research paper, one column | 15 | Headings, references, a few formulas |
| Research paper, two columns | 12 | Reading order across columns |
| Paper with formulas | 15 | Dense math |
| Apple quarterly financial statements | 4 | Ruled financial tables |
| IRS Publication 15-T excerpt | 8 | Long tax tables, table of contents |
| IRS form W-9 | 6 | Form layout, small print |
| Federal Reserve projections table | 5 | A wide table with multi-level headers |
| Apollo 11 mission report, typewritten scan | 4 | A 1969 scan, no text layer |
| Apple statements, scanned | 2 | A table with no text layer |
| Universal Declaration of Human Rights, Chinese | 7 | CJK text |
| Bank of Japan Outlook Report excerpt | 10 | Japanese, charts |
| Universal Declaration of Human Rights, Korean | 7 | Hangul text |
| Raspberry Pi 5 product brief | 7 | Images, text over pictures |
| Stanford CS231n lecture slides | 12 | Slides, many small images |
How the scores work
Each output is compared with the text that poppler's pdftotext extracts from the same PDF.
- Words found is the share of the reference's words that appear in the Markdown. Running headers, footers and page numbers are taken out of the reference first, so a converter that removes them, as pdftomd does by default, isn't marked down for it. Chinese, Japanese and Korean are counted by character.
- Reading order is the longest common subsequence of words between reference and output, as a share of the reference. It drops when columns are interleaved or table cells are read in the wrong order. Treat differences of a point or two as noise.
- Broken words counts line-end hyphens left unjoined, such as
pur- suant. - Leftover headers counts lines that repeat four or more times in the output, which is almost always a page header or footer that wasn't removed. Image placeholder lines aren't counted.
For tables, we checked the 35 rows of the Apple statement against the PDF: a row counts when its label and every value land in their own cells, in order. pdftomd got 30 rows exactly right, PyMuPDF4LLM 28 and MarkItDown 5. On the scanned copy of the same statement, pdftomd got 29 of 30 rows and PyMuPDF4LLM all 30.
Speed
On the same 4-core machine, pdftomd converted all 16 PDFs in 21 seconds in the browser, including OCR on 9 scanned pages. PyMuPDF4LLM took 64 seconds, also with OCR, and MarkItDown 11 seconds without any. Your times will differ with your hardware; a phone is slower than a laptop.
What this benchmark doesn't tell you
- We make one of the converters. Our clean-up was developed with documents like these in view, which favours us. We publish the scripts and report the rows where we lose, but test on your own PDFs before you decide.
- Automatic scores miss things. A table with a shifted column can keep all its words; a heading at the wrong level costs nothing here. That's why pdftomd flags table pages for you to check.
- Docling and Marker aren't in this round. Both need PyTorch and model downloads, and Marker also runs a separate inference server. Both use layout models and are often recommended for complex layouts and formulas; see PDF to Markdown in Python for how to run them.
- Online converters aren't in the table. Their engines change without notice and can't be pinned to a version, so a score would be out of date the day after we measured it.
Which converter should you use?
| If you want | Use |
|---|---|
| A result now, in the browser, with nothing uploaded | pdftomd |
| A Python library in a pipeline, and the AGPL fits your project | PyMuPDF4LLM |
| One library for Word, PowerPoint, Excel and HTML as well as PDF | MarkItDown, knowing PDFs come out as plain text |
| The best result on formula-heavy papers, with a GPU | Marker or Docling |
Reproduce it
Download the benchmark kit: the browser runner, the Python runner for the libraries, the scorer, the script that produced the table above, and the raw scores. The PDFs come from the public sources named in the table; their exact addresses and page ranges are in the kit's README.
PyMuPDF4LLM is developed by Artifex and MarkItDown by Microsoft. pdftomd isn't affiliated with either.