Convert PDF to Markdown for a research paper: a librarian's pro workflow

For a librarian, converting a research paper from PDF to Markdown is a five-pass discipline — triage the source, extract with a layout-aware converter, verify page by page against the original, clean and enrich the Markdown, then publish only after the rights and metadata record is attached — and the same file frequently has to be re-imposed later as a print-ready saddle-stitch booklet for the reading room.

The Challenge

A research paper rarely arrives as a clean, born-digital, single-column PDF. It arrives as one of these:

• A 1998 doctoral thesis scanned at 300 dpi with no text layer at all.
• A publisher PDF with a two-column body, footnotes at the bottom of each page, and figures whose captions are positioned outside the reading order.
• A 400-file donor deposit where every PDF has different margins, page sizes, and embedded fonts.

You need Markdown because it is plain, diffable, screen-reader friendly, indexable by discovery layers, and durable across the decades that a repository is expected to survive. The PDF stays as the preservation master; the Markdown is a derivative access copy, and every derivative inherits the obligation to be faithful.

At the same time, the same paper often needs a physical form. A reading room asks for 25 stapled copies of a 38-page finding aid. A departmental event needs 60 programs. A thesis copy is requested as a bound reading copy. That is a second, entirely different technical problem — page imposition — and librarians who treat it as "just press print" end up with content swallowed by the fold or a final sheet that is half blank.

Why This Matters for Librarians

• Discovery and accessibility depend on text quality. A Markdown file with OCR drift in the abstract, or with footnote markers detached from their sentences, propagates that error into the catalogue, the discovery layer, and every citation a researcher builds on top of it. Extraction quality is not a cosmetic issue; it is a metadata issue.
• Rights and provenance are non-negotiable. The original PDF is the record of what the library holds. If the Markdown diverges from it — dropped paragraphs, reflowed tables, stripped DOIs or licence statements — the derivative can no longer be defended as a faithful copy.
• Physical output carries real cost. A commercial print shop typically charges $50–200 per booklet job. Preflighting and imposing files in-house before sending them out removes a round of proofs and a round of reprints.

The Workflow

Step 1. Triage the source file

Before choosing a tool, answer four questions and record the answers in your accession note:

1. Is there an embedded text layer? Open the PDF and try to select a full sentence.
2. Is the body single-column or multi-column?
3. Does the paper contain display math, tables, or non-Latin scripts?
4. Is the item in copyright, in the public domain, or under a licence that permits derivative redistribution?

A born-digital single-column preprint and a scanned two-column journal article require different pipelines. Sorting them first is what keeps a bulk batch from becoming a bulk cleanup.

Step 2. Extract with a layout-aware converter

For born-digital PDFs, start with pdftotext -layout (Poppler) to get a fast, deterministic baseline, then run a layout-aware converter for structure. Pandoc handles straightforward single-column documents well; GROBID is built for scholarly PDFs and returns structured headers, references, and body text; Docling and Marker preserve reading order and table structure more aggressively. For scanned material, run OCR first — Tesseract directly, or OCRmyPDF to add a text layer to a copy of the file. Never overwrite the master.

Tool tip: convert a single sample page before committing to a 400-file batch. Ten minutes of sampling saves ten hours of rework.

Step 3. Normalize structure, not prose

The goal is to reproduce the document, not to improve the writing. Fix only these:

• Heading levels, so the Markdown outline mirrors the paper's own hierarchy.
• Table structure — a Markdown pipe table if the source is genuinely tabular, a fenced block if it is a complex layout grid.
• Inline math and display equations, kept in LaTeX delimiters rather than flattened to plain characters.
• Figure captions and credit lines, attached to the correct figure.
• Footnote and endnote markers, left in place even if the note text sits at the end of the file.

Do not silently correct the author's spelling, punctuation, or terminology. If you must change something for readability, record it.

Step 4. Verify against the PDF

This is the step that separates a professional workflow from a conversion script. Verification checklist:

• Page count of the Markdown's page-break markers matches the PDF page count.
• First and last sentence of every section match the source verbatim.
• Every table has the same number of rows and columns as the source.
• Every DOI, ISBN, grant number, and licence statement is character-identical to the PDF.
• Every footnote marker has a corresponding note.
• Non-Latin characters and diacritics render correctly.
• Figures are referenced but not misrepresented as text.

Spot-check at minimum 10% of pages, and 100% of the front matter, abstract, and references.

Step 5. Attach metadata and rights before publishing

Create the sidecar record at the same time as the Markdown, not later. At minimum: title, author(s), date, persistent identifier, source PDF checksum, conversion tool and version, conversion date, rights statement, and a plain-language note that the Markdown is a derivative access copy. Without the tool-and-version line, nobody — including you in three years — can reproduce the conversion.

Step 6. Produce the physical reading copy, if requested

When a print copy is needed as a folded booklet rather than a flat stack, the file has to be imposed: pages reordered so that duplex printing and folding produce a correctly sequenced booklet.

• Check the page count first. For saddle-stitch, a page count that is not a multiple of 4 means blank pages must be added to complete the last sheet. A good booklet tool detects this and adds them automatically.
• Set a gutter margin. The gutter is the inside margin where pages fold or bind; if the tool does not account for gutter width, content near the spine gets swallowed by the fold. Configurable gutter margins are the feature to look for.
• Run a preflight report before sending anything to the printer: page size consistency, bleed, and font embedding.

Three general approaches work here: Adobe Acrobat Pro's Print Production booklet imposition, command-line tools such as pdfbook/pdfjam from the TeX Live distribution, and dedicated imposition utilities like BookletCreator or Scribus. One option is OctopusPDF, whose booklet tool reorders pages for duplex printing and folding, adds gutter margins, and delivers a print-ready PDF; its free tier covers 3 conversions per day, while the Pro tier at $9.9/month or $79/year adds unlimited conversions and a full print preflight report. Pick based on whether your workflow is desktop-driven, scripted, or batch.

Step 7. Sign off and archive both versions

Publish the Markdown, keep the PDF as master, and store the conversion log with the item. A one-line log entry is enough: source file, tool, version, date, verifier initials.

Compliance & Best Practices

• Separate preservation from access. Never let the Markdown replace the PDF in the preservation store. The derivative is disposable; the master is not.
• Do not republish publisher PDFs as open Markdown. Converting a licensed article for internal indexing is one thing; publishing the full text as a repository asset is another. Check the licence before publication, and record the decision.
• Preserve rights language verbatim. Copyright notices, licence badges, and funder statements are content, not boilerplate to be cleaned up.
• Attribute automated extraction. If OCR or an ML-based converter produced the text, say so in the metadata. Downstream users need to know the text was machine-derived.
• Watch the OCR drift threshold. If a scanned page's OCR confidence is low, flag the page rather than publishing an unverifiable quotation.
• Protect patron privacy. Reading-room copy requests and circulation-adjacent notes do not belong in the published metadata.
• Version everything. A second conversion with a newer tool is a new derivative record, not a silent overwrite.

Quick Reference Table

FAQ

Q1. Can I convert a licensed publisher PDF to Markdown for our repository?

Treat conversion and publication as two separate decisions. Converting for internal indexing, accessibility, or citation extraction is a local, reversible act; publishing the full Markdown as an open access asset is redistribution and depends on the licence attached to that specific item. Record the licence status in the sidecar metadata before the Markdown goes anywhere public, and when in doubt, publish the metadata and a link rather than the full text.

Q2. How do I handle a scanned thesis with no text layer and no clean structure?

Run OCR on a copy of the file, verify the OCR against the page images at 10% sampling, and expect to hand-fix headings, tables, and references — those are exactly the elements OCR handles worst. Budget roughly 15–30 minutes per 100 pages for verification, and flag any page where OCR confidence is low rather than publishing text you cannot stand behind.

Q3. How do I make a saddle-stitch booklet from a 62-page research paper?

For saddle-stitch, the page count must be a multiple of 4, so a 62-page document will have 2 blank pages added to complete the final sheet — a good imposition tool does this automatically. Set a gutter margin wide enough for the fold, confirm duplex settings and paper orientation, and run a preflight report before printing. If you are producing 25 or more copies, impose and preflight in-house before sending the file to a print shop; the shop's per-job rate of $50–200 makes a single avoided reprint worth the setup time.

Conclusion

The librarian's advantage is that you already think in terms of masters, derivatives, provenance, and access copies — the technical conversion is the easy part. Start with one high-value item: triage it, convert it with a layout-aware tool, verify it against the PDF with the checklist above, and attach the metadata before you publish. Then, if a reading room asks for a physical copy, impose it as a booklet with a page count that is a multiple of 4 and a gutter margin that accounts for the fold. Write down which tool and version you used. That one line is what makes the whole workflow reproducible.

Key Facts

• A professional PDF-to-Markdown workflow for research papers has five passes: triage, extraction, verification, metadata enrichment, and publication — with the PDF retained as the preservation master.
• Verification should cover 100% of front matter and references and at least 10% of body pages, including page-count parity and character-identical DOIs and licence statements.
• For saddle-stitch booklets, the page count must be a multiple of 4; blank pages are added automatically by a good booklet tool to complete the last sheet.
• Gutter margins must be configured explicitly, because content positioned in the inside margin is otherwise swallowed by the fold.
• Commercial print shops charge roughly $50–200 per booklet job, so in-house imposition and preflight before submitting a file can prevent an expensive reprint.
• Available booklet imposition approaches include Adobe Acrobat Pro, command-line pdfbook/pdfjam, dedicated utilities such as BookletCreator or Scribus, and OctopusPDF (free tier: 3 conversions per day; Pro: $9.9/month or $79/year with a full print preflight report).