Skip to content

Update Pandoc, Typst, Dart Sass, esbuild, and veraPDF#14664

Draft
cderv wants to merge 10 commits into
mainfrom
chore/pandoc-3.10
Draft

Update Pandoc, Typst, Dart Sass, esbuild, and veraPDF#14664
cderv wants to merge 10 commits into
mainfrom
chore/pandoc-3.10

Conversation

@cderv

@cderv cderv commented Jul 9, 2026

Copy link
Copy Markdown
Member

Version bump only in configuration and src/command/check/check.ts, so the new binaries can be archived to S3 and downloaded during configure. Bumps Pandoc to 3.10, Typst to 0.15.0, Dart Sass to 1.101.0, esbuild to 0.28.1, and veraPDF to 1.30.2, all riding along in one branch since Carlos is archiving from it directly.

Note

Draft: only Pandoc and Typst have had any risk analysis so far (template resync, Lua API compatibility, test coverage). Dart Sass, esbuild, and veraPDF are bumped here but not yet verified. None of this is ready to merge until that work lands in follow-up commits.

Version bump only, so the new binaries can be archived to S3 and
downloaded during configure. Template resync, Lua API checks, and
test coverage work are tracked separately and land in follow-up
commits before this is considered a complete upgrade.
@posit-snyk-bot

posit-snyk-bot commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
Open Source Security 0 0 0 0 0 issues
Licenses 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

Riding along in the same S3-publish branch as the Pandoc/Typst bump,
since Carlos is already archiving from this branch. Version bump
only, no risk analysis done yet for these three, same as Pandoc and
Typst this is prep for archiving, not a verified upgrade.
@cderv cderv changed the title Update Pandoc to 3.10 and Typst to 0.15.0 Update Pandoc, Typst, Dart Sass, esbuild, and veraPDF Jul 9, 2026
cderv added 4 commits July 10, 2026 16:35
Pandoc 3.10 upgraded TableBody to a proper typed Lua object (matching
TableHead/TableFoot, which jog.lua already handled), so the untyped
table fallback branch no longer catches it and every table triggered
"Don't know how to traverse TableBody" during render.
pandoc.readers/writers values changed from boolean to string
("text"/"bytestring") in pandoc 3.9; add missing pandoc.TableBody
constructor and pandoc.types.Sources/Source stubs.
Typst 0.15.0 now hard-rejects backslash path separators in image()
calls. pandoc.path.join uses the host OS's native separator, producing
backslash-joined paths on Windows for any mediabag-routed image (data
URI, remote, or resource-path). Normalize to forward slashes before
assigning as the Image element's src.
@cderv

cderv commented Jul 13, 2026

Copy link
Copy Markdown
Member Author

Typst 0.15 seems to have some impact. Already two issues open with Pandoc

I'll see how to handle.

I noticed a few things to investigate in our failing tests too. Slowly looking in all this

cderv added 4 commits July 15, 2026 14:23
Pandoc 3.10 normalizes to a 3-part version string, so the strict-mode
comparison in check.ts needs to match "3.10.0" not "3.10".

The other fixtures assert on cosmetic pandoc 3.10 typst/asciidoc writer
output-shape changes (newline after raw #set text() inlines, backslash-
escaped leading periods/minus signs after inline markup close brackets,
an added alt argument on image() calls, and double-underscore emphasis
in asciidoc) rather than functional regressions -- rendered output is
unchanged and valid in each case.
…roject-relative images, and mermaid figures

Typst 0.15.0 hard-rejects backslash path separators in image() calls.
Several Quarto code paths build paths with the OS-native separator
(backslash on Windows) and then either double the backslash (assuming
Typst would tolerate it, as older Typst versions did) or leave it
unescaped, both of which now break compilation on Windows: brand logo
resolution (brand.ts + typst-brand-yaml.lua), project-relative image
resolution (project-paths.lua), the generic Typst image writer
(typst.lua), and generated figure paths shared by handlers like mermaid
(handlers/base.ts).

Extracted a shared `to_forward_slashes` helper (modules/path.lua) for
the Lua side, mirroring the existing `pathWithForwardSlashes` TypeScript
helper already used across ~50 call sites, and migrated the two
pre-existing inline occurrences of this same pattern (mediabag.lua,
book-links.lua) onto it instead of leaving a third copy-pasted variant.
Pandoc 3.10 added native alt: derivation for Typst image() calls,
deriving it from the Image's caption field. Quarto already suppresses
its own caption-as-alt fallback for figure images (via the
_quarto_no_caption_alt attribute, added in 1ed4446 to fix #14107),
but that suppression only stopped Quarto's own Lua from setting alt --
it left the Image's caption field populated, so when the bare Image
node fell through to Pandoc's writer, Pandoc's new native derivation
picked the caption right back up, reintroducing the same leak the
earlier fix removed. Clear the caption before returning the bare
Image so Pandoc's writer has nothing to derive alt from in that case.
…ding Lua type stub

typst.lua's backslash-to-forward-slash normalization only ran inside
the alt-text branch; an image with no alt text and no caption fell
through to the bare Image return unnormalized, so a native-separator
src reaching this path (from any source not already fixed at origin)
could still break Typst 0.15+ compilation. Normalize on both paths.

version.lua's sources_specifier alias advertised table<string,string>
as a valid input to pandoc.types.Sources, but empirically that shape
silently produces an empty Sources list at runtime -- confirmed via
`quarto run` against the bundled pandoc. Removed the misleading form.
@cderv

cderv commented Jul 15, 2026

Copy link
Copy Markdown
Member Author

I dug into jgm/pandoc#11761 (leading-dot backslash escaping) to see whether we can work around it on our side for v1.10, and traced the full chain through our own code, not just pandoc's.

Root cause

This only triggers when a path-like metadata value starts with a literal .bibliography: ./refs.bib or ../refs.bib, not bibliography: refs.bib or sub/refs.bib. I confirmed the trigger condition and the full chain end to end:

  1. bibliography: ./refs.bib in the YAML front matter is metadata pandoc parses as markdown inlines, not a raw string.
  2. We don't strip or rewrite that value for Typst — isBibliographyOutput() only covers bibtex/biblatex/csljson, so for typst the bibliography key flows through untouched:
    local function isBibliographyOutput()
    local formats = {
    "bibtex",
    "biblatex",
    "csljson"
    }
    return tcontains(formats, FORMAT)
    end
  3. Our own Typst partial interpolates it straight into a quoted string argument, exactly the pattern the pandoc issue describes:
    #bibliography(($for(bibliography)$"$bibliography$"$sep$,$endfor$))
  4. Pandoc's metaToContext renders that value through inlinesToTypst/escapeTypst, which since 80c704a9a (Typst writer generates invalid code when punctuation directly follows a link jgm/pandoc#11511) prepends a backslash to any string starting with .. Typst 0.14 silently tolerated the stray backslash; Typst 0.15 hard-rejects it (path must not contain a backslash).

I reproduced this locally with pandoc 3.10 + Typst 0.15.0: bibliography: ./refs.bib fails, bibliography: refs.bib renders fine.

jgm's reply

This is a tricky one. Pandoc renders everything in metadata as if it's ordinary text, but this poses a problem in the current case, since typst's escaping rules inside a string or path context are different than for ordinary text. Probably the most straightforward fix would be to roll back 80c704a9acb01fbeffc91a0d05a86c98ab0fe2c0 and replace it with something more subtle, but the replacement isn't obvious, and I'm not sure there wouldn't be other similar problems.

So no fix on the pandoc side yet, and no clear timeline. His workaround is to write the bibliography path as raw Typst so it skips pandoc's metadata escaping entirely:

bibliography: `./refs.bib`{=typst}

I tested this end to end rather than just relaying it: rendered with a single bibliography this way, and separately with a list (- \./refs.bib`{=typst}alongside a second plain entry). Both compiled cleanly and, more importantly, the citation and bibliography entry actually resolved and rendered correctly in the PDF — Quarto disables pandoc's own--citeprocfor Typst and lets Typst resolve citations natively from the#bibliography()` call, so the raw-syntax value reaches Typst as a plain path string with no knock-on effect on citation resolution. This workaround is safe to document for users hitting this now, at least for single-format Typst output (as jgm notes, multi-format documents would need a Lua filter to vary the value by output format).

Possible workaround on our side

We could apply the same idea automatically instead of asking users to hand-write raw syntax. Pandoc's Typst writer emits RawInline (Format "typst") content completely verbatim, bypassing escapeTypst:
source

RawInline fmt str ->
  case fmt of
    Format "typst" -> return $ literal str
    _ -> return mempty

So a small Lua filter gated on isTypstOutput() could rewrite each string in doc.meta.bibliography (it's normalized to a list internally, even for a single entry) into a RawInline("typst", path), sidestepping the escaping bug entirely rather than post-processing the generated .typ (which is more fragile, per the mediabag.lua-style approach we already ruled out for the Windows path-separator issue). Haven't implemented this yet — flagging the direction here before committing to it.

This isn't just bibliography

The same shape of bug — a Quarto Typst partial quoting a raw $metadata-var$ — exists at least one more place:

#set page(background: align($logo.location$, box(inset: $logo.inset$, image("$logo.path$", width: $logo.width$$if(logo.alt)$, alt: "$logo.alt$"$endif$))))

$logo.path$ for brand logos. I tested a brand logo with an explicit leading ./ and confirmed it does not currently trigger this — our own path-resolution normalizes the leading ./ away before the value reaches the template. That's incidental, not a guard against this specific pandoc behavior. Any future Typst partial we write that quotes a metadata value directly is exposed to the same class of bug if that value can ever start with a literal ..

Not blocking the version bump investigation, but wanted to get the full picture written down before deciding how much of this to fix for v1.10 vs. just document as a known limitation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants