Skip to content

Add markdown dialect support with GitHub Flavored Markdown as the first dialect #30

Description

The object hierarchy delivered in #8 for 1.3 models CommonMark exactly and nothing else. CommonMark is the common denominator, but almost no markdown a user encounters in the wild is only CommonMark. A README.md, a pull request body, and an issue description are all GitHub Flavored Markdown, and a table in any of them parses as ordinary paragraph text under a strict CommonMark parser.

This issue adds dialect support, targeting 1.5.

Request

Desired capability

A dialect selector on ConvertFrom-Markdown and ConvertTo-Markdown that extends the base grammar with a named set of constructs, with GFM as the first dialect:

$doc = Get-Content -Raw 'README.md' | ConvertFrom-Markdown -Dialect GitHub

$doc.Descendants('Table') | ForEach-Object { $_.Header.GetText() }
$doc.Descendants('ListItem') | Where-Object IsTask | Where-Object { -not $_.IsChecked }

Documents parsed with a dialect render back through the same dialect, so a GFM table survives a round trip instead of degrading into text.

GFM as the first dialect

GFM is a strict superset of CommonMark, so it adds constructs without redefining any existing ones:

Construct GFM spec Model
Tables §4.10 New node types MarkdownTable, MarkdownTableRow, MarkdownTableCell, and a MarkdownTableAlignment enum
Task list items §5.3 IsTask and IsChecked properties added to MarkdownListItem — a task list item is a list item, not a new type
Strikethrough §6.5 New inline node MarkdownStrikethrough
Extended autolinks §6.9 An IsExtended property on MarkdownAutolink, distinguishing <https://x> from bare www.x
Disallowed raw HTML §6.11 Not modelled — it is an output-sanitization rule for HTML rendering, which this module does not do

Acceptance criteria

  • ConvertFrom-Markdown and ConvertTo-Markdown accept an optional -Dialect parameter that defaults to CommonMark, so existing behavior is unchanged for callers that do not pass it
  • -Dialect GitHub parses and renders every GFM extension construct listed above
  • A document parsed with one dialect records that dialect, so ConvertTo-Markdown renders it consistently without the caller repeating the parameter
  • Dialect constructs are additive: no CommonMark node type changes shape, and no CommonMark document parses differently under a dialect than it does under CommonMark
  • Conformance is measured against the GFM specification's own example set, in the same way #8 measures CommonMark conformance
  • Adding a further dialect requires no change to the CommonMark parser or node types

Other dialects

GFM is the priority because it is what GitHub renders. Other dialects in common use, listed for scope awareness — none are committed to here:

Dialect Where it is used Notable additions over CommonMark
GitHub Flavored Markdown GitHub Tables, task lists, strikethrough, extended autolinks
Python-Markdown with PyMdown Extensions MkDocs and Material for MkDocs — the stack this organization's own documentation sites use Admonitions, content tabs, nested fences, snippets, definition lists, footnotes, attribute lists
kramdown Jekyll and GitHub Pages Attribute lists, definition lists, footnotes, block inline-attribute syntax
Pandoc Markdown Academic and publishing workflows Citations, footnotes, definition lists, grid tables, math
MultiMarkdown Publishing toolchains Metadata, footnotes, citations, table spans
Obsidian Obsidian vaults Wikilinks, embeds, callouts, block references
MDX React documentation sites JSX embedded in markdown

The two worth considering after GFM are Python-Markdown with PyMdown Extensions — because this organization publishes its documentation with Material for MkDocs — and kramdown, for GitHub Pages.

Dependencies

  • PSModule/Markdown#8 — 1.3 delivers the CommonMark object hierarchy, parser, and renderer, including the lookup-table seam this issue extends

Technical decisions

Parameter name — -Dialect: Both "flavor" and "dialect" are in circulation; GFM itself uses "dialect" in its own introduction ("GFM is the dialect of Markdown that is currently supported for user content on GitHub.com"). Dialect also reads better next to a value like GitHub.

Parameter type — a MarkdownDialect enum: Values CommonMark (default) and GitHub. An enum gives tab completion and makes invalid values unrepresentable. Combining dialects is not supported — a document is written in one dialect.

Additive, never destructive: A dialect may add node types, add properties to existing node types, and add entries to the parser's block-start and inline-delimiter tables. It may not remove or redefine CommonMark constructs. This keeps -Dialect GitHub a strict superset in behavior as well as in specification, and it means a CommonMark document parses identically either way.

Extension mechanism — table registration, not parser forks: #8 drives block starts and inline delimiters from lookup tables. A dialect contributes entries to those tables. This is the same approach Markdig takes with its pipeline extensions, and it is why adding GFM requires no changes to the CommonMark parser.

Dialect recorded on the document: MarkdownDocument gains a [MarkdownDialect] $Dialect property set at parse time. ConvertTo-Markdown reads it, so a document round-trips through the dialect it was parsed with unless the caller overrides it. Adding a property is additive, so this is not a breaking change to #8.

Task list items are properties, not a type: A GFM task list item is a list item whose first inline content is a checkbox marker. Modelling it as IsTask and IsChecked on MarkdownListItem keeps list traversal uniform — code that walks list items does not need to know about a second item type.

Release shape: A new optional parameter with a backward-compatible default, a new property on MarkdownDocument, and new node types. Entirely additive, so a minor bump — 1.5.

Test approach: The GFM specification publishes its examples in the same machine-readable form as CommonMark. The conformance harness from #8 is reused, run a second time with -Dialect GitHub over the GFM example set. A regression test asserts that every CommonMark example produces an identical tree under both dialects.


Implementation plan

Dialect plumbing

  • Define the MarkdownDialect enum with CommonMark and GitHub
  • Add [MarkdownDialect] $Dialect to MarkdownDocument
  • Add -Dialect to ConvertFrom-Markdown, defaulting to CommonMark
  • Add -Dialect to ConvertTo-Markdown, defaulting to the document's own Dialect
  • Register dialect contributions into the block-start and inline-delimiter tables

Table support

  • Define MarkdownTable, MarkdownTableRow, MarkdownTableCell, and MarkdownTableAlignment
  • Add table detection to the block parser as a registered block start
  • Implement column alignment parsing from the delimiter row
  • Implement table rendering in ConvertTo-Markdown

Task list items

  • Add IsTask and IsChecked to MarkdownListItem
  • Detect the checkbox marker during list item parsing
  • Render the checkbox marker when IsTask is set

Strikethrough

  • Define MarkdownStrikethrough
  • Register the ~ delimiter run in the inline delimiter table
  • Implement rendering

Extended autolinks

  • Add IsExtended to MarkdownAutolink
  • Implement bare www., http://, https://, and email detection during the inline pass
  • Render extended autolinks without angle brackets

Tests

  • Run the GFM specification example set through the conformance harness with -Dialect GitHub
  • Assert every CommonMark example produces an identical tree under both dialects
  • Assert a GFM document parsed with -Dialect CommonMark degrades predictably rather than erroring

Documentation

  • Document -Dialect in both functions' comment-based help
  • Document the dialect model and the GFM node types in README.md
  • Note that the Set-MarkdownTable DSL output requires -Dialect GitHub to round-trip

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions