aboutsummaryrefslogtreecommitdiff
path: root/src/Text/Pandoc/Readers/HTML.hs
AgeCommit message (Collapse)Author
2025-12-12Fix imports to avoid warning about foldl'.John MacFarlane
`foldl'` is exported by Prelude by base 4.20+. So we need to do some qualified importing to avoid warnings about redundant imports.
2025-12-05HTML reader: parse inline style elements as RawInline.John MacFarlane
This is a cleaner fix to #10643 than the reverted commit 7fe8c92. Styles are not stripped, but with this change they will not interfere with inline parsing. Closes #11246.
2025-12-05Revert "HTML reader: ignore style tags in the body."John MacFarlane
This reverts commit 7fe8c928212197354196724e9b95b71a854fdca9. See #10643 and #11246.
2025-10-11HTML reader: allow blank space between open and close iframe.John MacFarlane
2025-08-26HTML reader: don't drop the initial newline in a pre element.John MacFarlane
Closes #11064.
2025-03-10HTML reader: ignore style tags in the body.John MacFarlane
They are invalid but do occur in the wild. Closes #10643.
2025-03-08HTML reader: skip MathJaX-introduced cruft.John MacFarlane
See #10673.
2025-01-07HTML reader: add size information for fa svg icons.John MacFarlane
If the icon has class fa-fw or fa-w16 or fa-w14, we add a width attribute to prevent the icon from appearing full-width in PDF or docx output. Closes #10134.
2024-12-18HTML reader: don't canonicalize data: URIs.John MacFarlane
It can be very expensive to call network-uri's URI parser on these. See #10075.
2024-12-14Store a function instead of a BooleanJoseph C. Sible
Instead of storing isDisplay and then always choosing displayMath or math based on that, just store displayMath or math directly.
2024-12-07HTML reader: parse footnotes defined by dpub-aria roles.John MacFarlane
Closes #5294.
2024-09-03HTML reader: only parse main element's contents (if present).John MacFarlane
If main has an id or class, we include a div with that id or class; otherwise just the contents. Closes #10140.
2024-09-01HTML reader: read TeX annotation in mathml content if present.John MacFarlane
KaTeX (and probably other tools that produce MathML from TeX) includes an annotation tag with the original TeX; we extract this if present instead of converting the MathML. See #9971.
2024-08-31HTML reader: better handle KaTeX-generated math.John MacFarlane
KaTeX emits the mathml followed by a span with an HTML fallback. Previously pandoc was converting both. We now ignore the HTML fallback span, marked with class `katex-html`. Closes #9971.
2024-05-28HTML reader: better handle non-li elements in ul and ol.John MacFarlane
For example, a `p` after a closed `li` will be incorporated into the previous `li`. This mirrors what browsers do with this invalid HTML. Closes #9809.
2024-04-25Update copyright dates to 2024.John MacFarlane
2023-12-06Revert "Use base64 instead of base64-bytestring."John MacFarlane
This reverts commit 6625e9655ed2bb0c4bd4dd91b5959a103deab1cb. base64 is currently buggy on 32-bit systems. Closes #9233.
2023-11-15HTML reader: Fix handling of invalidly nested sublists.John MacFarlane
This revises the fix to #8150 (and the test case) and closes #9187. HTML in the (invalid) form: <ul> <li>L1</li> <ul> <li>L1.1</li> </ul> </ul> is treated by browsers like <ul> <li>L1 <ul> <li>L1.1</li> </ul> </li> </ul> not <ul> <li>L1 <li><ul> <li>L1.1</li> </ul> </li> </ul> as pandoc previously assumed. This change will give a similar treatment to <ul> <li>L1</li> <p>foobar</p> </ul> which also seems to match browser behavior.
2023-10-27Fix spacing.John MacFarlane
2023-09-16HTML reader: parse task lists using input elements (#9066)Seth Speaks
Allow the HTML reader to parse task lists of the sort produced by pandoc. Closes #9047
2023-08-09HTML reader: avoid duplicate id on header and div.John MacFarlane
Closes #8991.
2023-05-17HTML reader: fix iframe with data URI of an image.John MacFarlane
Closes #8856. In this case we don't want to try to parse the data at the URL. Instead, create an image inside a div.
2023-03-20HTML reader: fix behavior with `-native_spans-raw_html`.John MacFarlane
Previously with this configuration, `<span>`s were not treated as inline elements at all. Closes #8711.
2023-01-13Support complex figures. [API change]Albert Krewinkel
Thanks and credit go to Aner Lucero, who laid the groundwork for this feature in the 2021 GSoC project. He contributed many changes, including modifications to the readers for HTML, JATS, and LaTeX, and to the HTML and JATS writers. Shared (Albert Krewinkel): - The new function `figureDiv`, exported from `Text.Pandoc.Shared`, offers a standardized way to convert a figure into a Div element. Readers (Aner Lucero): - HTML reader: `<figure>` elements are parsed as figures, with the caption taken from the respective `<figcaption>` elements. - JATS reader: The `<fig>` and `<caption>` elements are parsed into figure elements, even if the contents is more complex. - LaTeX reader: support for figures with non-image contents and for subfigures. - Markdown reader: paragraphs containing just an image are treated as figures if the `implicit_figures` extension is enabled. The identifier is used as the figure's identifier and the image description is also used as figure caption; all other attributes are treated as belonging to the image. Writers (Aner Lucero, Albert Krewinkel): - DokuWiki, Haddock, Jira, Man, MediaWiki, Ms, Muse, PPTX, RTF, TEI, ZimWiki writers: Figures are rendered like Div elements. - Asciidoc writer: The figure contents is unwrapped; each image in the the figure becomes a separate figure. - Classic custom writers: Figures are passed to the global function `Figure(caption, contents, attr)`, where `caption` and `contents` are strings and `attr` is a table of key-value pairs. - ConTeXt writer: Figures are wrapped in a "placefigure" environment with `\startplacefigure`/`\endplacefigure`, adding the features caption and listing title as properties. Subfigures are place in a single row with the `\startfloatcombination` environment. - DocBook writer: Uses `mediaobject` elements, unless the figure contains subfigures or tables, in which case the figure content is unwrapped. - Docx writer: figures with multiple content blocks are rendered as tables with style `FigureTable`; like before, single-image figures are still output as paragraphs with style `Figure` or `Captioned Figure`, depending on whether a caption is attached. - DokuWiki writer: Caption and "alt-text" are no longer combined. The alt text of a figure will now be lost in the conversion. - FB2 writer: The figure caption is added as alt text to the images in the figure; pre-existing alt texts are kept. - ICML writer: Only single-image figures are supported. The contents of figures with additional elements gets unwrapped. - HTML writer: the alt text is no longer constructed from the caption, as was the case with implicit figures. This reduces duplication, but comes at the risk of images that are missing alt texts. Authors should take care to provide alt texts for all images. Some readers, most notably the Markdown reader with the `implicit_figures` extension, add a caption that's identical to the image description. The writer checks for this and adds an `aria-hidden` attribute to the `<figcaption>` element in that case. - JATS writer: The `<fig>` and `<caption>` elements are used write figures. - LaTeX writer: complex figures, e.g. with non-image contents and subfigures, are supported. The `subfigure` template variable is set if the document contains subfigures, triggering the conditional loading of the *subcaption* package. Contants of figures that contain tables are become unwrapped, as longtable environments are not allowed within figures. - Markdown writer: figures are output as implicit figures if possible, via HTML if the `raw_html` extension is enabled, and as Div elements otherwise. - OpenDocument writer: A separate paragraph is generated for each block element in a figure, each with style `FigureWithCaption`. Behavior for single-image figures therefore remains unchanged. - Org writer: Only the first element in a figure is given a caption; additional block elements in the figure are appended without any caption being added. - RST writer: Single-image figures are supported as before; the contents of more complex images become nested in a container of type `float`. - Texinfo writer: Figures are rendered as float with type `figure`. - Textile writer: Figures are rendered with the help of HTML elements. - XWiki: Figures are placed in a group. Co-authored-by: Aner Lucero <[email protected]>
2023-01-10Update copyright years, it's 2023!Albert Krewinkel
2022-10-18T.P.Shared: remove `escapeURI`, `isURI`.John MacFarlane
These are now exported by Text.Pandoc.URI, and removing them from Shared helps make module structure more straightforward.
2022-10-18Revert "HTML reader: avoid duplicating any existing identifier..."John MacFarlane
This reverts commit e5fbddd3b6c0c7a3b76b313edbe55242e3b138fc.
2022-10-18Revert "T.P.Parsing: export `registerIdentifier`."John MacFarlane
This reverts commit 20492d523c8324e36781cfbbc8092c796f94b151.
2022-10-18T.P.Parsing: export `registerIdentifier`.John MacFarlane
[API change] Use this in the HTML reader to register identifiers to avoid duplicates created by `auto_identifiers`.
2022-10-18HTML reader: avoid duplicating any existing identifier...John MacFarlane
with `auto_identifiers`. Closes #8383.
2022-10-16T.P.Parsing: export errorMessages, messageString.John MacFarlane
[API change]
2022-10-16T.P.Parsing: Remove gratuitious renaming of Parsec types.John MacFarlane
We were exporting Parser, ParserT as synonyms of Parsec, ParsecT. There is no good reason for this and it can cause confusion. Also, when possible, we replace imports of Text.Parsec with T.P.Parsing. The idea is to make it easier, at some point, to switch to megaparsec or another parsing engine if we want to. T.P.Parsing new exports: Stream(..), updatePosString, SourceName, Parsec, ParsecT [API change]. Removed exports: Parser, ParserT [API change].
2022-09-28HTML Reader: Tests for 'tt' and 'code' elements (#8330).Justin Wood
tt is a deprecated element in HTML, but is still used in the wild in some places, support reading it as just another 'code' element. Commit 4abb9d0ad8dbb88fbc443a78d5a1b116cb7a5816 was originally part of this PR as well.
2022-09-28HTML reader: fix regression for `<tt>`.John MacFarlane
It was no longer being parsed as Code. See #8330.
2022-08-30Mediawiki reader: Parse table cell with attribs, to support rowspan, colspan ↵Ruqi
(#8231)
2022-08-21Fix regression with data uris in 2.19.1.John MacFarlane
In 2.19.1 we used the base64URL encoding rather than base64. This works in Safari, apparently, but not in other browsers. Closes #8239.
2022-08-14Use base64 instead of base64-bytestring.John MacFarlane
It is supposed to be faster and more standards-compliant.
2022-08-03HTML reader: allow sublists that are not marked as items.Albert Krewinkel
The HTML standard requires all list items to be marked with a `<li>` tag, but some tools fail to do so for sublists. The reader now accepts these unwrapped lists as sublists. Closes: #8150
2022-07-06Use `formatCode` from #7525 in HTML and MediaWiki (#8162)Elliot Bobrow
Move formatting from inside inline code elements to the outside in order to retain formatting.
2022-02-09Fix parsing of epub footnotes.John MacFarlane
Closes #7884.
2022-02-07HTML reader: give warnings and emit empty note...John MacFarlane
when parsing `<a epub:type="noteref">` and the identifier doesn't correspond to anything in the note table. Previously we just silently skipped these cases. See #7884.
2022-01-02Copyright notices: update for 2022Albert Krewinkel
2021-11-24HTML reader: parse attributes on links and images.John MacFarlane
Closes #6970.
2021-10-22Use simpleFigure in Readers.Aner Lucero
2021-07-06Recognize data-external when reading HTML img tags (#7429)Michael Hoffmann
Preserve all attributes in img tags. If attributes have a `data-` prefix, it will be stripped. In particular, this preserves a `data-external` attribute as an `external` attribute in the pandoc AST.
2021-05-17HTML writer: keep attributes from code nested below pre tag.Albert Krewinkel
If a code block is defined with `<pre><code class="language-x">…</code></pre>`, where the `<pre>` element has no attributes, then the attributes from the `<code>` element are used instead. Any leading `language-` prefix is dropped in the code's *class* attribute are dropped to improve syntax highlighting. Closes: #7221
2021-05-15HTML writer: parse `<header>` as a DivAlbert Krewinkel
HTML5 `<header>` elements are treated like `<div>` elements.
2021-05-14HTML reader: keep h1 tags as normal headers (#7274)Albert Krewinkel
The tags `<title>` and `<h1 class="title">` often contain the same information, so the latter was dropped from the document. However, as this can lead to loss of information, the heading is now always retained. Use `--shift-heading-level-by=-1` to turn the `<h1>` into the document title, or a filter to restore the previous behavior. Closes: #2293
2021-05-14HTML reader: don't fail on unmatched closing "script" tag.Albert Krewinkel
Prevent the reader from crashing if the HTML input contains an unmatched closing `</script>` tag. Fixes: #7282
2021-05-09Change reader types, allowing better tracking of source positions.John MacFarlane
Previously, when multiple file arguments were provided, pandoc simply concatenated them and passed the contents to the readers, which took a Text argument. As a result, the readers had no way of knowing which file was the source of any particular bit of text. This meant that we couldn't report accurate source positions on errors or include accurate source positions as attributes in the AST. More seriously, it meant that we couldn't resolve resource paths relative to the files containing them (see e.g. #5501, #6632, #6384, #3752). Add Text.Pandoc.Sources (exported module), with a `Sources` type and a `ToSources` class. A `Sources` wraps a list of `(SourcePos, Text)` pairs. [API change] A parsec `Stream` instance is provided for `Sources`. The module also exports versions of parsec's `satisfy` and other Char parsers that track source positions accurately from a `Sources` stream (or any instance of the new `UpdateSourcePos` class). Text.Pandoc.Parsing now exports these modified Char parsers instead of the ones parsec provides. Modified parsers to use a `Sources` as stream [API change]. The readers that previously took a `Text` argument have been modified to take any instance of `ToSources`. So, they may still be used with a `Text`, but they can also be used with a `Sources` object. In Text.Pandoc.Error, modified the constructor PandocParsecError to take a `Sources` rather than a `Text` as first argument, so parse error locations can be accurately reported. T.P.Error: showPos, do not print "-" as source name.