Repository navigation
Allow for autodoc to parse Markdown docstrings #228
Description
Activity
There's also the question of numpydoc, which defines its own syntax for some things like parameters. Should myst use the same syntax, but just using Markdown markup in the text? Or should it use something more markdownic?
I have just added definition list syntax rendering 😄 : see https://myst-parser.readthedocs.io/en/latest/using/syntax-optional.html#definition-lists
I think this could come in handy for an autodoc extension. Something like:
# Parameters param1 : Description of param1 param2 : Description of param2Thats maybe more markdownic?
I don't know. There's also the Google docstring style, which is a little different (and preferred by many people). It would probably be a good idea to get broader community feedback on these things.
Reacted by Alexey Strokach, jtpavlock, Alex Tremblay, Jannis Mainczyk, GT and Agriya KhetarpalIt would probably be a good idea to get broader community feedback on these things.
Yep absolutely
But note, numpydoc and Google formats are both built around rST syntax.
A markdown extension would use markdown-it-py to initially parse the docstring, and so any format has to be compatible with it in some fashion: utilising existing syntax plugins, or writing new ones.If it matters I'm working on decoupling parsing from rendering of docstring in IPython/Jupyter ; basically saying the if you can write a parser that goes from
__doc__to some well defined data structure with the right fields/info, then IPython (and by extension Jupyter) will know how to render it properly/nicely. (This could also pull some informations out of__signature__).So, if the raw rendering to user In IPython/Jupyter is bothering you and influencing the syntax you are choosing, this will likely become less of an issue for users.
Thanks @Carreau, I'll bear that in mind 😄
While you're here; I just added https://myst-parser.readthedocs.io/en/latest/using/syntax-optional.html#auto-generated-header-anchors, so that you can write e.g.
[](path/to/doc.md#heading-anchor)and it will work correctly both directly on GitHub and building via sphinx.These anchor slugs, I've found, are a bit changeable in their implementation across renderers, but generally they are converging to the GitHub "specification".
Jupyter Notebook/Lab seems to be a bit outdated in this respect (or at least the versions I tested)?
They don't lower-case or remove punctuation, etc.I'm surprised by this, because I thought they were both generally built around markedjs at the moment (please move to markdown-it 😉), which does implement this behaviour: https://github.com/styfle/marked/blob/a41d8f9aa69a4095aedae93c6e6ee5522a588217/lib/marked.js#L1991
I'm very much interested in this feature as I've been using Markdown doc-strings for a while and would like to move from recommonmark to MyST.
By the way, it took me quite a while to get to this GitHub issue here. It would have helped if the section in the docs regarding the
autodocextension clearly stated that Markdown is not supported in doc-strings.That's a great point @John-Hennig - any interest in adding a PR to add a
```{warning}block there that also links to this issue in case folks want to give feedback?Reacted by John HennigOriginally posted by @asmeurer in #163 (comment)
This issue will be of relevance here: sphinx-doc/sphinx#8018
From the feedback autodoc issue it sounds like it might just be better to write a replacement for autodoc rather than trying to extend it?
Here is a trick to have Markdown docstring with commonmark. I guess it could be done with myst_parser.
Sphinx's Autodoc extension emits an event named autodoc-process-docstring every time it processes a doc-string. You can hook into that mechanism to convert the syntax from Markdown to reStructuredText.
import commonmark def docstring(app, what, name, obj, options, lines): md = '\n'.join(lines) ast = commonmark.Parser().parse(md) rst = commonmark.ReStructuredTextRenderer().render(ast) lines.clear() lines += rst.splitlines() def setup(app): app.connect('autodoc-process-docstring', docstring)
It's funny that you posted that as I made this comment on that a few hours ago.
Here is a trick to have Markdown docstring with commonmark. I guess it could be done with myst_parser.
Yes, it does work with MyST. Since my earlier comment here, I have replaced Recommonmark with MyST in my projects and, as before, I'm using Commonmark.py to render the Markdown doc-strings. I've also updated my Stackoverflow answer to reflect that and mention MyST now that Recommonmark has been deprecated.
This works great for me, actually. But all I need in doc-strings is syntax highlighting of code examples. So nothing fancy. People who want advanced features such as math rendering, cross references, or possibly NumPy style, will have to wait for native doc-string support in MyST.
@John-Hennig Great, could you share your code with MyST? TIA.
Today I found https://github.com/mkdocstrings/mkdocstrings, is it related to the scope of this issue?
23 remaining items
Oh, didn't realize
:paramis now MyST, thanks! I was also interested in NumPy-style docstringsWhen we talk about numpy/google style, I would start by asking;
would you agree that, given we now have type annotations and type checking, it is no longer good practice to put types in the docstring?
That would simplify things a littleReacted by Jonas Wunderlich, Mark Shui Hu, GT and Agriya KhetarpalI would.
Then, if you don't want sphinx style, I would suggest a bit of a hybrid, that would work for both rst and myst:
basically a heading followed by a field list, e.g.for MyST
# Parameters :x: a description :y: a description
or for RST
Parameters ----------- :x: a description :y: a description
this would be very easy to parse, just with the standard rst/myst parser,
then you just run a "transform" on the AST, that finds these headings and "propogates" them down to the field list, i.e. to get back to the sphinx style:param x: a description :param y: a descriptionReacted by Hans-Martin von Gaudecker, Juan Luis Cano Rodríguez, Ryan Morshead, Stacy Kim, Yury Gorishniy, Francisco, Mark Shui Hu and Agriya KhetarpalThat field list syntax would be perfectly acceptable for me personally coming from Google style docstrings. With that said, it would certainly be helpful for projects trying to transition to MyST if Google/Numpy styles were supported as it would require less work and receive less push back from those who might already find RST->MyST to be an uncomfortable change.
That field list syntax would be perfectly acceptable for me personally coming from Google style docstrings. With that said, it would certainly be helpful for projects trying to transition to MyST if Google/Numpy styles were supported as it would require less work and receive less push back from those who might already find RST->MyST to be an uncomfortable change.
Agreed. Though a converter script might do the job and ease the maintenance burden.
Note something like this may be of use: https://pypi.org/project/docstring-parser/
Google/numpy are a bit weird, in that they are "pseudo rst", with effectively a bespoke "structure" with nested rst. But I guess one could parse the structure first, even with myst, then parse properly
Is there a possible workaround (maybe using/adding other dependencies) for auto parsing docstrings that use both myst and napoleon?
Is there a possible workaround (maybe using/adding other dependencies) for auto parsing docstrings that use both myst and napoleon?
Oh indeed, thats what I mean by the above, you just need a "hook" in: https://github.com/sphinx-extensions2/sphinx-autodoc2/blob/13933a5b25a780e03f227414d432420706962212/src/autodoc2/sphinx/docstring.py#L125
to allow for "re-interpretation" of the docstringThat solution however requires using sphinx-autodoc2 which is not as popular as I would like, so I would rather wait. I need a more battle-tested approach.
That solution however requires using sphinx-autodoc2 which is not as popular as I would like, so I would rather wait. I need a more battle-tested approach.
Ah well thats a chicken and egg 😅 I only created it a few weeks ago, and need your guys help to test/improve it, all issues/PRs welcome 🙏
Reacted by Juan Luis Cano Rodríguez, Nicolás Quiroz and FranciscoGreat Project! But we also ran into the issue of not wanting to use the Sphinx-Style when I started transitioning one of our projects to
autodoc2. I was able to convert some of our Numpy-Docstrings using Pyment, but we ultimately put it on hold. I think the proposed Heading + Fieldlist style would already be enough for us.Note something like this may be of use: https://pypi.org/project/docstring-parser/
Google/numpy are a bit weird, in that they are "pseudo rst", with effectively a bespoke "structure" with nested rst. But I guess one could parse the structure first, even with myst, then parse properlyThat's the approach I took with Griffe: it parses the different styles into the same data structures/classes. Basically, it parses a docstring into a list of sections, each section having its own specific kind and contents (regular text, arguments, returns, exceptions, etc.).
It's (almost) markup agnostic: regular text sections as well as as any item description (parameter, returned value, etc.) can be written in Markdown, rST, Asciidoc, whatever the end user prefers. I wrote almost because Griffe's parsers still check for fenced code blocks (using triple-backticks) to prevent parsing of sections inside Markdown code blocks. This is not an issue for rST since they would be indented and therefore not matched.
Anyway, just a shameless plug 😄 See usage examples here: https://mkdocstrings.github.io/griffe/parsing_docstrings/
Reacted by Shahriar HeidrichReacted by Shahriar HeidrichWhat if we just add
eval-mystdirective and wrap all docstrings into it inautodoc-process-docstringhook?I toyed with this approach for a bit and it seems to work:
MVP implementation
This is quick-and-dirty implementation to test the concept.
from typing import Any from sphinx.application import Sphinx from sphinx.util.docutils import SphinxDirective from myst_parser.parsers.sphinx_ import MystParser class EvalMystDirective(SphinxDirective): has_content = True def run(self): # Document tracks node IDs and other things, so we can't lust copy it. # Instead, we monkeypatch its `children` node without disrupting # anything else. This way, all `document.note_*` functions still work, # and all nodes returned from MyST parser have correct document reference. # Note: the code could be less hacky if `MystParser` accepted # `document` and `content_node` as separate parameters. document = self.state.document prev_children = document.children children = document.children = [] try: parser = MystParser() parser.parse("\n".join(self.content), document) finally: document.children = prev_children return children def handle_myst( app: Sphinx, what_: str, name: str, obj: Any, options: dict[str, bool], lines: list[str], ): lines[:] = [".. eval-myst::", ""] + [" " + line for line in lines] def setup(app: Sphinx): app.add_directive("eval-myst", EvalMystDirective) app.connect("autodoc-process-docstring", handle_myst) return {"version": "0.0.0", "parallel_read_safe": True}
This is a minimal implementation, but can be expanded to allow better control over which modules use which syntax, etc.
The only thing I didn't figure out is transforms, specifically
ResolveAnchorIdswill have to run for every document that includes MySt content, even if this document is RST.- added a commit that references this issue
on Apr 1, 2026
Originally posted by @asmeurer in #163 (comment)
This issue will be of relevance here: sphinx-doc/sphinx#8018