Skip to content

Have an option to output warnings from doctests #41574

Description

@Susurrus

Running cargo test runs doctests by default, and shows warnings for regular code, but not for the doctests themselves. This can be enabled by running cargo test -- --nocapture. I'd like to have warnings also show in our CI tests, even for doctests, but setting -- --nocapture shows all the output which we don't want (and can't control, because they come from the underlying libraries complaining about our tests that are supposed to fail).

I'd think having warnings on for doctests by default would be good, but that wouldn't be backwards compatible. Maybe a --doctest-warnings option could be added in lieu of that?

I believe this was done in 5d145c1, so discussion there might be worth reading. It seems that there are a bunch of warnings that come up with doctests that should be ignored, but there are also ones that can appear that shouldn't be. The question is how do we address both needs?

Activity

  1. added
    T-rustdocRelevant to the rustdoc team, which will review and decide on the PR/issue.
    C-enhancementCategory: An issue proposing an enhancement or a PR with one.
    and removed
    C-enhancementCategory: An issue proposing an enhancement or a PR with one.
    on Apr 27, 2017
  2. GuillaumeGomez commented on Apr 28, 2017

    @GuillaumeGomez
    Member

    I can put it back if you want. Just waiting for @rust-lang/docs team confirmation.

  3. frewsxcv commented on Apr 28, 2017

    @frewsxcv
    Contributor

    This might be a tools team question, less-so a docs team one.

  4. GuillaumeGomez commented on Apr 29, 2017

    @GuillaumeGomez
    Member

    Let's make them join the issue then!

    cc @rust-lang/tools

  5. changed the title [-]doctests don't print warnings by default[/-] [+]Have an option to output warnings from doctests[/+] on Apr 29, 2017
  6. carols10cents commented on Apr 29, 2017

    @carols10cents
    Member

    I've updated the title to better reflect what I think this request is, please correct it if I've misunderstood.

    Many doc tests are small and would have warnings for unused variables, etc. I wouldn't want to see these by default, but I'd be ok with a flag to turn on just output of doc test warnings. I don't think I'd use it, but I wouldn't mind it being there.

    Is there any way to get a sense of how many people would use this if it existed and the kinds of problems it would prevent, in order to prioritize this?

  7. alexcrichton commented on Apr 29, 2017

    @alexcrichton
    Member

    The standard library handles this by denying all warnings, but it seems reasonable to have a flag. I don't think we can change the default behavior.

  8. GuillaumeGomez commented on Apr 30, 2017

    @GuillaumeGomez
    Member

    Ok, so I think we can consider this as acceptable. I'll add the flag.

  9. added
    B-unstableBlocker: Implemented in the nightly compiler and unstable.
    on May 5, 2017
  10. added a commit that references this issue on May 5, 2017
  11. added
    T-dev-toolsRelevant to the dev-tools subteam, which will review and decide on the PR/issue.
    and removed on May 18, 2017
  12. 14 remaining items

  13. added a commit that references this issue on Apr 6, 2018
  14. steveklabnik commented on Jul 23, 2019

    @steveklabnik
    Contributor

    Triage: not aware of any movement on this front, though @BurntSushi did come up with a good use-case for this in #55632

  15. GuillaumeGomez commented on Jun 13, 2020

    @GuillaumeGomez
    Member

    Just realized it's been ages ago. We have a --display-warnings option on rustdoc (nightly only for the moment). I'll send a PR for stabilization. Closing this issue then.

    EDIT: actually, I think there are issues remaining. Just not sure if they should be considered as issues or not... Re-opening the issue until we have this option stabilized.

  16. added
    A-doctestsArea: Documentation tests, run by rustdoc
    on Aug 27, 2020
  17. camelid commented on Oct 2, 2021

    @camelid
    Member

    Re-opening the issue until we have this option stabilized.

    The option is still unstable; the PR that closed this just renamed it.

  18. reopened this on Oct 2, 2021
  19. jyn514 commented on Nov 26, 2021

    @jyn514
    Member

    Many doc tests are small and would have warnings for unused variables, etc. I wouldn't want to see these by default, but I'd be ok with a flag to turn on just output of doc test warnings. I don't think I'd use it, but I wouldn't mind it being there.

    I just opened #91259 which
    a) doesn't show any warnings by default,
    b) lets you use --test-args --show-output to show all warnings but the unused lints, and
    c) lets you add #![warn(unused)] to get all warnings without exceptions.

    Does that seem like a reasonable solution? Or do you think rustdoc should show all lints besides unused by default? That's not quite a breaking change, but I'd want to do a crater run to see how noisy it ends up being (especially since I don't think we should add a way to disable the warnings per-invocation if we do that).

  20. jyn514 commented on Nov 26, 2021

    @jyn514
    Member

    do you think rustdoc should show all lints besides unused by default?

    I tried implementing this just now and it was a little tricky, since if rustdoc passes --show-output to libtest, it will show output from all tests, including compile_fail tests. Not sure how this could work.

  21. added a commit that references this issue on Nov 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-doctestsArea: Documentation tests, run by rustdocB-unstableBlocker: Implemented in the nightly compiler and unstable.C-tracking-issueCategory: An issue tracking the progress of sth. like the implementation of an RFCT-dev-toolsRelevant to the dev-tools subteam, which will review and decide on the PR/issue.T-rustdocRelevant to the rustdoc team, which will review and decide on the PR/issue.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions