Skip to content

Machine readable output for dotnet list package #7752

Closed
NuGet/NuGet.Client
#4855

Description

@adamdriscoll

Details about Problem

It is difficult to parse the output of dotnet list package. We have requirements in our organization to audit packages that we are using in our projects. Without having a good way to parse the output of this command, it makes it difficult to structure the list of packages in a way that can be fed into another auditing system.

Running the following command produces the following output:

dotnet list package --include-transitive
Project 'UniversalDashboard' has the following package references
   [net472]:
   Top-level Package                                           Requested          Resolved
   > Microsoft.AspNetCore                                      2.1.3              2.1.3
   > Microsoft.AspNetCore.Authentication.Cookies               2.1.2              2.1.2
   > Microsoft.AspNetCore.Authentication.Facebook              2.1.2              2.1.2
   > Microsoft.AspNetCore.Authentication.Google                2.1.2              2.1.2
   > Microsoft.AspNetCore.Authentication.JwtBearer             2.1.2              2.1.2
   > Microsoft.AspNetCore.Authentication.MicrosoftAccount      2.1.2              2.1.2
   > Microsoft.AspNetCore.Authentication.Twitter               2.1.2              2.1.2
   > Microsoft.AspNetCore.Mvc                                  2.1.2              2.1.2
   > Microsoft.AspNetCore.ResponseCompression                  2.1.1              2.1.1
   > Microsoft.AspNetCore.Routing                              2.1.1              2.1.1
   > Microsoft.AspNetCore.Server.Kestrel.Https                 2.1.3              2.1.3
   > Microsoft.AspNetCore.Server.Kestrel.Transport.Libuv       2.1.3              2.1.3
   > Microsoft.AspNetCore.Session                              2.1.1              2.1.1
   > Microsoft.AspNetCore.SignalR                              1.0.3              1.0.3
   > Microsoft.AspNetCore.StaticFiles                          2.1.1              2.1.1
   > Microsoft.AspNetCore.WebSockets                           2.1.1              2.1.1
   > Microsoft.DotNet.PlatformAbstractions                     2.1.0              2.1.0
   > Microsoft.Extensions.Caching.Memory                       2.1.2              2.1.2
   > Microsoft.Extensions.Logging.Debug                        2.1.1              2.1.1
   > Microsoft.Extensions.Logging.Filter                       1.1.2              1.1.2
   > Newtonsoft.Json                                           11.0.2             11.0.2
   > NLog                                                      4.5.8              4.5.8
   > NLog.Web.AspNetCore                                       4.6.0              4.6.0
   > PowerShellStandard.Library                                5.1.0-preview-06   5.1.0-preview-06
   > Quartz                                                    3.0.6              3.0.6
   > System.Net.NameResolution                                 4.3.0              4.3.0
   > System.Reflection.Emit                                    4.3.0              4.3.0
   > System.Runtime.Loader                                     4.3.0              4.3.0
   > System.Security.Principal                                 4.3.0              4.3.0

   Transitive Package                                                Resolved
   > Libuv                                                           1.10.0
   > Microsoft.AspNetCore.Antiforgery                                2.1.1
   > Microsoft.AspNetCore.Authentication                             2.1.2
   > Microsoft.AspNetCore.Authentication.Abstractions                2.1.1
   > Microsoft.AspNetCore.Authentication.Core                        2.1.1
   > Microsoft.AspNetCore.Authentication.OAuth                       2.1.2
   > Microsoft.AspNetCore.Authorization                              2.1.1
   > Microsoft.AspNetCore.Authorization.Policy                       2.1.1
   > Microsoft.AspNetCore.Connections.Abstractions                   2.1.3
   > Microsoft.AspNetCore.Cors                                       2.1.1
   > Microsoft.AspNetCore.Cryptography.Internal                      2.1.1
   > Microsoft.AspNetCore.DataProtection                             2.1.1
   > Microsoft.AspNetCore.DataProtection.Abstractions                2.1.1
   > Microsoft.AspNetCore.Diagnostics                                2.1.1
   > Microsoft.AspNetCore.Diagnostics.Abstractions                   2.1.1
   > Microsoft.AspNetCore.HostFiltering                              2.1.1
   > Microsoft.AspNetCore.Hosting                                    2.1.1
   > Microsoft.AspNetCore.Hosting.Abstractions                       2.1.1
   > Microsoft.AspNetCore.Hosting.Server.Abstractions                2.1.1
   > Microsoft.AspNetCore.Html.Abstractions                          2.1.1
   > Microsoft.AspNetCore.Http                                       2.1.1
   > Microsoft.AspNetCore.Http.Abstractions                          2.1.1
   > Microsoft.AspNetCore.Http.Connections                           1.0.3
   > Microsoft.AspNetCore.Http.Connections.Common                    1.0.3
   > Microsoft.AspNetCore.Http.Extensions                            2.1.1
   > Microsoft.AspNetCore.Http.Features                              2.1.1
   > Microsoft.AspNetCore.HttpOverrides                              2.1.1
   > Microsoft.AspNetCore.JsonPatch                                  2.1.1
   > Microsoft.AspNetCore.Localization                               2.1.1
   > Microsoft.AspNetCore.Mvc.Abstractions                           2.1.2
   > Microsoft.AspNetCore.Mvc.ApiExplorer                            2.1.2
   > Microsoft.AspNetCore.Mvc.Core                                   2.1.2
   > Microsoft.AspNetCore.Mvc.Cors                                   2.1.2
   > Microsoft.AspNetCore.Mvc.DataAnnotations                        2.1.2
   > Microsoft.AspNetCore.Mvc.Formatters.Json                        2.1.2
   > Microsoft.AspNetCore.Mvc.Localization                           2.1.2
   > Microsoft.AspNetCore.Mvc.Razor                                  2.1.2
   > Microsoft.AspNetCore.Mvc.Razor.Extensions                       2.1.1
   > Microsoft.AspNetCore.Mvc.RazorPages                             2.1.2
   > Microsoft.AspNetCore.Mvc.TagHelpers                             2.1.2
   > Microsoft.AspNetCore.Mvc.ViewFeatures                           2.1.2
   > Microsoft.AspNetCore.Razor                                      2.1.1
   > Microsoft.AspNetCore.Razor.Design                               2.1.1
   > Microsoft.AspNetCore.Razor.Language                             2.1.1
   > Microsoft.AspNetCore.Razor.Runtime                              2.1.1
   > Microsoft.AspNetCore.ResponseCaching.Abstractions               2.1.1
   > Microsoft.AspNetCore.Routing.Abstractions                       2.1.1
   > Microsoft.AspNetCore.Server.IISIntegration                      2.1.1
   > Microsoft.AspNetCore.Server.Kestrel                             2.1.3
   > Microsoft.AspNetCore.Server.Kestrel.Core                        2.1.3
   > Microsoft.AspNetCore.Server.Kestrel.Transport.Abstractions      2.1.3
   > Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets           2.1.3
   > Microsoft.AspNetCore.SignalR.Common                             1.0.3
   > Microsoft.AspNetCore.SignalR.Core                               1.0.3
   > Microsoft.AspNetCore.SignalR.Protocols.Json                     1.0.3
   > Microsoft.AspNetCore.WebUtilities                               2.1.1
   > Microsoft.CodeAnalysis.Analyzers                                1.1.0
   > Microsoft.CodeAnalysis.Common                                   2.8.0
   > Microsoft.CodeAnalysis.CSharp                                   2.8.0
   > Microsoft.CodeAnalysis.Razor                                    2.1.1
   > Microsoft.CSharp                                                4.5.0
   > Microsoft.DiaSymReader.Native                                   1.7.0

The problem with this type of output is that it is difficult to parse. npm provides two options that make this easier when listing packages. They include both a -parsable and a -json parameter. I think dotnet list package should provide something similar.

I think that dotnet list package should do something similar.

F:\universal-dashboard\src\UniversalDashboard>dotnet --version
2.2.102

I'm happy to work on a PR for this. I've found the responsible code and could implement a "parse-friendly" version but wanted feedback from the community\team before doing anything.

https://github.com/NuGet/NuGet.Client/blob/dev/src/NuGet.Core/NuGet.CommandLine.XPlat/Utility/ProjectPackagesPrintUtility.cs


Update 12/10/2021:
After diving into this issue, it's quite large issue, it looks none of dotnet command implemented it, this is 1st time. Let's divide and conquer this issue, here're sub issues:

Activity

  1. added this to the Backlog milestone on Feb 1, 2019
  2. added
    help wantedConsidered good issues for community contributions.
    on Feb 1, 2019
  3. nkolev92 commented on Feb 1, 2019

    @nkolev92
    Member

    Hey @adamdriscoll
    Thanks for opening this issue.
    It is something we considered initially, but wasn't added for V1.

    @rrelyea @anangaur to chip in on how they'd want this done.

    I personally prefer a json option.

  4. anangaur commented on Feb 1, 2019

    @anangaur

    Json seems fine as most of NuGet generated artifacts are json too.

    @adamdriscoll , If you could start with a one pager that includes the proposal on output i.e. the structure of the data and how sample output would look like with different options, that can be a good start. We can review the proposal and then suggest/make changes as required. Since this is not on our priority list, it would be difficult for us to propose a full spec on this in near term.

  5. ignatandrei commented on Mar 19, 2019

    @ignatandrei

    json will be good to have.

  6. xavierdecoster commented on Jul 18, 2019

    @xavierdecoster
    Contributor

    Having a --json option for dotnet list package would be very convenient indeed, especially when --outdated and --deprecated come into the picture. Making the output machine-readable unlocks additional tooling and automation scenarios.

    Based on today's (and planned) console output, here's a suggestion.
    Would love feedback/input and make sure we understand what data and format would be desired for this JSON output.

    outdated.json

    Command: dotnet list package --outdated --include-transitive --json

    {
      "outdated": [
        {
          "framework": "netcoreapp2.0",
          "topLevelPackages": [
            {
              "id": "NuGet.Common",
              "requestedVersion": "4.8.0-preview1.5146",
              "resolvedVersion": "4.8.0-preview1.5146",
              "latestVersion": "4.9.1-zlocal.52990"
            }
          ],
          "transitivePackages": [
            {
              "id": "NuGet.Frameworks",
              "requestedVersion": "4.8.0-preview3.5278",
              "latestVersion": "4.9.1-zlocal.52990"
            }
          ]
        }
      ]
    }
    

    deprecated.json

    Command: dotnet list package --deprecated --json

    {
      "deprecated": [
        {
          "framework": "netcoreapp2.0",
          "topLevelPackages": [
            {
              "id": "My.Legacy.Package",
              "resolvedVersion": "2.0.0",
              "reasons": ["Legacy"],
              "alternativePackage": {
                "id": "My.Awesome.Package",
                "versionRange": "[3.0.0,)"
              }
            },
            {
              "id": "My.Buggy.Package",
              "resolvedVersion": "1.1.0",
              "reasons": ["Critical Bugs"],
              "alternativePackage": {
                "id": "My.NotBuggy.Package",
                "versionRange": "*"
              }
            },
            {
              "id": "My.CompletelyBroken.Package",
              "resolvedVersion": "0.9.0",
              "reasons": ["Legacy", "Critical Bugs"],
              "alternativePackage": null
            }
          ],
          "transitivePackages": null
        }
      ]
    }
    

    installed-packages.json

    Command: dotnet list package --json

    This would be very similar to the outdated.json example above but simply listing all installed packages.

  7. anangaur commented on Jul 18, 2019

    @anangaur

    Should it be null or empty object list for no transitivePackages?

    For multi-targetting, there is a trade-off (size of the file) between listing the packages separately for each framework vs. having this info as an attribute for each package. I like the current approach better though.

  8. xavierdecoster commented on Jul 18, 2019

    @xavierdecoster
    Contributor

    Should it be null or empty object list for no transitivePackages?

    No strong opinion :)

    For multi-targetting, there is a trade-off (size of the file) between listing the packages separately for each framework vs. having this info as an attribute for each package. I like the current approach better though.

    The current model happens to correspond with the in-memory model used for console printing, which is rather convenient.

  9. anangaur commented on Jul 29, 2019

    @anangaur
  10. changed the title [-]dotnet list package is difficult to parse[/-] [+]Machine readable output for dotnet list package[/+] on Jul 29, 2019
  11. nkolev92 commented on Jul 29, 2019

    @nkolev92
    Member

    @anangaur @xavierdecoster Can you please put the spec in a PR to the https://github.com/NuGet/Client.Engineering repo?

    It's very difficult to leave comments on a spec that's in the wiki.

  12. 81 remaining items

  13. removed
    help wantedConsidered good issues for community contributions.
    on Sep 28, 2022
  14. dburriss commented on Nov 4, 2022

    @dburriss

    Awesome to see the progress on this @erdembayar !
    Will this be assigned to a milestone at any point? I have a usecase and I was going to parse the text but if it is feasible to wait for this to ship I would prefer that.

  15. added this to the 6.5 milestone on Nov 4, 2022
  16. erdembayar commented on Nov 4, 2022

    @erdembayar
    Contributor

    Awesome to see the progress on this @erdembayar Erick Yondon FTE ! Will this be assigned to a milestone at any point? I have a usecase and I was going to parse the text but if it is feasible to wait for this to ship I would prefer that.

    Thank you for your interest, I just merged the PR for json output, in that PR I refactored the coupled code to MVC style code so next time someone can add yaml,xml outputs with little effort.
    Most likely this would be in NuGet 6.5 (VS17.5), I still need to complete dotnet/sdk#28564 by next week.

  17. kaylumah commented on Nov 9, 2022

    @kaylumah

    Very cool :) hoop to see this soon :)

  18. erdembayar commented on Nov 30, 2022

    @erdembayar
    Contributor

    If anyone want to test out, it's already in public nightly build: https://github.com/dotnet/installer#table
    We'll update the doc soon: https://github.com/NuGet/docs.microsoft.com-nuget/issues/2922
    In case of there is any issue with generating json output then it will return 1 instead of 0 if you're using it for CI pipeline automation, so you can check what is going on.

    image

    image

    image

    image

    image

  19. erdembayar commented on Dec 1, 2022

    @erdembayar
    Contributor

    Alternatively, you can install latest binary from here .\dotnet-install.ps1 -Channel 7.0.2xx -Quality daily

  20. Postlagerkarte commented on Jan 12, 2023

    @Postlagerkarte

    @erdembayar Thanks for the great work! However, I am a bit confused. The documentation has been updated a few days ago (https://learn.microsoft.com/en-us/dotnet/core/tools/dotnet-list-package) - so I was assuming that this feature is ready to use in production. Sadly the latest .NET 7.0.2 from January 10, 2023 contains .NET SDK 7.0.102 and is missing the feature. I know that I can manually install the latest build but I want to use it within an Azure Pipeline. Can you advise?

  21. erdembayar commented on Jan 12, 2023

    @erdembayar
    Contributor

    @erdembayar Thanks for the great work! However, I am a bit confused. The documentation has been updated a few days ago (https://learn.microsoft.com/en-us/dotnet/core/tools/dotnet-list-package) - so I was assuming that this feature is ready to use in production. Sadly the latest .NET 7.0.2 from January 10, 2023 contains .NET SDK 7.0.102 and is missing the feature. I know that I can manually install the latest build but I want to use it within an Azure Pipeline. Can you advise?

    Here is the PR#32855 for doc change. I explicitly asked not to merge until it's GA, but someone merged it, considering it was for main branch not our live branch of docs, I didn't ask to revert.
    But it looks like they moved all changes from main to live 2 days ago even though not all new features are GA yet. I'll ask why it happened, by the way this change is in NET SDK 7.0.2xxx, not in 7.0.102.
    Sorry for the confusion, most likely we just revert the change until NET SDK 7.0.2xxx is GA.

  22. Postlagerkarte commented on Jan 12, 2023

    @Postlagerkarte

    Thanks for clarifying! Any idea when NET SDK 7.0.2xxx will be GA?

  23. erdembayar commented on Jan 12, 2023

    @erdembayar
    Contributor

    Thanks for clarifying! Any idea when NET SDK 7.0.2xxx will be GA?

    Sorry, I couldn't tell exactly, but looking at https://dotnet.microsoft.com/en-us/download/dotnet/7.0 release dates, it usually happens every 1 month, so I estimate sometime in Feb,2023.

  24. erdembayar commented on Feb 18, 2023

    @erdembayar
    Contributor

    Thanks for clarifying! Any idea when NET SDK 7.0.2xxx will be GA?

    Sorry, I couldn't tell exactly, but looking at https://dotnet.microsoft.com/en-us/download/dotnet/7.0 release dates, it usually happens every 1 month, so I estimate sometime in Feb,2023.

    This feature is GA now with .NET SDK 7.0.200 🥳, please download from https://dotnet.microsoft.com/en-us/download/dotnet/7.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions