Skip to content

Test failure: System.Diagnostics.Tests.ProcessTests/TestCheckChildProcessUserAndGroupIds #28922

Closed
#46302
Closed
Test failure: System.Diagnostics.Tests.ProcessTests/TestCheckChildProcessUserAndGroupIds#28922
#46302
Assignees
adamsitnik
Labels
area-System.Diagnostics.Processdisabled-testThe test is disabled in source code against the issuetest-bugProblem in test source code (most likely)
Milestone
6.0.0

Description

@ghost
ghost
opened on Mar 11, 2019

Opened on behalf of @wfurt

The test System.Diagnostics.Tests.ProcessTests/TestCheckChildProcessUserAndGroupIds has failed.

Failure Message:

System.Diagnostics.RemoteExecutorTestBase+RemoteInvokeHandle+RemoteExecutionException : Remote process failed with an unhandled exception.

Stack Trace:


Child exception:
  Xunit.Sdk.XunitException: Expected: 4, 20, 24, 25, 27, 29, 30, 44, 46, 109, 110, 998, 1000
Actual: 4, 20, 24, 25, 27, 29, 30, 44, 46, 109, 110, 1000
   at System.AssertExtensions.Equal[T](HashSet`1 expected, HashSet`1 actual) in /__w/1/s/src/CoreFx.Private.TestUtilities/src/System/AssertExtensions.cs:line 349
   at System.Diagnostics.Tests.ProcessTests.CheckUserAndGroupIds(String userId, String groupId, String groupIdsJoined, String checkGroupsExact) in /__w/1/s/src/System.Diagnostics.Process/tests/ProcessTests.Unix.cs:line 515

Child process:
  System.Diagnostics.Process.Tests, Version=4.2.1.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51 System.Diagnostics.Tests.ProcessTests Int32 CheckUserAndGroupIds(System.String, System.String, System.String, System.String)

Child arguments:
  1000, 1000, 4,20,24,25,27,29,30,44,46,109,110,998,1000, True

Build : 3.0 - 20190310.5 (Core Tests)
Failing configurations:

  • Ubuntu.1604.Amd64-x64
    • Release

Activity

  1. wfurt commented on Mar 11, 2019

    @wfurt
    wfurt
    on Mar 11, 2019
    Member

    cc: @tmds It seems like 998 is missing for some reason. That looks different from the duplicates.

  2. stephentoub commented on Mar 11, 2019

    @stephentoub
    stephentoub
    on Mar 11, 2019
    Member

    That looks different from the duplicates.

    I believe this is the first time we've seen it fail again since I added the extra diagnostics to output the full contents of both sets.

  3. stephentoub commented on Mar 11, 2019

    @stephentoub
    stephentoub
    on Mar 11, 2019
    Member

    cc: @danmosemsft

  4. danmoseley commented on Mar 11, 2019

    @danmoseley
    danmoseley
    on Mar 11, 2019
    Contributor

    In a build the previous day, identical result:

    https://mc.dot.net/#/product/netcore/30/source/official~2Fdotnet~2Fcorefx~2Frefs~2Fheads~2Fmaster/type/test~2Ffunctional~2Fcli~2F/build/20190309.4/workItem/System.Diagnostics.Process.Tests/analysis/xunit/System.Diagnostics.Tests.ProcessTests~2FTestCheckChildProcessUserAndGroupIds

    Xunit.Sdk.XunitException: Expected: 4, 20, 24, 25, 27, 29, 30, 44, 46, 109, 110, 998, 1000
                                              Actual: 4, 20, 24, 25, 27, 29, 30, 44, 46, 109, 110, 1000

    So what is the mysterious 998

  5. wfurt commented on Mar 11, 2019

    @wfurt
    wfurt
    on Mar 11, 2019
    Member

    I can get repro machine tomorrow and can check if I can make it fail again.

  6. tmds commented on Mar 11, 2019

    @tmds
    tmds
    on Mar 11, 2019
    Member

    Thanks @wfurt .

    The list that contains 998 is determined by calling id -G <username>.
    The child process will launch successfully when the actual groups is a subset of the requested groups. The test is more strict and requires an exact match (checkGroupsExact is true on Linux):

    https://github.com/dotnet/corefx/blob/fd0911e27f6c6f958f213bac8b3fb99b7700167b/src/System.Diagnostics.Process/tests/ProcessTests.Unix.cs#L513-L520

  7. stephentoub commented on Mar 11, 2019

    @stephentoub
    stephentoub
    on Mar 11, 2019
    Member

    Again here:
    https://mc.dot.net/#/user/dotnet-bot/pr~2Fdotnet~2Fcorefx~2Frefs~2Fpull~2F35946~2Fmerge/test~2Ffunctional~2Fcli~2F/20190311.6/workItem/System.Diagnostics.Process.Tests/analysis/xunit/System.Diagnostics.Tests.ProcessTests~2FTestCheckChildProcessUserAndGroupIds

  8. wfurt commented on Mar 11, 2019

    @wfurt
    wfurt
    on Mar 11, 2019
    Member

    the mysterious 998 is 'docker'. So far I was not able to reproduce the failure using repro machine. At last, the groups and OS should be identical to official machines. The test output is sorted but in 'id -G' output the missing group is last.

    helixbot@a001DOJ:/mnt/workspace/Work/d5bc6e62-4804-4bcd-a225-9386663e3c88/Exec$ id -G
1000 4 20 24 25 27 29 30 44 46 109 110 998
helixbot@a001DOJ:/mnt/workspace/Work/d5bc6e62-4804-4bcd-a225-9386663e3c88/Exec$ grep 998 /etc/group
docker:x:998:helixbot

  9. tmds commented on Mar 11, 2019

    @tmds
    tmds
    on Mar 11, 2019
    Member

    I'm also part of a docker group (981 on my machine).
    What does getgroups return when you strace the groups command?

    $ id -G tmds
1000 10 981 1001
$ strace -e getgroups groups 
getgroups(0, NULL)                      = 4
getgroups(4, [10, 981, 1000, 1001])     = 4

  10. wfurt commented on Mar 11, 2019

    @wfurt
    wfurt
    on Mar 11, 2019
    Member

    I did 100+ rounds of the test and I did not see any failure so far.

    getgroups(0, NULL)                      = 13
getgroups(13, [4, 20, 24, 25, 27, 29, 30, 44, 46, 109, 110, 998, 1000]) = 13

  11. tmds commented on Mar 11, 2019

    @tmds
    tmds
    on Mar 11, 2019
    Member

    One thing that may be causing this is the helixbot user was added to the docker group, but the process that is launching the tests isn't aware of this. This may happen if that process was already running before the helixbot user was added to the docker group. Is this something that could occur in the CI environment?

  12. danmoseley commented on Mar 11, 2019

    @danmoseley
    danmoseley
    on Mar 11, 2019
    Contributor

    @MattGal do you know the answer to @tmds question here?

  13. MattGal commented on Mar 11, 2019

    @MattGal
    MattGal
    on Mar 11, 2019
    Member

    I don't think that'd be the case, but I will investigate after 4 PM or so PST.

  14. wfurt commented on Mar 12, 2019

    @wfurt
    wfurt
    on Mar 12, 2019
    Member

    I did ~ 2000 runs of whole test set on repro machine and I did not get single failure.
    I don't know if helixbot user is part of base image @MattGal but if not that would be place to look.
    To generalize on @tmds comment, adding user to group will not be visible unless you login again. (or re-parse /etc/groups

    furt@net-dale:/tmp$ ./test
gr = 6
furt@net-dale:/tmp$ sudo usermod -a -G foo furt
furt@net-dale:/tmp$ ./test
gr = 6

<login/logout>

furt@net-dale:/tmp$ ./test
gr = 7

    test simply primps number of groups user is in printf("gr = %d\n",getgroups(0, NULL));

  15. MattGal commented on Mar 12, 2019

    @MattGal
    MattGal
    on Mar 12, 2019
    Member

    I'm trying to chat with @wfurt offline but for the queue mentioned by @stephentoub while docker is on the machine in question, I see no evidence that the work item was attempted to be run inside a docker container. Machines in Ubuntu.1604.Amd64.Open are just plain Azure D2_V3 vms. Still looking...

  16. 6 remaining items

  17. wtgodbe commented on Jun 25, 2019

    @wtgodbe
    wtgodbe
    on Jun 25, 2019
    Member

    The test is actually disabled: dotnet/corefx#35949. May be the same issue as https://github.com/dotnet/corefx/issues/38833

  18. msftgits
    transferred this issue fromdotnet/corefxon Feb 1, 2020
  19. msftgits
    added this to the Future milestone on Feb 1, 2020
  20. maryamariyan
    added
    untriagedNew issue has not been triaged by the area owner
    on Feb 23, 2020
  21. adamsitnik
    removed
    untriagedNew issue has not been triaged by the area owner
    on Jul 6, 2020
  22. adamsitnik
    added
    test-bugProblem in test source code (most likely)
    disabled-testThe test is disabled in source code against the issue
    on Dec 16, 2020
  23. ghost added
    in-prThere is an active PR which will close this issue when it is merged
    on Dec 16, 2020
  24. adamsitnik
    self-assigned this
    on Dec 16, 2020
  25. adamsitnik
    modified the milestones: Future, 6.0.0 on Dec 16, 2020

  26. adamsitnik commented on Dec 18, 2020

    @adamsitnik
    adamsitnik
    on Dec 18, 2020
    Member

    I was able to reproduce the failure using our CI in #46138

    I've verified that it's caused by the difference in what id -G (what we use for obtaining the expected value) and getgroups (what we use for getting the actual value) return even without starting any new processes:

    eba057f

    https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-46138-merge-21aa167a19b94858a8/System.Diagnostics.Process.Tests/console.aa0d1c27.log?sv=2019-07-07&se=2021-01-07T10%3A48%3A30Z&sr=c&sp=rl&sig=yOZ94Wil3yQaONO%2FVo4b293jCZUZ4jwNFtdVd7K33pY%3D

        System.Diagnostics.Tests.ProcessTests.TestCheckChildProcessUserAndGroupIds [FAIL]
      id -G returned: 4
      getgroups returned: 
      Stack Trace:
        /_/src/libraries/System.Diagnostics.Process/tests/ProcessTests.Unix.cs(557,0): at System.Diagnostics.Tests.ProcessTests.TestCheckChildProcessUserAndGroupIds()
  27. adamsitnik
    closed this as completedin #46302on Dec 24, 2020
  28. ghost removed
    in-prThere is an active PR which will close this issue when it is merged
    on Dec 24, 2020
  29. ghost locked as resolved and limited conversation to collaborators on Jan 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area-System.Diagnostics.Processdisabled-testThe test is disabled in source code against the issuetest-bugProblem in test source code (most likely)

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions