Repository navigation
Rejected transitive central dependencies should be removed from the restore graph - #3374
Conversation
| }; | ||
| } | ||
|
|
||
| private static bool TryResolveConflictsForCPVM<TItem>(this GraphNode<TItem> root, List<VersionConflictResult<TItem>> versionConflicts) |
There was a problem hiding this comment.
Doing a diff between TryResolveConflictsForCPVM and TryResolveConflicts, there are only 2 differences:
-
ForCPVMversion calls an overload ofForEachwith an extra skipNode parameter. However, theForEachoverload withskipNodecalls the overload withskipNodewith anullvalue. Therefore, if this method had theskipNodefunc as a method parameter, then both the CPVM and non-CPVM codepaths could use the same codepath. -
CPVM version calls
RejectCentralTransitiveBecauseOfRejectedParents.RejectCentralTransitiveBecauseOfRejectedParentslooks like a no-op for non-CPVM graphs. Alternatively we chould guard the method call in anifstatement.
It looks to me like we can get rid of this new method, which I think is important because it implements business logic, (not just trivial code), so this type of code duplication is real technical debt that can lead to bugs if one of the two implementations change in the future.
There was a problem hiding this comment.
I would let the two methods different. In my opinion it is easier to follow the logic in this way. It will also reduce the regression risks to existent path.
There was a problem hiding this comment.
Long term, I'm with the @zivkan.
While it does make it easier to follow given the context we have today, have 2 different code paths with duplication would just amplify the confusion for a new reader.
| }; | ||
| } | ||
|
|
||
| private static bool TryResolveConflictsForCPVM<TItem>(this GraphNode<TItem> root, List<VersionConflictResult<TItem>> versionConflicts) |
There was a problem hiding this comment.
Long term, I'm with the @zivkan.
While it does make it easier to follow given the context we have today, have 2 different code paths with duplication would just amplify the confusion for a new reader.
|
Do we have any performance numbers? I see a couple of things that stand out to me but it would be nice to have some microbenchmarks for what we consider best and worse case so we can compare before and after this change. |
Talked with @nkolev92 offline regarding the comment
The code is behaving as expected. @davidfowl please let me know if there are additional suggestions that I need to look into? |
zivkan
left a comment
There was a problem hiding this comment.
You'll need to rebase your branch onto the latest dev to get the build to work. MicroBuild's new CI agents have an issue that doesn't work with the VS account management assembly, so we had to skip those tests for the time being.
| /// <summary> | ||
| /// Used in case that a node is removed from its outernode and needs to keep reference of its parents. | ||
| /// </summary> | ||
| internal BlockingCollection<GraphNode<TItem>> ParentNodes { get; } |
There was a problem hiding this comment.
Why are we using a blocking collection? Is it solely because of IsAddingCompleted?
There was a problem hiding this comment.
Yes, it was because the IsAddingCompleted. However this was an extra and probably not necessary caution because the iterations of the collections is executed after all the parents are added. Running performance tests iterating over BlockingCollection is more expensive so changed to IList.
| if (!n.ParentNodes.Any(p => !(tracker.IsDisputed(p.Item) || tracker.IsAmbiguous(p.Item)))) | ||
| { | ||
| tracker.MarkAmbiguous(n.Item); | ||
| n.ForEach(x => tracker.MarkAmbiguous(x.Item)); |
There was a problem hiding this comment.
Why do you need to do this explicit walk of child nodes here? Can you do this in a single pass without the nested ForEach by moving the predicate?
There was a problem hiding this comment.
It is a single pass over each of the central transitive nodes. I added a predicate to skip any nodes that are already marked as Ambiguous, I am not sure if I addressed your concerns though.
| }); | ||
| } | ||
|
|
||
| private static void RejectCentralTransitiveBecauseOfRejectedParents<TItem>(this GraphNode<TItem> root) |
There was a problem hiding this comment.
This looks really expensive to run on each node both from a big O perspective and number of allocations perspective. That's why I was asking about performance tests. The ASP.NET Core repo has > 100 central dependencies, does this perform well at that scale?
There was a problem hiding this comment.
Yes the big O can be large depending on the graph structure. I did run benchmarks for rejects/acceptances scenarios. The results are below (graph used has 2,047 nodes and a subset of these are central transitive). I will still keep thinking of solutions to reduce it.
The total central dependencies count is important, for the CPVM scenarios the count of the subset that becomes transitive dependencies is more important.
| Name | Time milliseconds |
|---|---|
| 2_10_2_AnalyseGraphWith4AcceptedCentralTransitiveDeps | 2 |
| 2_10_2_AnalyseGraphWith4RejectedCentralTransitiveDependencies | 3.56 |
| 2_10_2_AnalyseGraphWithoutCentralTransitiveDepsAsync_HalfGraphRejected | 2.66 |
| 2_10_4_AnalyseGraphWith16AcceptedCentralTransitiveDeps | 2.61 |
| 2_10_4_AnalyseGraphWith16RejectedCentralTransitiveDependencies | 4.06 |
| 2_10_4_AnalyseGraphWithoutCentralTransitiveDepsAsync_HalfGraphRejected | 2.82 |
| 2_10_6_AnalyseGraphWith64AcceptedCentralTransitiveDeps | 3.09 |
| 2_10_6_AnalyseGraphWith64RejectedCentralTransitiveDependencies | 4.31 |
| 2_10_6_AnalyseGraphWithoutCentralTransitiveDepsAsync_HalfGraphRejected | 3.09 |
|
I would've wanted to review this first before merging. There had been many changes since I was last requested for review, so I wasn't sure when to take a look. The code now looks significantly different from when it was approved. |
|
@cristinamanum please file an issue for the potential performance problems with this |
Added NuGet/Home#9740 |
The changes performed were around trying to get the best performance that I could based on benchmarks data. Please let me know additional feedback and I will be happy to address it. |
Bug
Fixes issue: Rejected transitive central dependencies should be removed from the restore graph.
A visualization of the problem:
Project
-> PackageA 1.0.0 -> PackageB 1.0.0 -> CentralPackageC 1.0.0 ->...
-> PackageD 2.0.0 -> PackageB 2.0.0
During the graph resolution PackageB 1.0.0 becomes Rejected and will be dropped from the graph in the favor of PackageB 2.0.0. In this context the CentralPackageC 1.0.0 should not be in the graph anymore. The flatten graph should include only PackageA 1.0.0, PackageD 2.0.0, PackageB 2.0.0
Without this fix the flatten graph is:
PackageA 1.0.0, PackageD 2.0.0, PackageB 2.0.0, CentralPackageC 1.0.0
Fix
Details:
The nodes added as central dependency nodes keep reference of their parents.
In the case of opted in central package versions projects, the restore graph is analyzed :
Testing/Validation
Tests Added: Yes
Reason for not adding tests:
Validation: Unit/functional test and manual testing