Repository navigation
feat: enable template histogram creation and collection to accept list of tasks - #421
alexander-held wants to merge 2 commits into
Conversation
| TemplateHistogramInformation = Tuple[ | ||
| Dict[str, Any], Dict[str, Any], Dict[str, Any], Optional[Literal["Up", "Down"]] | ||
| ] |
There was a problem hiding this comment.
This probably would be good to replace by a lightweight class to wrap the information.
1d83bf7 to
fd93dbb
Compare
1d29183 to
2f75c8a
Compare
2f75c8a to
5536dd8
Compare
|
This will require a different approach to handle remote workers without access to a shared filesystem. To handle this, histograms would have to be passed back and saved at the head node. The current API does not surface the call to Currently to-do / think about:
|
This updates the
templates.build,templates.collect, andtemplates.postprocessimplementations to accept a list of templates to produce via a newtemplate_listkeyword argument. The full list can be obtained viaroute.required_templates. By default, all templates will still be processed.This allows users to parallelize the template histogram production / collection, partially addressing #401. An example of this would be the following:
compared to the non-parallelized version, which is unchanged:
breaking change:
route.apply_to_all_templatestoroute.apply_to_templatesand changed signature: now takes a list of tasks, which can be generated byroute.required_templates