Repository navigation
[RFC] Namespace for Optimization Specific Annotation #2391
Description
As we start to get some optimization specific operators to annotate additional information needed by relay. Here is a rough list that I am aware of:
- on_device, annotate which device we should run computation on
- stop_gradient, stop gradient in AD
- stop_fusion, stop to fuse continuous ops, useful for accelerator ware partition
- simulated_quantize, used for intermediate annotation for quantization.
These are not conventional operators like numpy-style or normal nn operator, but are helpful to store additional information in the IR. It would be great to categorize them into the same namespace and provide a unified guideline in terms of support-levels.
Here are some proposals, please feel free to make additional names:
- relay.opt.on_device (opt stands for optimization)
- relay.annotation.on_device (annotation)
- relay.ann.on_device (ann for annotation, alternatively "announcement" :)
As usual, API design is one of the most crucial parts of the development and I would like to invite everyone in the community to participate in this RFC. Let us aim to reach a consensus in a week.
Guideline choices for support level:
- 9, allocate a specific level for annotation
- 10, currently used for experimental
cc @dmlc/tvm-comitter
Activity
zhiics commentedon Jan 8, 2019
@tqchen Thanks for the RFC. I personally would prefer relay.annotation.on_device since some of the passes are probably not for optimization but really for annotation purpose.
As for support level, I would vote for using the separate level 9 for annotation.
ZihengJiang commentedon Jan 9, 2019
Prefer
relay.annotation.on_devicejroesch commentedon Jan 9, 2019
For these kind of ops I think
annotationmakes the most sense, we can always choose to further categorize them in the future as we gain more of them. It seems like simulated_quantize belongs else where though? is it really an annotation in the same way that the device ops are?I do like
optas well, but maybe that is the namespace optimizations should live in.zhiics commentedon Jan 10, 2019
@tqchen @jroesch @ZihengJiang
So I think we will go for opening a new
annotationnamespace.As for the support level, what do you guys think?
ZihengJiang commentedon Jan 10, 2019
vote for 10, since it is experimental so we can change it to 9 after it is stable
tqchen commentedon Jan 10, 2019
The current proposal seems to converge toward
annotation, let me suggest that we start with level 10 and move to level 9 once we think things are stable.Let us leave this RFC open for a week so others have time to take a look and share their thoughts, in the meanwhile, we can proceed with changes in the PR accordingly.
zhiics commentedon Jan 10, 2019
No problem. I will then go ahead with
annotationbut keep level 10 for now.tqchen commentedon Jan 15, 2019
Thanks, everyone for the helpful discussions. The current consensus is:
annotationas namespace@ZihengJiang please also update your PR of quantization accordingly(annotate.simulated_quantize)