TVM 0.5 Release Note

@ZihengJiang

The release is managed by the TVM PMC, moderators for this release are @ZihengJiang and @yzhliu

The community welcomes new reviewers @nishi-t @were @siju-samuel @jroesch @xqdan @zhiics @grwlf @ajtulloch @vinx13 @junrushao1994 @FrozenGene @liangfu , new committers @srkreddy1238 @eqy @masahi @nhynes @phisiart @merrymercy @Laurawly @adityaatluri @Huyuwei

Change List

New Features

Fully featured 8-bit network support
- 8bit quantizer
- Arbitrary bits quantization algorithm
- Intel cpu support
NVidia GPU 8-bit kernel
- int8 gemm recipe
- int8 conv2d
- Autotvm integration
Automated tuning and scheduling
- AutoTVM optimizations for mobile GPUs
- AutoTVM optimizations for CUDA
- AutoTVM optimizations for x86
Initial release of the differentiable programming IR, Relay
- Generic & informative Relay error reporting [Relay] Add generic & informative Relay error reporting #2408
- Relay IR text format support [Relay][RFC] Relay IR Text Format #1781
- Support control flows
- A Normal Form Canonicalization [Relay] A Normal Form Canonicalization #2251
- Type system support
- End to end compilation
  - Frontend support: Caffe2 [Relay][Frontend] Caffe2 Support #2507 , CoreML [Relay][Frontend] CoreML Support #2476 , Keras [Relay][Keras] force const dtype to be float32 #2376 , MXNet [RELAY][FRONTEND] Initial MXNet frontend support. #2163 , ONNX, TFLite [TFLite] Support TFLite FP32 Relay frontend. #2365
  - Operator coverage [RELAY][OP] Relay Operator Sprint #1799 [RELAY][OP] Relay Op Sprint (Part 2) #2051
- FoldScaleAxis [RELAY][PASS] FoldScaleAxis Forward #2020
- SimplifyInference [RELAY/PASS] Simplify inference. #2033
- CombineParallelConv2D [RELAY][PASS] CombineParallelConv2D #2089
- InstrumentBoundCheckers pass [PASS] InstrumentBoundCheckers pass #2079
- Bind & FoldConstant [RELAY][PASS] Bind, FoldConstant #2100
- Alter Op Layout [Relay] Alter Op Layout #2150
- General OpFusion [RELAY][PASS] General OpFusion. #2090
CodeGen
- Gcc / g++ compatible C code generator for TVM [BACKEND][CODEGEN] C codegen with tests #2161
- Device type annotation for heterogeneous compilation [relay][pass] Annotation for heterogeneous compilation #2361
- Cache packed func ptr, lift alloca [CODEGEN][LLVM] Cache packed func ptr, lift alloca #2070
- Generalize compute to tensor region [LANG] Generalize compute to tensor region #1476
Runtime
- Relay interpreter and compiler [RELAY][COMPILER] Initial Relay interpreter and compiler for TVM runtime system. #1954
- Heterogeneous runtime Heterogeneous Runtime #1695
- Language bindings: Golang runtime [RUNTIME][GOLANG] TVM runtime for golang v0.1 #1470 , Rust runtime Add rust runtime #1597
- Add min_repeat_ms to time_evaluator [RUNTIME] Add min_repeat_ms to time_evaluator #2200
- Bundled interpreter demonstration [Apps] [RFC] Bundled interpreter demonstration #2297
- Enable PlanMemory in the graph runtime [RELAY][BACKEND] Enable PlanMemory in the graph runtime. #2120
Language Binding
- Rust frontend [RUST][FRONTEND] Add rust frontend v0.1 #2292
VTA
- Improved RPC for VTA [VTA] Improved RPC for VTA #2043
Hybrid python programming model
- Support for scheduling [Hybrid Script] Supporting scheduling hybrid script #2416
- Support for Inter-function call [Hybrid Script] Inter-function call supported! #2287
- Backend support [Hybrid script] Backend support #2477
TOP
- Initial support for sparse tensor computation
- Improve ARM CPU depthwise convolution performance [ARM][Performance] Improve ARM CPU depthwise convolution performance #2345
- Port winograd ops to relay [RELAY] Port winograd ops to relay #2356
Tutorials and docs
- Relay language docs [Relay][Docs] Relay Language Reference #2232
- Tutorials on how to use SGX backend
- How to write a pass in python
- General lowering flow of TVM
- How to do tensorize
- TFLite frontend tutorial [Doc] TFLite frontend tutorial #2508
- Keras seq2seq model for translation tutorial [Tutorial]NLP Sequence to sequence model for translation #1815
- Committer guide and tips [DOCS][COMMUNITY] Committer guide and tips #2468
- Code review guideline on API designs [DOCS][COMMUNITY] Improve code review guideline on API designs #2459

Bug-fixes

Fix bug of ignoring user-defined shape in Tensorflow frontend. [NNVM][TENSORFLOW] bugfix. #2444
Prevent generating 0-dim tensors in Tensorflow frontend. [FRONTEND][TENSORFLOW] Bugfix #2326
Fix the bug of incorrect inferred layout when registered alter_layout function returns None. [Bugfix] Recover original layout when alter_layout function return None #2101
Fix the return type of read character from a buffer. [BUGFIX] GetChar() in base64.h should return int, not char #2255
Fix subgraph loading from JSON. Fix load subgraph from json #1980
Fix incorrect value index in hybrid script. [BUGFIX] [Hybrid Script] fix in-correct value index in hybrid script #2268
Fix out-of-bound access when buffer.access_ptr contains non-zero extent. Fix non-zero extent of access_ptr out of range #1939
Add checker for vectorizing over reduce-axis. Vectorizing a reduce_axis does not raise an error #1795
Bug fix for fusing injective operation and conv2d. [TOPI, CUDA] Bug fix properly bind gpu threads to injective op #1603 [NNVM] Bug fix Prevent fusing convolution with injective op #1608 [NNVM] Bugfix operator fusion for residual block with layout transform #1760 [NNVM] Bug fix Add missing check when deciding conv and injective ops are in the same group #1622
Fix bilinear mode in topi::nn::upsampling. Bugfix #1692. Constant folding and result comparision allowance (atol) #1708
Fix the bug with which the Else statement disappears after inject virtual thread pass. Fix a bug in inject-virtual-thread #2039
Preventing negation of an undefined Expr in some cases. [TVM] Fix negating undefined in DetectLinearEquation #1816
Fix CUDA path lookup when multiple CUDA versions are installed. [BUG] Fix incorrect libcuda.so found by cmake when multiple versions of CUDA exist #1788
Fix bugs for SSD sorting and multbox detection. Fixed bugs for SSD sorting and multbox detection #1578
Fix incorrect elem_offset calculation in Buffer. fix buffer elem_offset calculation #1762
Fix rewrite_unsafe_select pass so that LLVM CodeGen generates br instruction only when the type is i1. [LLVM CodeGen] Solve LLVM CodeGen br instruction accept not-i1 type issue #2381
Fix incorrect intermediate results in debug mode. [BugFix] Copy intermediate result in debug runtime #2520
Fix for SSD full GPU support and SSD tutorial update. [BugFix] SSD fully supported on GPUs, updated deploy_ssd tutorial #2510
Fix LLVM CodeGen br instruction accept not-i1 type issue. [LLVM CodeGen] Solve LLVM CodeGen br instruction accept not-i1 type issue #2381
Fixed constant folding for mod operation in CanonicalSimplify. [arith] fixed constant folding for mod operation in CanonicalSimplify #2487
Fix missing reduction init predicates. missing predicates in split loops #2249
Fix for quantize. [BUGFIX] Fix for quantize. #2573
fix storage_rewrite bug when input is big. [TVM][Bugfix] fix storage_rewrite bug when input is big #2580

Contributors

Code reviewers

@tqchen
@liangfu quantization, relay, topi, frontend
@zhiics relay, runtime, frontend
@nhynes quantization, rust
@Huyuwei frontend
@yzhliu relay, frontend, perf
@xqdan hybrid script, tvm/lang
@ZihengJiang relay
@vinx13 relay/pass, topi
@masahi relay/pass, frontend, doc, topi
@grwlf frontend, topi, relay, quantization
@tmoreau89 vta, relay, backend, runtime
@kazum frontend
@nishi-t frontend, topi
@PariksheetPinjari909 frontend
@jroesch relay, frontend, doc
@srkreddy1238 relay/op, frontend
@siju-samuel relay/op, frontend
@junrushao1994 relay
@icemelon9 relay, perf, tvm/lang, codegen
@ajtulloch relay, frontend
@alex-weaver relay
@kevinthesun hybrid script, topi, relay
@Laurawly topi
@were hybrid script, topi
@FrozenGene frontend, topi, relay/pass
@eqy relay, topi, runtime, rust
@zhreshold frontend, relay/op
@merrymercy relay/op, topi, runtime, frontend
@derisavi-huawei symbolic integers

Code contributions

@tqchen tvm
@vinx13 relay/pass, topi
@siju-samuel topi, relay/op
@merrymercy autotvm, topi, relay/pass
@srkreddy1238 relay/op, frontend/tf
@MarisaKirisame relay
@slyubomirsky relay, docs
@jroesch relay
@nhynes rust
@wweic docs, relay/pass
@yzhliu perf, frontend
@zhiics relay/pass, relay/op, runtime
@were hybrid script
@icemelon9 perf, relay/pass, relay/op
@joshpoll relay, docs
@sgrechanik-h codegen
@kazum frontend/keras, topi
@masahi relay/op, docs
@FrozenGene perf, frontend/tf
@liangdzou docs
@junrushao1994 relay/op
@eqy autotvm, runtime
@apivovarov docs
@ajtulloch runtime, nnpack
@kevinthesun relay/op, perf
@ZihengJiang relay/pass, quantization
@hlu1 nnpack, frontend/caffe2
@lixiaoquan nnvm
@imorinaga frontend/mxnet
@liangfu topi, docs
@xqdan codegen
@PariksheetPinjari909 frontend/darknet
@alexeyr frontend/tensorflow
@Rasterer topi
@yangchen-MS codegen
@anijain2305 relay/op
@grwlf topi
@Huyuwei topi, frontend/keras
@denis0x0D runtime/trace, relay/pass
@mutinifni codegen
@derisavi relay/pass
@tmoreau89 vta
@Laurawly topi, perf
@zhreshold frontend, topi
@kun-zh codegen
@reminisce relay/op
@ehsanmok rust
@cnuernber perf
@cowanmeg topi, codegen
@yuruofeifei topi

Thanks to everyone who have pushed to last release cycle in the past months. We would like to propose the release of v0.5 on Jan 28th.

We encourage everyone in the community to put their weights to review and vote the release. @dmlc/tvm-team

Please reply this thread on

Things that we missed in the release note
Bugfixes that need to be included in this release

We have brought the initial support for sparse tensor computation in TOPI (see #1289), though it would be migrated to use hybrid script instead of IR builder in the future.

Hoping to make the rust frontend ready for this release.

We have new frontend TFLite support (see #2365 ).

@ehsanmok

Pending PR for this release:

Required: #2507 #2476
Good to have: #2508 #2493 #2510 #2292

@ehsanmok as we're approaching the due, do we have chance to get the rust frontend in?

BugFix:
[LLVM CodeGen] Solve LLVM CodeGen br instruction accept not-i1 type issue #2381 . It could solve conv2d+ PRelu crash issue on ARM CPU. We have met two threads report this issue on the forum: https://discuss.tvm.ai/t/compile-failure-after-successfull-auto-tune/1561 and https://discuss.tvm.ai/t/debug-failed-to-export-library-with-llvm-on-arm-cpu-when-using-prelu/1006/4

@yzhliu

@yzhliu it should be ready if the last issue with CI build resolves quickly and there's no other requested changes which seems there's none.
cc: @nhynes

@yzhliu

@yzhliu the rust frontend is almost ready. It'll probably take a few days of concerted effort to get it ready to merge, but I think it'll make it in time.

@ZihengJiang

@ZihengJiang Bug fix for SSD full GPU support and SSD tutorial update #2510 could be added to the list.

Thanks to everyone who have pushed to last release cycle in the past months. We would like to propose the release of v0.5 on Jan 28th.

We encourage everyone in the community to put their weights to review and vote the release. @dmlc/tvm-team

Please reply this thread on

Things that we missed in the release note

Bugfixes that need to be included in this release

@Laurawly

@Laurawly @FrozenGene added. Given the current status of the rust frontend, I will add it to the 0.5 release announcement. @nhynes @ehsanmok

@ZihengJiang

@ZihengJiang The link of TFLite is refered to CoreML, correct link should be #2365

@FrozenGene

@FrozenGene Thanks, my bad. Fixed.

#2477 Can you also add this PR?

@ZihengJiang

@ZihengJiang @yzhliu would you mind keeping us updated about the release? we've long passed the voting deadline but it's still open!

@ehsanmok

Hey @ehsanmok , sorry about this. We are working on the release #2604. One thing is that it would be better to have several bug-fix PRs in the 0.5 release like #2495 #2573 #2580 , which will lead to dismatch with the voted commit number. We are discussing whether we should have another round of vote. @tqchen @yzhliu

I would suggest to start another voting thread and keep that open for two days

v0.5 has been tagged https://github.com/dmlc/tvm/releases/tag/v0.5

0.6 roadmap is available at #2623

Navigation Menu

Repository navigation

TVM 0.5 Release Note #2448

Change List

New Features

Bug-fixes

Contributors

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

TVM 0.5 Release Note #2448

Description

Change List

New Features

Bug-fixes

Contributors

Activity

ZihengJiang commented on Jan 17, 2019

liangfu commented on Jan 18, 2019

ehsanmok commented on Jan 18, 2019

FrozenGene commented on Jan 24, 2019

yzhliu commented on Jan 28, 2019

FrozenGene commented on Jan 28, 2019

ehsanmok commented on Jan 28, 2019

nhynes commented on Jan 28, 2019

Laurawly commented on Feb 1, 2019

ZihengJiang commented on Feb 2, 2019

FrozenGene commented on Feb 2, 2019

yzhliu commented on Feb 2, 2019

were commented on Feb 14, 2019

ehsanmok commented on Feb 15, 2019

ZihengJiang commented on Feb 15, 2019

tqchen commented on Feb 16, 2019

ZihengJiang commented on Feb 19, 2019

ZihengJiang commented on Feb 19, 2019

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions