Skip to content

[Mobile package] Update required operator config with additional ops for wav2vec2. #8079

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Edward Chen (edgchen1) merged 2 commits into from
Jun 17, 2021
Merged

[Mobile package] Update required operator config with additional ops for wav2vec2. #8079

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3de3fc0
[Mobile package] Update required operator config with additional ops …
edgchen1 Jun 17, 2021
66ee793
Add comment about LayerNormalization registration.
edgchen1 Jun 17, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions tools/ci_build/github/android/mobile_package.required_operators.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ ai.onnx;12;Abs,Add,And,ArgMax,ArgMin,AveragePool,Cast,Ceil,Clip,Concat,ConstantO
ai.onnx;13;Abs,Add,And,ArgMax,ArgMin,AveragePool,Cast,Ceil,Clip,Concat,ConstantOfShape,Conv,ConvTranspose,Cos,CumSum,DepthToSpace,DequantizeLinear,Div,DynamicQuantizeLinear,Elu,Equal,Exp,Expand,Flatten,Floor,Gather,GatherND,Gemm,Greater,GreaterOrEqual,Identity,If,LRN,LeakyRelu,Less,LessOrEqual,Log,LogSoftmax,Loop,MatMul,Max,MaxPool,Mean,Min,Mul,Neg,NonMaxSuppression,NonZero,Not,Or,PRelu,Pad,Pow,QuantizeLinear,Range,Reciprocal,ReduceMax,ReduceMean,ReduceMin,ReduceProd,ReduceSum,Relu,Reshape,Resize,ReverseSequence,Round,ScatterND,Shape,Sigmoid,Sin,Size,Slice,Softmax,SpaceToDepth,Split,Sqrt,Squeeze,Sub,Sum,Tanh,ThresholdedRelu,Tile,TopK,Transpose,Unique,Unsqueeze,Where

# other ops found in test models
ai.onnx;12;GlobalAveragePool,MatMulInteger,QLinearConv,QLinearMatMul
ai.onnx;13;GlobalAveragePool,MatMulInteger,QLinearConv,QLinearMatMul
ai.onnx;12;Erf,GlobalAveragePool,InstanceNormalization,MatMulInteger,QLinearConv,QLinearMatMul
ai.onnx;13;Erf,GlobalAveragePool,InstanceNormalization,MatMulInteger,QLinearConv,QLinearMatMul

# Control flow ops
# - If and Loop are covered by the tflite converter list
Expand All @@ -24,7 +24,9 @@ ai.onnx;12;Scan
ai.onnx;13;Scan

# internal ops added by optimizers
com.microsoft;1;DynamicQuantizeMatMul,FusedConv,FusedGemm,FusedMatMul,MatMulIntegerToFloat,NhwcMaxPool,QLinearAdd,QLinearAveragePool,QLinearConv,QLinearGlobalAveragePool,QLinearMul,QLinearSigmoid
# Note: LayerNormalization is an internal op even though it is (incorrectly) registered in the ONNX domain.
ai.onnx;1;LayerNormalization

@skottmckay Scott McKay (skottmckay) Jun 17, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to comment that this is incorrectly registered in the ONNX domain even though it's an internal op.

May be possible to add a registration in the correct domain and mark the old one as deprecated. IIRC that's what we do when internal ops migrate to official onnx ops.

com.microsoft;1;BiasGelu,DynamicQuantizeMatMul,FusedConv,FusedGemm,FusedMatMul,Gelu,MatMulIntegerToFloat,NhwcMaxPool,QLinearAdd,QLinearAveragePool,QLinearConv,QLinearGlobalAveragePool,QLinearMul,QLinearSigmoid,SkipLayerNormalization
# NHWC transformer also uses this, so assuming it's valuable enough to include
com.microsoft;1;QLinearLeakyRelu

Expand Down
3 changes: 2 additions & 1 deletion tools/ci_build/github/android/mobile_package.required_operators.readme.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,4 +76,5 @@ Other
- SuperResolution (https://pytorch.org/tutorials/advanced/super_resolution_with_onnxruntime.html)
- DeepLabV3 (https://pytorch.org/tutorials/beginner/deeplabv3_on_android.html)
- EfficientNet (https://github.com/lukemelas/EfficientNet-PyTorch)
- SSD Mobilenet V1 and V2 (https://github.com/qfgaohao/pytorch-ssd)
- SSD Mobilenet V1 and V2 (https://github.com/qfgaohao/pytorch-ssd)
- wav2vec 2.0 (adapted from https://github.com/pytorch/ios-demo-app/blob/9845c51c2b417416ccd11277c46e54f9ab811d37/SpeechRecognition/create_wav2vec2.py)