Bug in onnxruntime\onnxruntime\core\providers\acl\nn\conv.cc?

[capriccioo]

Describe the bug
arm_compute::NEDepthwiseConvolutionLayer3x3> is used in onnxruntime\onnxruntime\core\providers\acl\nn\conv.cc line 211:
auto layer = std::make_shared<arm_compute::NEDepthwiseConvolutionLayer3x3>();

but I searched the definition of NEDepthwiseConvolutionLayer3x3 in the repository ARM-software/ComputeLibrary and got nothing.

System information

• ONNX Runtime version: 1.3.0

To Reproduce
When building onnxruntime with option --use_acl, I got the error below:
/home/test/code/onnxruntime/onnxruntime/core/providers/acl/nn/conv.cc:211:52: error: ‘NEDepthwiseConvolutionLayer3x3’ is not a member of ‘arm_compute’
auto layer = std::make_shared<arm_compute::NEDepthwiseConvolutionLayer3x3>();

Expected behavior
Is this a bug or i had done something wrong?

[capriccioo]

No definition of NEDepthwiseConvolutionLayer3x3 in this file.

[goloskokovic]

try using ComputeLibrary v19.08
https://github.com/ARM-software/ComputeLibrary/blob/v19.08/arm_compute/runtime/NEON/functions/NEDepthwiseConvolutionLayer.h

[capriccioo]

Thanks.
I downloaded ComputeLibrary v19.08 and compiled it and then build onnxruntime with option --use_acl almost successfully. I got error at the last step: build_wheel:

Running subprocess in '/home/test/code/onnxruntime/build/Linux/Release'
['/usr/bin/python3', '/home/test/code/onnxruntime/setup.py', 'bdist_wheel', '--use_acl']
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help

error: option --use_acl not recognized

If I build wheel manually: python3 /home/test/code/onnxruntime/setup.py bdist_wheel,
I got a whl file which is identical to the file build without option "--use_acl". onnxruntime with ACL is not support python?

[capriccioo]

I know this.
What confused me is the whl files generated by two ways below are exactly same:

1. ./build.sh --config Release --update --build --build_wheel --use_acl;
   python3 /home/test/code/onnxruntime/setup.py bdist_wheel
2. ./build.sh --config Release --update --build --build_wheel

the output whl files have not only the same filename, but also the same size.
And when I install them and run my python code for performance test , they have the same performance.

So I am wondering: does onnxruntime with ACL support python?
Also I have the same question on onnxruntime with armnn.

[capriccioo]

sess.get_providers() shows thers is only CPUExecutionProvider.
So how to build a whl file which has ACLExecutionProvider?

[capriccioo]

sess.get_providers() shows thers is only CPUExecutionProvider.
So how to build a whl file which has ACLExecutionProvider?

also print providers from onnxruntime not session to be sure. as in session are providers you set and in onnxruntime are available from the build

I still have no idea how to use ACLExecutionProvider in python...

[goloskokovic]

@jywu-msft
#3586

[capriccioo]

@jywu-msft
#3586

I build onnxruntime by command line:
./build.sh --use_acl

An error came out at the last step:

[100%] Linking CXX executable onnxruntime_test_all
[100%] Built target onnxruntime_test_all
Makefile:140: recipe for target 'all' failed
make: *** [all] Error 2
Traceback (most recent call last):
File "/home/test/code/onnxruntime/tools/ci_build/build.py", line 1805, in
sys.exit(main())
File "/home/test/code/onnxruntime/tools/ci_build/build.py", line 1700, in main
build_targets(args, cmake_path, build_dir, configs, args.parallel)
File "/home/test/code/onnxruntime/tools/ci_build/build.py", line 868, in build_targets
run_subprocess(cmd_args, env=env)
File "/home/test/code/onnxruntime/tools/ci_build/build.py", line 402, in run_subprocess
env=my_env, shell=shell)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/usr/local/bin/cmake', '--build', '/home/test/code/onnxruntime/build/Linux/Debug', '--config', 'Debug', '--', '-j4']' returned non-zero exit status 2.

[jywu-mysoft]

which version of ACL are you using?
If you use ACL version 19.08 you need to specify it.
see: https://github.com/microsoft/onnxruntime/blob/master/tools/ci_build/build.py#L336
it defaults to ACL_1905
so you would need to use --use_acl=ACL_1908

[capriccioo]

Still no whl file generated.
All I want to know is how to use ACLExecutionProvider in python...

[goloskokovic]

repeat every step as for DML
rename onnxruntime_pybind_state.txt to onnxruntime_pybind_state.cc

[jywu-mysoft]

capriccioo , if you're still encountering issues, can you please provide full logs and details for reproducing?
Prabhat Roy (@prabhat00155) has been testing ACL on onnxruntime and can probably provide some tips and guidance.

[capriccioo]

repeat every step as for DML
rename onnxruntime_pybind_state.txt to onnxruntime_pybind_state.cc

I downloaded the master branch of onnxruntime, and repeat every step as for DML, then build onnxruntime:
./build.sh --config Release --update --build --build_wheel --use_acl --parallel --skip_test
After the build finished, I got a whl file:
onnxruntime_ACL_1905-1.3.1-cp36-cp36m-linux_aarch64.whl
I installed it and import onnxruntime in python, I got an error message:

_/home/test/.local/lib/python3.6/site-packages/onnxruntime/capi/_pybind_state.py:14: UserWarning: Cannot load onnxruntime.capi. Error: '/home/test/.local/lib/python3.6/site-packages/onnxruntime/capi/onnxruntime_pybind11_state.so: undefined symbol: _ZN11onnxruntime34CreateExecutionProviderFactory_ACLEi'.
warnings.warn("Cannot load onnxruntime.capi. Error: '{0}'.".format(str(e)))
Traceback (most recent call last):
File "", line 1, in
File "/home/test/.local/lib/python3.6/site-packages/onnxruntime/init.py", line 13, in
from onnxruntime.capi.pybind_state import get_all_providers, get_available_providers, get_device, set_seed,
ImportError: cannot import name 'get_all_providers'

[capriccioo]

I build onnxruntime with option --use_acl=ACL_1908 （I intalled ComputerLibrary19.08）, and I got the same error when import onnxruntime in python.

[capriccioo]

kAclExecutionProvider missing in onnxruntime_pybind_state.cc#L276: all_providers

looks like calling CreateExecutionProviderFactory_ACL is making this problem. go to onnxruntime_pybind_state.cc#L438 and set: onnxruntime::CreateExecutionProviderFactory_ACL(1)

also check if you have in onnxruntime_pybind_state.cc#L345:
} else if (type == kAclExecutionProvider) {

What do you mean?
I checked onnxruntime_pybind_state.cc and found:
else if (type == kAclExecutionProvider) {
#ifdef USE_ACL
RegisterExecutionProvider(sess, *onnxruntime::CreateExecutionProviderFactory_ACL(0));
#endif
}

Should I modify the "0" here to "1"?
Should I modify the "0" in onnxruntime_pybind_state.cc#L438 to "1"?