Enable nuget packages for on device training - #13637
Conversation
Ashwini Khade (askhade)
changed the title
Ashwini Khade (askhade)
changed the title
| SourceFiles="$(NativeBuildOutputDirAbs)\$(OrtPackageId).$(PackageVersion).nupkg" | ||
| DestinationFolder="$(NativeBuildOutputDirAbs)\nuget-artifacts" | ||
| DestinationFolder="$(NativeBuildOutputDirAbs)\nuget-local-artifacts" | ||
| /> |
There was a problem hiding this comment.
There are some CI files that I think use the nuget-artifacts directory. Do those need updating?
\tools\ci_build\github\azure-pipelines\templates\c-api-cpu.yml
\tools\ci_build\github\windows\extract_nuget_files.ps1
\tools\ci_build\github\window\extract_nuget_files_gpu.ps1
Same question for generate_nuspec_for_native_nuget.py #Closed
There was a problem hiding this comment.
No.
The CI files use this directory to store all the downloaded artifacts from different jobs which the packaging pipeline depends on. "generate_nuspec_for_native_nuget.py" script then uses the existence of this directory to determine whether this is ADO pipeline build or local build.
This step merely copies the native nuget package to this "nuget-local-artifacts" dir for better discovery. After this step there are 2 copies of the native package: 1. in <native_build_dir>{config}{config} and 2. <native_build_dir>{config}{config}\nuget-local-artifacts
We can delete this copy step altogether.
|
Looks fine. I assume you've verified generating, installing and using the pkg. Are there going to be tests that exercise this? |
Baiju Meswani (baijumeswani)
left a comment
Baiju Meswani (baijumeswani)
left a comment
There was a problem hiding this comment.
Training related updates look good.
ee41293
| </Compile> | ||
| </ItemGroup> | ||
|
|
||
| <ItemGroup Condition="'$(TrainingEnabledNativeBuild)' == 'true'"> |
There was a problem hiding this comment.
I may misunderstand it :). does this mean, if enable_training is enabled, training_api related will also be built? so is there any need we have enable_training_on_device?
There was a problem hiding this comment.
This is the test project. Here we use "TrainingEnabledNativeBuild" to determine whether the native ort build includes training apis and use this to determine which tests to run.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
This pull request introduces 1 alert and fixes 1 when merging aadc65f into ce0025d - view on LGTM.com new alerts:
fixed alerts:
Heads-up: LGTM.com's PR analysis will be disabled on the 5th of December, and LGTM.com will be shut down ⏻ completely on the 16th of December 2022. It looks like GitHub code scanning with CodeQL is already set up for this repo, so no further action is needed 🚀. For more information, please check out our post on the GitHub blog. |
Changming Sun (snnn)
left a comment
Changming Sun (snnn)
left a comment
There was a problem hiding this comment.
For your information, right now we have some problems on running CUDA tests in C#. Yi Zhang (@mszhanyi) can provide more information. Right now we disabled the tests. So your change is not tested on that part.
### Description This PR enables building nuget packages locally for on device training using --build_nuget arg. This PR also enables the C# bindings by default in the managed package. If a user triggers any training apis when the native binary is not built for training, an exception with message "Training is disabled in the current build. Please build ONNXRuntime from source with the build flags enable_training and enable_training_on_device. " is thrown. Build command for creating nuget packes for on device training: build.bat --enable_training --enable_training_on_device --build_nuget 2 Nuget packages are built 1. Microsoft.ML.OnnxRuntime.Managed 2. Microsoft.ML.OnnxRuntime.Training OR Microsoft.ML.OnnxRuntime.Training.Gpu ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. -->
### Description This PR enables building nuget packages locally for on device training using --build_nuget arg. This PR also enables the C# bindings by default in the managed package. If a user triggers any training apis when the native binary is not built for training, an exception with message "Training is disabled in the current build. Please build ONNXRuntime from source with the build flags enable_training and enable_training_on_device. " is thrown. Build command for creating nuget packes for on device training: build.bat --enable_training --enable_training_on_device --build_nuget 2 Nuget packages are built 1. Microsoft.ML.OnnxRuntime.Managed 2. Microsoft.ML.OnnxRuntime.Training OR Microsoft.ML.OnnxRuntime.Training.Gpu ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. -->
7 participants
Description
This PR enables building nuget packages locally for on device training using --build_nuget arg.
This PR also enables the C# bindings by default in the managed package. If a user triggers any training apis when the native binary is not built for training, an exception with message "Training is disabled in the current build. Please build ONNXRuntime from source with the build flags enable_training and enable_training_on_device. " is thrown.
Build command for creating nuget packes for on device training:
build.bat --enable_training --enable_training_on_device --build_nuget
2 Nuget packages are built
Motivation and Context