This repository was archived by the owner on Mar 30, 2022. It is now read-only.
Repository navigation
This repository was archived by the owner on Mar 30, 2022. It is now read-only.
Runtime Error: symbol _$s11AllKeyPathss0B12PathIterablePTl Not Found on macOS #585
Description
macOS 11.0.1. Xcode 12.2 beta 3. Swift For TensorFlow 0.12. My project successfully builds and links, but I immediately get this runtime error when I'm running my unit tests:
...
[1146/1146] Linking MyModelAppPackageTests
* Build Completed!
error: terminated(1): /Library/Developer/Toolchains/swift-tensorflow-RELEASE-0.12.xctoolchain/usr/libexec/swift/pm/swiftpm-xctest-helper /Users/xander/dev/my_model/.build/x86_64-apple-macosx/debug/MyModelAppPackageTests.xctest /var/folders/zk/5rt0n9yx4p9frf4ybzt_mcz40000gn/T/TemporaryFile.CMRnsY output:
error: unableToLoadBundle("/Users/xander/dev/my_model/.build/x86_64-apple-macosx/debug/MyModelAppPackageTests.xctest")
2020-12-17 12:02:10.911 swiftpm-xctest-helper[30233:7363767] Error loading /Users/xander/dev/my_model/.build/x86_64-apple-macosx/debug/MyModelAppPackageTests.xctest/Contents/MacOS/MyModelAppPackageTests: dlopen(/Users/xander/dev/myp_model/.build/x86_64-apple-macosx/debug/MyModelAppPackageTests.xctest/Contents/MacOS/MyModelAppPackageTests, 265): Symbol not found: _$s11AllKeyPathss0B12PathIterablePTl
Referenced from: /Library/Developer/Toolchains/swift-tensorflow-RELEASE-0.12.xctoolchain/usr/lib/swift/macosx/libswiftTensorFlow.dylib
Expected in: /usr/lib/swift/libswiftCore.dylib
in /Library/Developer/Toolchains/swift-tensorflow-RELEASE-0.12.xctoolchain/usr/lib/swift/macosx/libswiftTensorFlow.dylib
It looks like it can't find the symbol _$s11AllKeyPathss0B12PathIterablePTl
Is it perhaps related to this code in my project:
public extension KeyPathIterable {
/*
This is based on clipByGlobalNorm here:
https://github.com/tensorflow/swift-models/blob/542a03c09aa7bd6e326a18b5d086f2bec8b4b24c/Models/Text/BERT/Utilities.swift#L23
Get all tensors in a gradient and get their min values, max values, and if any values are nan
See a similar hasNaN callback here: https://github.com/tensorflow/swift-models/pull/526/files#diff-2b4e6467b43db097c85ce302b963fefe2ae32a9d42e7147143e0d088a5cc6c55R103
*/
func gradientMinsAndMaxes<Scalar: TensorFlowFloatingPoint>() -> ([Scalar], [Scalar], Bool) {
var mins: [Scalar] = []
var maxes: [Scalar] = []
var hasNan: Bool = false
for kp in self.recursivelyAllWritableKeyPaths(to: Tensor<Scalar>.self) {
let tensor: Tensor<Scalar> = self[keyPath: kp]
mins.append(tensor.min().scalarized())
maxes.append(tensor.max().scalarized())
if hasNan == false {
if tensor.isNaN.any() {
hasNan = true
}
}
}
return (mins, maxes, hasNan)
}
func numParamters<Scalar: TensorFlowFloatingPoint>() -> Scalar {
var numParameters: Scalar = 0
for kp in self.recursivelyAllWritableKeyPaths(to: Tensor<Scalar>.self) {
let tensor: Tensor<Scalar> = self[keyPath: kp]
numParameters += Scalar(tensor.shape.dimensions.cumulativeProduct())
}
return numParameters
}
}These KeyPathIterable extensions are very similar to swift-models code here and here.
- Note that this is specific to macOS. The same project using S4TF 0.12 on Linux does not produce this runtime error. The unit tests pass without issue.
- These unit tests do pass without issue on S4TF 0.11 on macOS, this is new with 0.12.
Activity
xanderdunn commentedon Dec 17, 2020
Wow this appears to be specific to running the unit tests in parallel. When I execute
swift test -Xswiftc -g -Xswiftc --target=x86_64-apple-macos10.15 -Xlinker -L/usr/local/lib, all tests complete without the runtime error.When I execute
swift test -Xswiftc -g -Xswiftc --target=x86_64-apple-macos10.15 -Xlinker -L/usr/local/lib --parallel --num-workers 6, I immediately get the above error.It appears to be 100% reproducible. I ran with
--parallel --num-workers 6seven times and it always immediately produced the error. I ran withOUT--parallel --num-workers 6twice and the tests completed both times. This takes over 15min so I only did it twice.Note that this never occurs on Linux. My default of running the unit tests in parallel never produces this error.
xanderdunn commentedon Dec 17, 2020
I updated my Package.swift to macOS 11:
and my build and test targets to macos11:
Same result. When I apply
--parallelI get the runtime error and otherwise I don't get the runtime error. Adding a filter also hits the runtime error:--filter MyTestName. It doesn't matter what the test name is:Testing with neither filter nor parallel works:
Filter can't find symbol:
Parallel can't find symbol:
So strange. 🤔
xanderdunn commentedon Dec 19, 2020
I am sometimes seeing this without the
--parallelor--filteradditions toswift test, so it may be some kind of race condition. The error referencesExpected in: /usr/lib/swift/libswiftCore.dylib. Is this correct? I would expect the path to be the libswiftCore.dylib inside the S4TF toolchain. I have this set in my environment:export DYLD_LIBRARY_PATH=/Library/Developer/Toolchains/swift-tensorflow-RELEASE-0.12.xctoolchain/usr/lib/swift/macosx:"${DYLD_LIBRARY_PATH}".dan-zheng commentedon Dec 19, 2020
Thanks for reporting Xander!
The
Symbol not found: _$s11AllKeyPathss0B12PathIterablePTllinker issue is related to ad-hoc additions to the Swift standard library onapple/swift:tensorflowbranch.We're currently making a push to de-fork the Swift compiler and use standard toolchains for building Swift for TensorFlow. Here's some more context. When that effort is done, this linker issue should naturally go away!
xanderdunn commentedon Dec 19, 2020
@dan-zheng De-forking will be huge! Thanks for the context. I still find it curious that this first started appearing in 0.12 but never happened with 0.11, but I'm satisfied to test this again after the de-forking is complete. For now I'm able to run my tests on Linux only. Feel free to close or keep open until de-forking happens.
FruityLeo commentedon Dec 29, 2020
Hi,
I have just tried to install and run my first scripts (Usage.md) and the compiler is also not able to find the symbol
s11AllKeyPathss0B12PathIterablePTl.Build was successful but running the executable script gives the error below. Running the script with
swift -O inference.swiftworks flawlessly.I am also on macOs 11.0.1, Xcode 12.3 and followed the instructions from the guidelines.
Is this related or did I made a mistake when installing?