Repository navigation
Conversation
Just faster and using much less memory. Implements https://github.com/SpexAI/PointClouds/issues/12
localai-org-maint-bot
left a comment
There was a problem hiding this comment.
The new float calibration path discards a valid calibration point in src/depth_upscale.cpp. Normalization maps the smallest finite relative depth to zero, then the fit loop skips every predictor <= 0.0. Zero here is a valid normalized predictor, not a missing sensor sample (the latter is already represented by sensor_depth[i] == 0).
A minimal case is a 1x3 relative map {1, 2, 3}, sensor ranges {1000, 2000, 3000}, degree 2 and gaussian_sigma 0. Normalization produces {0, 0.5, 1}; only two samples enter the three-coefficient fit, so it returns insufficient overlapping samples despite three non-degenerate calibration pairs. Please retain finite zero-valued predictors in the normalized float path and add this case to test_depth_upscale. Preserve any intentionally different legacy uint8 semantics explicitly.
The unconditional find_package(TIFF REQUIRED) also changes the dependency requirements for every existing embedding of the library, including callers that do not use depth-upscale. Please make that dependency opt-in, or coordinate and validate the build/package changes in LocalAI before merge.
Reviewed the complete diff at 035147c. These are source-level findings; CMake, a C++ compiler, and model fixtures are unavailable here. Maintainer edits are disabled for this fork, so I cannot push the calibration repair directly to this branch.
035147c to
87575ef
Compare
|
Should be fixed now |
| @@ -0,0 +1,2 @@ | |||
| cmake -B build -DDA_BUILD_CLI=ON -DCMAKE_CUDA_ARCHITECTURES=native -DDA_GGML_CUDA=ON | |||
There was a problem hiding this comment.
Is this a leftover? It seems like your specific config?
Add TIFF reader support
Add depth-upscaling: interpolating a low-res depth image into a higher-res main image, so that we get for every main pixel depth and angle information.
depth-upscaleis the SpexAI-specific capture workflow: it combines a DA2 or single-fileDA3 prediction from an RGB TIFF with a paired sensor TIFF. The sensor image stores each uint16
depth sample in its first two interleaved uint8 channels (high byte, then low byte); the command
resizes and smooths the model map, polynomial-fits it to valid sensor pixels, and writes an
uncompressed single-channel uint16 TIFF in calibrated sensor units. The library API also accepts
a sparse range image already projected into the main camera by PointClouds, preserving its
millimetre scale rather than stretching each result to 0..65535.
I also added a onxx to gguf helper.
We've used to do this with pytorch, which was eating memory, and is far less stable than C++