Add Apache Avro wrap

[stephanlachnit]

Adds a wrap for the C++ bindings of Apache Avro. Since it's only possible to build it without Boost using main, we need to wait until a new release.

TODO:

• Get release tarball from https://dlcdn.apache.org/avro/ and adjust folder structure
• Tests (optional)

/cc @WillAyd

[stephanlachnit]

Can we add program_names to a wrap for a program that might be disabled to be built or is that an issue?

[WillAyd]

Cool - thanks for doing this!

[stephanlachnit]

@bgilbert any idea why the CI fails with this cryptic message:

Run ./tools/fake_tty.py ./tools/sanity_checks.py
  ./tools/fake_tty.py ./tools/sanity_checks.py
  shell: /usr/bin/bash -e {0}
  env:
    TEST_ANNOTATE_CONTEXT: yes
    TEST_SKIP_BUILD: yes
Traceback (most recent call last):
  File "/home/runner/work/wrapdb/wrapdb/./tools/fake_tty.py", line 6, in <module>
    sys.exit(os.waitstatus_to_exitcode(pty.spawn(sys.argv[1:])))
                                       ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/pty.py", line 195, in spawn
    os.execlp(argv[0], *argv)
  File "<frozen os>", line 563, in execlp
  File "<frozen os>", line 580, in execvp
  File "<frozen os>", line 603, in _execvpe
PermissionError: [Errno 13] Permission denied
Error: Process completed with exit code 1.

[bgilbert]

Can we add program_names to a wrap for a program that might be disabled

Yes, add a skip_program_check in ci_config.json.

any idea why the CI fails with this cryptic message

The patch to sanity_checks.py removes its executable bit for some reason.

[stephanlachnit]

@WillAyd how should the dependency be named? For avro-c the pkg-config file is named avro-c, but avro-cpp does not have a pkg-config file upstream. The CMake target is called Avro - should we go with that, or rather go with a more descriptive avro-cpp? Probably makes sense to add pkg-config support upstream then as well.

[WillAyd]

Hmm following the c pattern it sounds like avro-cpp is best. @eli-schwartz may have a suggestion here as well

[eli-schwartz]

We definitely want to align with upstream. I agree that given the name "avro-c" it makes a decent amount of sense to use "avro-cpp" as well.

Avro upstream has only added CMake package config files for avro-cpp, not for avro-c, and it did that in January which hasn't been incorporated in a release yet. So it can be changed.

https://issues.apache.org/jira/browse/AVRO-3088 asked for "this way one could just use find_package(avrocpp) but it looks like unfortunately a decision was made to provide find_package(Avro) instead, which would mean it will be hard to have consistent naming.

[WillAyd]

FYI @wgtmac - you may be interested in the discussion around the package name for Avro

[stephanlachnit]

Ok, so I think I go the symbol visibility to work. There are some warning for certain inline methods but I don't think they are critical to fix. The only thing that is missing is the armv7 and x86 failure on Alpine, which seem to be actual 32-bit bugs. However, I don't think that those should be a blocker for WrapDB...?

In any case, we need to wait for 1.12.2 which includes the symbol visibility patch.

[eli-schwartz]

The only thing that is missing is the armv7 and x86 failure on Alpine, which seem to be actual 32-bit bugs. However, I don't think that those should be a blocker for WrapDB...?

You are correct -- they aren't blockers, just a useful heads up. Please report them upstream anyway, of course. :)

[stephanlachnit]

@martin-g @WillAyd any news on a 1.12.2 release? I saw apache/avro#3724 but not much activity otherwise

[martin-g]

I don't have any other information too.

[manuzhang]

@stephanlachnit avro 1.12.2 has been released, can you revive this PR?

[stephanlachnit]

@stephanlachnit avro 1.12.2 has been released, can you revive this PR?

Thanks for the headsup - I rebased and now it seems to work nicely (Alpine CI failures are unrelated).

[stephanlachnit]

There seems to be an interesting failure on x86 and armv7:

../subprojects/avro-cpp-1.12.2/test/DataFileTests.cc(428): error: in "DataFile tests_ test9.df/DataFileTest__testReaderSyncSeek": check num == count has failed [0 != 1000]
../subprojects/avro-cpp-1.12.2/test/DataFileTests.cc(430): error: in "DataFile tests_ test9.df/DataFileTest__testReaderSyncSeek": check actual.size() == count has failed [0 != 1000]
../subprojects/avro-cpp-1.12.2/test/DataFileTests.cc(469): error: in "DataFile tests_ test10.df/DataFileTest__testReaderSyncDiscovery": check sync_points_syncing == sync_points_reading has failed
../subprojects/avro-cpp-1.12.2/test/DataFileTests.cc(472): error: in "DataFile tests_ test10.df/DataFileTest__testReaderSyncDiscovery": check sync_points_syncing.size() > 10 has failed [1 <= 10]
../subprojects/avro-cpp-1.12.2/test/DataFileTests.cc(497): error: in "DataFile tests_ test11.df/DataFileTest__testReaderSplits": check actual_count == count has failed [0 != 1000]

[stephanlachnit]

@WillAyd if you want to take another at this PR, it is ready to be merged

[WillAyd]

lgtm