Repository navigation
Support for FreeBSD-arm64 #71338
Description
Activity
OSX/BSD builds use the target system's libunwind:
from src/coreclr/pal/src/CMakeLists.txtif(CLR_CMAKE_TARGET_OSX OR CLR_CMAKE_TARGET_FREEBSD) # On OSX and *BSD, we use the libunwind that's part of the OS set(CLR_CMAKE_USE_SYSTEM_LIBUNWIND 1) endif(CLR_CMAKE_TARGET_OSX OR CLR_CMAKE_TARGET_FREEBSD)which makes it add only
include_directories(SYSTEM $ENV{ROOTFS_DIR}/usr/local/include)Yup, I think we just need to update the list of compatible context registers for freebsd-arm64 to make it work.
If that would only make sense to me I would maybe help more, still learing on the fly. I found #40149 and doing quick change:
diff --git a/src/coreclr/pal/src/include/pal/context.h b/src/coreclr/pal/src/include/pal/context.h index fec5139fd5e..2cc4941d653 100644 --- a/src/coreclr/pal/src/include/pal/context.h +++ b/src/coreclr/pal/src/include/pal/context.h @@ -407,7 +407,7 @@ inline void *FPREG_Xstate_Ymmh(const ucontext_t *uc) #if defined(HOST_ARM64) -#ifndef TARGET_OSX +#if !defined(TARGET_OSX) && !defined(TARGET_FREEBSD) #define MCREG_X0(mc) ((mc).regs[0]) #define MCREG_X1(mc) ((mc).regs[1]) @@ -482,7 +482,7 @@ const fpsimd_context* GetConstNativeSigSimdContext(const native_context_t *mc) return GetNativeSigSimdContext(const_cast<native_context_t*>(mc)); } -#else // TARGET_OSX +#elif defined(TARGET_OSX) // TARGET_OSX #define MCREG_X0(mc) ((mc)->__ss.__x[0]) #define MCREG_X1(mc) ((mc)->__ss.__x[1])
native compile fail with:
In file included from /usr/home/sec/runtime/src/coreclr/pal/src/exception/seh.cpp:407: /usr/home/sec/runtime/src/coreclr/pal/src/exception/seh-unwind.cpp:177:2: error: unsupported architecture #error unsupported architecture ^ /usr/home/sec/runtime/src/coreclr/pal/src/exception/seh-unwind.cpp:183:5: error: use of undeclared identifier 'ASSIGN_UNWIND_REGS' ASSIGN_UNWIND_REGS ^ 2 errors generated. --- pal/src/CMakeFiles/coreclrpal.dir/exception/seh.cpp.o --- *** [pal/src/CMakeFiles/coreclrpal.dir/exception/seh.cpp.o] Error code 1Is this the context registers that needs to be updated (looked at the code and there's no ARM64 for any of the targets)?
@janvorli There you go:
-- Performing Test COMPILER_SUPPORTS_F_STACK_PROTECTOR_STRONG - Success -- Performing Test COMPILER_SUPPORTS_W_IMPLICIT_FALLTHROUGH - Success -- Performing Test HAVE_LTO - Failed -- Performing Test COMPILER_SUPPORTS_F_ALIGNED_NEW - Success -- Performing Test HAVE_DIRENT_D_TYPE - Success -- Performing Test C_SUPPORTS_WUNGUARDED_AVAILABILITY - Success -- Performing Test HAVE_IN_PKTINFO - Failed -- Performing Test HAVE_IP_MREQN - Success -- Performing Test HAVE_STATFS_VFS - Failed -- Performing Test HAVE_STATFS_MOUNT - Success -- Performing Test HAVE_FLOCK64 - Failed -- Performing Test HAVE_IFADDRS - Success -- Performing Test HAVE_UTSNAME_DOMAINNAME - Failed -- Performing Test HAVE_STAT_BIRTHTIME - Success -- Performing Test HAVE_STAT_FLAGS - Success -- Performing Test HAVE_STAT_TIMESPEC - Success -- Performing Test HAVE_STAT_TIM - Success -- Performing Test HAVE_STAT_NSEC - Success -- Performing Test HAVE_DIRENT_NAME_LEN - Success -- Performing Test HAVE_STATFS_FSTYPENAME - Success -- Performing Test HAVE_STATVFS_FSTYPENAME - Failed -- Performing Test HAVE_GNU_STRERROR_R - Failed -- Performing Test HAVE_READDIR_R - Success -- Performing Test KEVENT_HAS_VOID_UDATA - Success -- Performing Test HAVE_FDS_BITS - Success -- Performing Test HAVE_PRIVATE_FDS_BITS - Success -- Performing Test HAVE_SENDFILE_4 - Failed -- Performing Test HAVE_SENDFILE_6 - Failed -- Performing Test HAVE_SENDFILE_7 - Success -- Performing Test HAVE_GETNAMEINFO_SIGNED_FLAGS - Success -- Performing Test HAVE_SHM_OPEN_THAT_WORKS_WELL_ENOUGH_WITH_MMAP - Success -- Performing Test HAVE_CLOCK_MONOTONIC - Success -- Performing Test HAVE_CLOCK_REALTIME - Success -- Performing Test BIND_ADDRLEN_UNSIGNED - Success -- Performing Test IPV6MR_INTERFACE_UNSIGNED - Success -- Performing Test INOTIFY_RM_WATCH_WD_UNSIGNED - Failed -- Performing Test HAVE_MKSTEMPS - Success -- Performing Test HAVE_MKSTEMP - Success -- Performing Test HAVE_NETINET_TCP_VAR_H - Success -- Performing Test HAVE_NETINET_UDP_VAR_H - Success -- Performing Test HAVE_NETINET_IP_VAR_H - Success -- Performing Test HAVE_NETINET_ICMP_VAR_H - Success -- Performing Test HAVE_TCP_H_TCPSTATE_ENUM - Failed -- Performing Test HAVE_GETDOMAINNAME_SIZET - Failed -- Performing Test HAVE_IN_EXCL_UNLINK - Success -- Performing Test HAVE_TCP_H_TCP_KEEPALIVE - Failed -- Performing Test HAVE_BUILTIN_MUL_OVERFLOW - Success -- Performing Test HAVE_UDAT_STANDALONE_SHORTER_WEEKDAYS - Success -- Performing Test HAVE_VM_FLAGS_SUPERPAGE_SIZE_ANY - Failed -- Performing Test HAVE_MAP_HUGETLB - Failed -- Performing Test HAVE_LTTNG_TRACEPOINT_H - Failed -- Performing Test HAVE_CPUSET_T - Success -- Performing Test HAVE_TM_GMTOFF - Success -- Performing Test HAVE_GREGSET_T - Failed -- Performing Test HAVE___GREGSET_T - Failed -- Performing Test HAVE_FPSTATE_GLIBC_RESERVED1 - Failed -- Performing Test HAVE_SYSINFO_WITH_MEM_UNIT - Failed -- Performing Test HAVE_FPREGS_WITH_CW - Failed -- Performing Test REALPATH_SUPPORTS_NONEXISTENT_FILES - Failed -- Performing Test SSCANF_SUPPORT_ll - Success -- Performing Test SSCANF_CANNOT_HANDLE_MISSING_EXPONENT - Failed -- Performing Test HAVE_LARGE_SNPRINTF_SUPPORT - Success -- Performing Test HAVE_BROKEN_FIFO_SELECT - Failed -- Performing Test HAVE_BROKEN_FIFO_KEVENT - Failed -- Performing Test HAVE_SCHED_GET_PRIORITY - Success -- Performing Test HAVE_WORKING_GETTIMEOFDAY - Success -- Performing Test HAVE_WORKING_CLOCK_GETTIME - Success -- Performing Test HAVE_CLOCK_MONOTONIC_COARSE - Success -- Performing Test HAVE_CLOCK_THREAD_CPUTIME - Success -- Performing Test MMAP_ANON_IGNORES_PROTECTION - Failed -- Performing Test ONE_SHARED_MAPPING_PER_FILEREGION_PER_PROCESS - Failed -- Performing Test PTHREAD_CREATE_MODIFIES_ERRNO - Failed -- Performing Test SEM_INIT_MODIFIES_ERRNO - Failed -- Performing Test HAVE_PROCFS_CTL - Failed -- Performing Test HAVE_PROCFS_MAPS - Failed -- Performing Test HAVE_PROCFS_STAT - Failed -- Performing Test HAVE_PROCFS_STATUS - Success -- Performing Test HAVE_COMPATIBLE_ACOS - Success -- Performing Test HAVE_COMPATIBLE_ASIN - Success -- Performing Test HAVE_COMPATIBLE_POW - Success -- Performing Test HAVE_VALID_NEGATIVE_INF_POW - Success -- Performing Test HAVE_VALID_POSITIVE_INF_POW - Success -- Performing Test HAVE_COMPATIBLE_ATAN2 - Success -- Performing Test HAVE_COMPATIBLE_EXP - Failed -- Performing Test HAVE_COMPATIBLE_ILOGB0 - Failed -- Performing Test HAVE_COMPATIBLE_ILOGBNAN - Success -- Performing Test HAVE_COMPATIBLE_LOG - Success -- Performing Test HAVE_COMPATIBLE_LOG10 - Success -- Performing Test UNGETC_NOT_RETURN_EOF - Failed -- Performing Test HAS_POSIX_SEMAPHORES - Success -- Performing Test GETPWUID_R_SETS_ERRNO - Failed -- Performing Test FILE_OPS_CHECK_FERROR_OF_PREVIOUS_CALL - Failed -- Performing Test UNWIND_CONTEXT_IS_UCONTEXT_T - Success -- Performing Test HAVE_XSWDEV - Success -- Performing Test HAVE_XSW_USAGE - Failed -- Performing Test HAVE_PUBLIC_XSTATE_STRUCT - Failed -- Performing Test HAVE_PR_SET_PTRACER - Failed -- Performing Test HAVE_FULLY_FEATURED_PTHREAD_MUTEXES - Success -- Performing Test HAVE_PTHREAD_THREADID_NP - Failed -- Performing Test HAVE_PTHREAD_GETTHREADID_NP - Successedit: also attached full log from build attempt.
build_log.txtIt seems to me that your last change was correct. But you also need to add #elif (defined(HOST_UNIX) && defined(HOST_ARM64)) before the #else with #error that fires. In that #elif, you'd add:
ASSIGN_REG(Pc) \ ASSIGN_REG(Sp) \ ASSIGN_REG(Fp) \ ASSIGN_REG(Lr) \ ASSIGN_REG(X19) \ ASSIGN_REG(X20) \ ASSIGN_REG(X21) \ ASSIGN_REG(X22) \ ASSIGN_REG(X23) \ ASSIGN_REG(X24) \ ASSIGN_REG(X25) \ ASSIGN_REG(X26) \ ASSIGN_REG(X27) \ ASSIGN_REG(X28)
I hope I got it right.
@janvorli Tried that, it failed with bunch of errors for every register:
In file included from /usr/home/sec/runtime/src/coreclr/pal/src/exception/seh.cpp:407: /usr/home/sec/runtime/src/coreclr/pal/src/exception/seh-unwind.cpp:190:2: error: use of undeclared identifier 'MCREG_X21' ASSIGN_UNWIND_REGS ^ /usr/home/sec/runtime/src/coreclr/pal/src/exception/seh-unwind.cpp:178:2: note: expanded from macro 'ASSIGN_UNWIND_REGS' ASSIGN_REG(X21) \ ^ /usr/home/sec/runtime/src/coreclr/pal/src/exception/seh-unwind.cpp:189:25: note: expanded from macro 'ASSIGN_REG' #define ASSIGN_REG(reg) MCREG_##reg(unwContext->uc_mcontext) = winContext->reg; ^ <scratch space>:95:1: note: expanded from here MCREG_X21 ^Leaving this empty, will go on but then fail again with similar errors. If I'm looking at this correctly, I think it's missing some #def for HOST_ARM64 and TARGET_FREEBSD inside
src/coreclr/pal/src/include/pal/context.hwith those registers?And also there's again
error: unknown type name 'fpsimd_context'- which FreeBSD doesn't have, but if I add|| defined(TARGET_FREEBSD)next to TARGET_OSX those errors seems to go away (don't know if that will work in the end though, as I going blind in here. Also the undefined registers errors are there, so my bet is that fixing those first would be real fix).I wodner, if there's libunwind used (which state FreeBSD is supported under x64 and arm64) - shouldn't this be used (if I assume this code here is about that).
As I saw @sdmaclea did the OSX part of changes - maybe some hints? :)
Just to make sure, this is native build under ARM64 FreeBSD host.
Ah, I haven't noticed that your change around line 490 was like this:
-#else // TARGET_OSX +#elif defined(TARGET_OSX) // TARGET_OSX
That way, you have eliminated both definitions of the MCREG_xxx for arm64. What does the ucontext_t struct look like on FreeBSD arm64? We'll either need to define the MCREG_xxx for the FreeBSD separately or reuse the one for Linux / OSX if it happens to be the same.
Ah, found it in FreeBSD sources. So it is different and we will need to have another
#elif TARGET_FREEBSDthere. It seems that the definitions should look like:#define MCREG_X21(mc) (mc.mc_gpregs.gp_x[21]) #define MCREG_X22(mc) (mc.mc_gpregs.gp_x[22]) #define MCREG_X23(mc) (mc.mc_gpregs.gp_x[23]) #define MCREG_X24(mc) (mc.mc_gpregs.gp_x[24]) #define MCREG_X25(mc) (mc.mc_gpregs.gp_x[25]) #define MCREG_X26(mc) (mc.mc_gpregs.gp_x[26]) #define MCREG_X27(mc) (mc.mc_gpregs.gp_x[27]) #define MCREG_X28(mc) (mc.mc_gpregs.gp_x[28]) #define MCREG_Fp(mc) (mc.mc_gpregs.gp_fp) #define MCREG_Lr(mc) (mc.mc_gpregs.gp_lr) #define MCREG_Sp(mc) (mc.mc_gpregs.gp_sp) #define MCREG_Pc(mc) (mc.mc_gpregs.gp_pc)
As for the floating point access we will also need an extra code here. Take the following with a grain of salt, it might not be exactly right or might need extra change at another place too:
inline struct fpregs* GetNativeSigSimdContext(native_context_t *mc) { return &(mc->mc_fpregs); } inline const struct fpregs* GetConstNativeSigSimdContext(const native_context_t *mc) { return GetNativeSigSimdContext(const_cast<native_context_t*>(mc)); }
@janvorli Thanks, I was looking at the same files and was about to post :)
Can't findPcandFpinucontextin structs btw.Managed to get past some errors, but now a lot of:
[ 6%] Built target eventpipe_objs Consolidate compiler generated dependencies of target mdruntimerw_wks_obj /usr/home/sec/runtime/src/coreclr/pal/src/thread/context.cpp:339:9: error: use of undeclared identifier 'BSDREG_Cpsr' ASSIGN_ALL_REGS ^ /usr/home/sec/runtime/src/coreclr/pal/src/thread/context.cpp:286:9: note: expanded from macro 'ASSIGN_ALL_REGS' ASSIGN_CONTROL_REGS \ ^ /usr/home/sec/runtime/src/coreclr/pal/src/thread/context.cpp:137:9: note: expanded from macro 'ASSIGN_CONTROL_REGS' ASSIGN_REG(Cpsr) \ ^ /usr/home/sec/runtime/src/coreclr/pal/src/thread/context.cpp:333:62: note: expanded from macro 'ASSIGN_REG' #define ASSIGN_REG(reg) MCREG_##reg(registers.uc_mcontext) = BSDREG_##reg(ptrace_registers); ^ <scratch space>:248:1: note: expanded from here BSDREG_Cpsr ^ [ 8%] Built target ceefgen_obj /usr/home/sec/runtime/src/coreclr/pal/src/thread/context.cpp:339:9: error: use of undeclared identifier 'BSDREG_Fp' /usr/home/sec/runtime/src/coreclr/pal/src/thread/context.cpp:286:9: note: expanded from macro 'ASSIGN_ALL_REGS' ASSIGN_CONTROL_REGS \ ^ /usr/home/sec/runtime/src/coreclr/pal/src/thread/context.cpp:138:9: note: expanded from macro 'ASSIGN_CONTROL_REGS' ASSIGN_REG(Fp) \ ^ ...I haven't saw any existing
BSDREG_Xfor defined (even for OSX)I'm starting to think does this road make sense. Shouldn't this reuse existing ARM64 code and/or use machine specific libunwind for the rest?
115 remaining items
You can use
x/32a $spto dump 32 addresses at the current sp. That will dump addresses and also symbols for ones that are native code addresses.Here you go
(lldb) x/32a $sp 0x436cc7d4ef0: 0x0000000000000000 0x436cc7d4ef8: 0x0000000000000000 0x436cc7d4f00: 0x0000000000000000 0x436cc7d4f08: 0x0000000000000000 0x436cc7d4f10: 0x0000000000000000 0x436cc7d4f18: 0x0000000000000000 0x436cc7d4f20: 0x0000000000000000 0x436cc7d4f28: 0x0000000000000000 0x436cc7d4f30: 0x0000000000000000 0x436cc7d4f38: 0x0000000000000000 0x436cc7d4f40: 0x0000000000000000 0x436cc7d4f48: 0x4010040140100401 0x436cc7d4f50: 0xa000000000000000 0x436cc7d4f58: 0xa000000000000000 0x436cc7d4f60: 0x0000000000000000 0x436cc7d4f68: 0x8020000000000000 0x436cc7d4f70: 0x0000000500000005 0x436cc7d4f78: 0x0000000000000000 0x436cc7d4f80: 0x03c903c903c903c9 0x436cc7d4f88: 0x03c903c903c903c9 0x436cc7d4f90: 0x0000000000000002 0x436cc7d4f98: 0x0000000000000001 0x436cc7d4fa0: 0x0000000000000000 0x436cc7d4fa8: 0x0000000000000000 0x436cc7d4fb0: 0x0000000000000000 0x436cc7d4fb8: 0x0000000000000000 0x436cc7d4fc0: 0x0000000000000000 0x436cc7d4fc8: 0x0000000000000000 0x436cc7d4fd0: 0x0000000000000000 0x436cc7d4fd8: 0x0000000000000000 0x436cc7d4fe0: 0x0000000000000000 0x436cc7d4fe8: 0x0000000000000000@janvorli Isn't this connected with dotnet/source-build#4007 - just saw this...
edit: using crossgen2 binary from 8.0.100 SDK makes the build complete without errors... so the issue is still in 8.0.3 (as I assume this crossgen2 was built during the vmr build) ?
edit 2: also hitting the issue with cross-compiled 9.0.100-preview.2.24103.2 used to compile 9-preview.2 from vmr :(
Has this been resolved in net9p3?
The bug is in
libunwind, fix for .net was to revert some recent changes in internal lib they use, but for FreeBSD we use the ports version which have this bug for some time now :(You can possibly temporarily switch to using the internal libunwind copy. It is essentially the same code as upstream with a couple of patches that we remove as soon as a fix is made upstream.
AFAIK there should only be two places that need changes:
runtime/src/coreclr/pal/src/CMakeLists.txt
Lines 1 to 5 in 7dc3669
runtime/src/native/corehost/apphost/static/CMakeLists.txt
Lines 225 to 227 in 7dc3669
That should change the behavior back to using the internal version
Right, that should be all that's needed.
Small update for arm64 interested. Doing crossbuild for 9-preview.7 is possible (there are 2 patches needed for ILC, more here #104497). Using that as bootstrap for building natively runtime and aspnet mostly work - produced ILC segfaults becuase of missing symbols export. sdk/installer build and produce working SDK and the rest. Should be possible to perform VMR build, when ILC thing is resolved.
edit: ilc bug was resolved, it was my fault not to replace old nuget, so building all repos is a go.
Tried to build 9.0.0 using internal libunwind (CLR_CMAKE_USE_SYSTEM_LIBUNWIND 0 in both places), (with libunwind installed system-wide, as needed by bootstrap SDK), but it fails with:
[ 53%] Linking CXX static library libRuntime.ServerGC.a ld.lld: error: undefined symbol: _Uaarch64_setcontext referenced by Gos-freebsd.c:69 (/mnt/sec/dotnet-core-freebsd-source-build/runtime/src/native/external/libunwind/src/aarch64/Gos-freebsd.c:69) Los-freebsd.c.o:(_ULaarch64_local_resume) in archive ../pal/src/libcoreclrpal.a clang++: error: linker command failed with exit code 1 (use -v to see invocation) --- ilasm/ilasm ---can this be related to fact, that we have
/usr/local/includein other places maybe or this is something different?Based on libunwind/libunwind@032abaa, we will need to add aarch64/setcontext.S next to
(right after line 99)Reacted by Szczepan ĆwiklińskiTagging subscribers to this area: @agocke
See info in area-owners.md if you want to be subscribed.
Hi,
Not to spam #14537 (as it's already quite big) :)
As full native build of either v6 and v7 is possible under FreeBSD-amd64, I went ahead and tried to make the same process work for ARM64.
After getting configure to success, I've tried:
As it was used before, to cross compile from Linux-amd64 to Freebsd-x64, this fail at very early stage with:
Command I've used
ROOTFS_DIR=/home/ubuntu/arm64 ./src/coreclr/build-runtime.sh -arm64 -debug clang14 -cross -os FreeBSD.This does look like something missing in configure step, as those headers are present on rootfs_dir and host system:
so I guess something during configure stage should be added to look in those directories?
Trying with both
-crossand without, it fails with the same error:Command used was the same as with ad. 1. This looks more complicated, as it have something to do with libunwind being used? No idea how to get over that :)
Including patches I've come up with.
If anyone expierenced with cross build could comment on that, to even confirm it's possible in the current state to make that work or would it require far more work?
If someone is interested in helping/testing/making this work I can provide access to either FreeBSD or Linux ARM64 box that I use.
runtime_freebsd_arm64.txt