Skip to content

Do not pass a raw pointer as the recursion key to rb_exec_recursive()#1381

Open
koic wants to merge 1 commit into
ruby:masterfrom
koic:fix_asan_failure_in_rb_exec_recursive_spec
Open

Do not pass a raw pointer as the recursion key to rb_exec_recursive()#1381
koic wants to merge 1 commit into
ruby:masterfrom
koic:fix_asan_failure_in_rb_exec_recursive_spec

Conversation

@koic

@koic koic commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

The spec added in 20ae94d (for truffleruby/truffleruby#4334) passes a pointer to a 4-byte C stack struct as both the obj and arg arguments of rb_exec_recursive().

While arg is passed through to the callback as-is, obj is used as a key of the recursion guard hash in CRuby. Storing it there fires the GC write barrier, which reads the (8-byte) object flags from the non-object address, and AddressSanitizer reports a stack-buffer-overflow:

==29649==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7f0d9b7c7b20 ...
READ of size 8 at 0x7f0d9b7c7b20 thread T0
    #0 RB_BUILTIN_TYPE include/ruby/internal/value_type.h:191:30
    #1 rb_gc_impl_writebarrier gc/default/default.c:6621:5
    #2 rb_gc_writebarrier gc.c:3433:5
    #3 rb_obj_written include/ruby/internal/gc.h:641:9
    #4 tbl_update hash.c:1726:5
    #5 rb_hash_aset hash.c:2977:9
    #6 exec_recursive thread.c:5602:13
    #7 kernel_spec_rb_exec_recursive_with_pointer spec/ruby/optional/capi/ext/kernel_spec.c:370:10

Address 0x7f0d9b7c7b20 is located in stack of thread T0 at offset 32 in frame
    #0 kernel_spec_rb_exec_recursive_with_pointer kernel_spec.c:368
  This frame has 1 object(s):
    [32, 36) 'data' (line 369) <== Memory access at offset 32 partially overflows this variable

User impact

Since this spec was synced into ruby/ruby (ruby/ruby@0d68ef6b77), the build (ubuntu-24.04, asan) job of the https://github.com/ruby/ruby-dev-builder daily builds has been crashing in optional/capi/kernel_spec.rb, blocking the ruby-head releases used by ruby/setup-ruby since 2026-07-08 (e.g. https://github.com/ruby/ruby-dev-builder/actions/runs/29044853326).

As a result, downstream CIs that use ruby-head via ruby/setup-ruby are stuck with the 2026-07-06 snapshot, which predates ruby/ruby@291d0d95ea ("onig_reg_copy_body: fix copying of exact_end"). This is how the problem was found: the "Spec - ubuntu head" job on the default branch of rubocop/rubocop keeps failing because Regexp.new(regexp) copies made for the AllowedPatterns config silently fail to match on that snapshot:
https://github.com/rubocop/rubocop/actions/runs/29061954887/job/86265559997

Fix

The intent of the spec is that the callback argument is passed as-is, so this change keeps the raw pointer for arg and uses a real Ruby object (self) as the recursion key. The example still passes on CRuby:

$ make test-spec MSPECOPT=spec/ruby/optional/capi/kernel_spec.rb
1 file, 117 examples, 297 expectations, 0 failures, 0 errors, 0 tagged

@koic koic force-pushed the fix_asan_failure_in_rb_exec_recursive_spec branch from 54679aa to 370361f Compare July 10, 2026 03:14
The spec added in 20ae94d passed a pointer to a 4-byte C stack struct as both the `obj`
and `arg` arguments of rb_exec_recursive(). While `arg` is passed through to the callback as-is,
`obj` is used as a key of the recursion guard hash in CRuby. Storing it there fires the GC write barrier,
which reads the (8-byte) object flags from the non-object address, and AddressSanitizer reports a stack-buffer-overflow:

```
    ==29649==ERROR: AddressSanitizer: stack-buffer-overflow ...
    READ of size 8 at 0x7f0d9b7c7b20 thread T0
        #0 RB_BUILTIN_TYPE include/ruby/internal/value_type.h:191:30
        ruby#1 rb_gc_impl_writebarrier gc/default/default.c:6621:5
        ...
        ruby#6 exec_recursive thread.c:5602:13
        ruby#7 kernel_spec_rb_exec_recursive_with_pointer
           spec/ruby/optional/capi/ext/kernel_spec.c:370:10
```

This crash has been blocking the daily `ruby-head` releases of ruby/ruby-dev-builder (`build (ubuntu-24.04, asan)`)
since 2026-07-08:
https://github.com/ruby/ruby-dev-builder/actions/runs/29044853326

As a result, downstream CIs that use `ruby-head` via ruby/setup-ruby are stuck with the 2026-07-06 snapshot,
which predates ruby/ruby@291d0d95ea ("onig_reg_copy_body: fix copying of exact_end"). This is how the problem was found:
the "Spec - ubuntu head" job on the default branch of rubocop/rubocop keeps failing because `Regexp.new(regexp)` copies made
for the `AllowedPatterns` config silently fail to match on that snapshot:
https://github.com/rubocop/rubocop/actions/runs/29061954887/job/86265559997

The intent of the spec is that the callback argument is passed as-is, so keep the raw pointer for `arg` and
use a real Ruby object as the recursion key.
@koic koic force-pushed the fix_asan_failure_in_rb_exec_recursive_spec branch from 370361f to 80e37cb Compare July 10, 2026 03:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant