Repository navigation
Conversation
change rand250 to rand256, the full SIGMA range
requires T[n + 1] to be accessable. Still fails though.
but really we should loop over PATT_SIZE (large) if not specified, as with smart
as with smart
wrong XSIZE
we crash when our XSIZE based structures are different.
fixed the tests, but it still underflows T
add m<2 constraints. tests still fail
not repro anymore
but broke a test. 56c51d2 worked fine.
but it's much slower now. add new fails
with OUTPUT expressions like p - orig_y
but the tests do still fail.
actually finds now violations. stops on first violation if the test should be OK. prefer builtin memcmp over loop unwinding.
some just take too much time, we would need 1m * 220 and have only 15m
there is a counter example with --depth 1024
detected by cbmc! we dont have testcases with m=1, even if they are trivial.
track the remaining n, as in libc1 (aka libc memmem)
the 10 portable best
from the paper
this is optimized for multiple patterns search on GPU's
from https://mattsills.github.io/2024/03/02/rabin-karp/ not yet SSE4.2 nor arm NEON optimized, only AVX2
same as ./algocfg good 0
via fork/alarm. The current workaround is make; (sleep 2s; ./kill-tests.sh &); ./smart -text all Not for windows. Closes GH #12
- shmids.h: exclude sys/ipc.h/sys/shm.h for __AVR__ (freestanding,
no shared memory); key_t/shmctl fall back like the _WIN32 case.
- main.h: guard pre_time/run_time usage in main() with __AVR__,
matching the existing timer-globals guard.
- smart.c: gate POSIX signal headers behind HAVE_POSIX_SIGNALS,
undefined for __AVR__ as well as _WIN32.
- GNUmakefile:
- add -mmcu=$(MCU) (default atmega328p) for ARCH=avr; without a
concrete device avr-gcc's generic multilib has no linker script,
so malloc() fails with 'undefined reference to __heap_end'.
- all: only builds $(BINS) for avr; smart/test/select/algocfg/
compilesm/show/textgen are host orchestration tools (dirent.h,
fork/exec, wall-clock timers) that can't compile or run on a
freestanding MCU, and good.lst/asan.lst need to execute algocfg.
- skip hpbm's -fopenmp on avr (-pthread isn't a valid avr-gcc
flag; the #pragma omp degrades to a serial loop without it).
- add NON_AVR: 46 algorithms declare SIGMA*SIGMA/DSIGMA-sized
tables (up to 256KB) that exceed avr-gcc's 32767-byte object
limit or any real AVR's SRAM, or need SSE/AVX (simdkr); same
pattern already used for NON_SSE on non-x86_64 targets.
Verified: make CC=avr-gcc BINDIR="bin/avr-gcc" builds 223/223
eligible algorithm binaries with exit 0; avr-size shows a sane
~3KB flash footprint; qemu-system-avr -M uno boots bin/avr-gcc/bf
without a fault. Native x86_64 make is unaffected.
Generalizes the AVR freestanding-target support into a portable
FREESTANDING mechanism and adds a bare-metal ARM branch, giving two
armv6 flavors:
- Bare-metal Cortex-M0 / ARMv6-M (arm-none-eabi-gcc, newlib, no OS):
MCPU defaults to cortex-m0 (override via MCPU=cortex-m3 etc.);
-mthumb --specs=nano.specs --specs=nosys.specs pull in newlib-nano
and stub syscalls so printf/malloc link without an OS underneath.
- Linux userland armv6 (arm-linux-gnueabihf-gcc, glibc, Raspberry Pi
1-class hardware under qemu-arm): needed no changes at all \u2014 the
existing generic non-x86_64 Makefile branch (BINDIR=bin/$(ARCH),
DRV=qemu-$(ARCH)) already handles any hosted Linux cross target.
Portability fix: source/algos/include/*.h previously keyed all
freestanding-target checks off __AVR__ specifically. Replaced with
the standard __STDC_HOSTED__ (0 under -ffreestanding, which the
Makefile now adds for every FREESTANDING target), so future
freestanding architectures need no header changes, only a Makefile
branch if they need extra flags.
- GNUmakefile:
- TARGET_OS (2nd triple component) identifies "no OS" targets
(arm-none-eabi has TARGET_OS=none); combined with ARCH=avr this
sets FREESTANDING, which now gates -ffreestanding, the
HELPERS/good.lst/asan.lst skip, and the hpbm -fopenmp skip
(previously all hardcoded to ARCH==avr).
- simdkr.c (x86 SSE/AVX-only) moved from the AVR-only exclusion
list into NON_SSE for both the aarch64/arm64 and generic
non-x86_64 branches: it was already silently broken on any real
non-x86_64 machine (e.g. actual aarch64 hardware), not just AVR.
- NON_AVR list unchanged otherwise (still avr-gcc's 32767-byte
object-size limit / SRAM-fit issues; ARM has neither problem by
default, so no NON_ARM exclusion list is needed).
- source/algos/include/main.h: system libc headers (newlib, used by
arm-none-eabi) redefine __nonnull/__nonnull_all for their own
internal prototypes when stdio.h etc. are included, silently
shadowing include/define.h's (already-computed) definitions since
define.h is included first. Restore them right after the system
includes so the project's own definitions are what's actually used
for the rest of the translation unit; this was breaking musl1.c/
musl2.c on arm-none-eabi with 'attributes should be specified
before the declarator'.
Verified:
- make CC=avr-gcc BINDIR=bin/avr-gcc: unchanged, 223/223 (regression
check after the __STDC_HOSTED__ refactor).
- make CC=arm-none-eabi-gcc BINDIR=bin/arm-none-eabi: 268/269 build
(all but the SSE-only simdkr); avr-size-equivalent ~8KB flash/512B
bss per binary.
- Wrote a minimal Cortex-M0 vector-table+Reset_Handler harness
(bypassing newlib's non-bootable-without-a-BSP default startup)
linking one of the compiled search() implementations directly;
booted under qemu-system-arm -M lm3s6965evb and confirmed the
correct occurrence count via semihosting exit code.
- Wrote a freestanding armv6 Linux syscall (svc) test exercising the
same search logic; ran correctly under qemu-arm user-mode
emulation, validating the toolchain+qemu-arm path the existing
generic Makefile branch relies on for Linux/glibc armv6 targets
independent of a full sysroot (not installable via dnf on this
host, so make CC=arm-linux-gnueabihf-gcc itself is untested
end-to-end; only the generic branch it exercises needed no code
changes).
- Native x86_64: full make -j4 rebuild, 0 real errors (grep noise
was only the -Wfatal-errors flag string); ./smart -simple ... and
./test musl1 -nv rand2 2 both ran correctly.
STM8 (sdcc-sdcc -mstm8):
SDCC is not a GCC driver: no -dumpmachine (would corrupt ARCH/TARGET/
TARGET_OS if called), no -ffreestanding (silently ignored), and -Wall
is parsed as SDCC's own -W<p|a|l><opts> passthrough syntax, i.e.
'-Wall' means 'pass the raw flag "ll" to the assembler' and breaks
the build. GNUmakefile detects sdcc by name and skips GNU-triple
detection entirely, synthesizes ARCH=stm8/TARGET_OS=none (routing it
through the existing FREESTANDING machinery), and resets CFLAGS to
-mstm8 -D__STDC_HOSTED__=0 instead of inheriting the GCC-flavored
ones. Per the ask, only the algorithm binaries are built
(all: $(BINS), already true for every FREESTANDING target).
Three portability fixes in shared headers, all gated so no other
target's behavior changes:
- main.h: SDCC's libc has no exit() and requires printf/puts to have
a user-supplied putchar(); stub both under #ifdef __SDCC (matches
how newlib's --specs=nosys.specs already covers arm-none-eabi, and
avr-libc needs neither).
- main.h: sys/types.h is now gated behind __STDC_HOSTED__ (nothing in
source/algos actually needs a type from it; avr-libc/newlib happen
to ship a harmless one, SDCC doesn't ship one at all).
- log2.h: LOG2()'s __builtin_clz fallback was gated on the glibc
hosted-target macro _WORDSIZE==32, an unrelated proxy that happened
to work only because every prior GCC target here is __GNUC__ with
_WORDSIZE defined consistently. Gated on __GNUC__ directly (the
actual capability being tested), with the existing portable
FIRSTBIT()/leftbit[] table as the non-GCC fallback.
- GRAPH.h: dropped a stray top-level ';' after TableGraph() that
GCC silently tolerates but SDCC's stricter parser rejects.
NON_STM8 excludes 69 algorithms on top of the 4 already in NON_SSE:
SIGMA*SIGMA/DSIGMA-sized preprocessing tables that don't fit an 8-bit
MCU (SDCC reports these as a hard 'Cannot allocate variable' error,
or worse, a multi-minute hang in its register allocator instead of
avr-gcc's immediate object-size error), plus memmem() (libc1, a GNU
extension SDCC's libc lacks), VLAs (lhc1..8, C99 SDCC doesn't
support), and a SDCC internal-compiler-error on __builtin_ffs (svm0,
not fixed like log2.h's since there's no existing portable fallback
to reuse for it and it's a single file).
mips32 (mips64-linux-gnu-gcc -mips32 -mabi=32) and riscv32
(riscv32-linux-gnu-gcc): both are ordinary hosted Linux/glibc targets
under qemu-user, exactly like the armv6-linux case already covered by
the generic non-x86_64 Makefile branch (BINDIR=bin/$(ARCH),
DRV=qemu-$(ARCH)) plus the NON_SSE list (already carries simdkr) -
needed no Makefile changes at all.
Verified:
- make CC=sdcc-sdcc BINDIR=bin/stm8: 199/199 planned algorithms build
(272 total - 4 NON_SSE - 69 NON_STM8), exit 0, real Intel-HEX
firmware images ~8-12KB each.
- Wrote a minimal STM8 smoke program (search() + result byte at
address 0, HALT) and ran it under SDCC's own uCsim (sdcc-ucsim_stm8
-t stm8s208): CPU correctly reaches the HALT instruction, memory
dump at 0x0000 reads 0x02 - the correct match count for the same
pat="abc" in txt="xxabcxxabcxx" fixture used in every other
target's smoke test this session.
- mips32: freestanding C (search() + raw syscall exit, no libc
needed) compiled with mips64-linux-gnu-gcc -mips32 -mabi=32 -EB,
ran correctly under qemu-mips (real O32 big-endian MIPS32, not just
hand-assembled asm - confirmed with a second, purely hand-written
.S version first, then the C-pipeline one, both correct).
- riscv32: same freestanding-C approach compiled with
riscv32-linux-gnu-gcc, ran correctly under qemu-riscv32.
(Neither mips64-linux-gnu nor riscv32-linux-gnu has a Fedora
cross-sysroot package, so make CC=<...>-gcc itself is untested
end-to-end against the real source/algos/*.c files for these two,
same caveat as armv6-linux previously; the toolchain+qemu-user
mechanism the generic branch relies on is confirmed correct.)
- Native x86_64: full make -j4 rebuild, 0 real errors.
- make CC=avr-gcc BINDIR=bin/avr-gcc: unchanged 223/223 (regression
check after the log2.h/GRAPH.h/main.h header edits).
- make CC=arm-none-eabi-gcc BINDIR=bin/arm-none-eabi: unchanged
268/268 (same regression check).
Previous commit claimed mips32/riscv32 needed no Makefile changes, verified only with hand-written freestanding syscall smoke tests because no cross-sysroot was available via dnf. Downloaded real, self-contained musl-libc cross toolchains (mips-linux-musl-cross, riscv32-linux-musl-cross from musl.cc) to actually compile source/algos/*.c, and found a genuine bug the freestanding tests couldn't have caught: timer.h's "#if defined(__linux__): #define clock_gettime(id, ts) syscall(SYS_clock_gettime, (id), (ts))" is a workaround from the project's very first commit (pre-2012 glibc < 2.17 needed -lrt for clock_gettime()). SYS_clock_gettime doesn't exist on riscv32 (a "time64-only" Linux ABI with no legacy 32-bit-time_t syscalls at all) and was renamed to SYS_clock_gettime32 on mips32/arm32/etc (post-y2038 time64 transition), so this broke source/algos/bf.c (and everything else using BEGIN_PREPROCESSING/BEGIN_SEARCHING) with "'SYS_clock_gettime' undeclared" on both new targets. Modern glibc (>=2.17, 2012) and musl both provide clock_gettime() as a plain libc function needing no raw syscall/-lrt workaround, and time.h is already included unconditionally above, so the workaround is simply obsolete: removed, falling through to the standard declaration. Verified against the real source/algos/*.c (272 total, 4 already in NON_SSE, 268 remaining): - mips-linux-musl-gcc (mips-linux-musl target, genuine MIPS32 O32 big-endian, musl libc): 268/268 compile; full `make CC=mips-linux-musl-gcc BINDIR=bin/mips32 CFLAGS="-O3 -Wall -DNDEBUG -static"` succeeds; ./bin/mips32/bf and ./bin/mips32/kmp under qemu-mips both print "found 2 occurrences" (correct) for the same abc/xxabcxxabcxx fixture used throughout this session, exit code 2. - riscv32-linux-musl-gcc: same - 268/268 compile, full make succeeds, qemu-riscv32 runs bf and kmp correctly. - Regression-checked every other target after this shared-header change: native x86_64 (0 errors, ./smart -simple ran correctly), avr-gcc (223/223 unchanged), arm-none-eabi-gcc (268/268 unchanged), sdcc-sdcc/stm8 (spot-checked bf.c; the removed code was already dead for STM8, gated behind __STDC_HOSTED__==0). No Makefile changes needed for mips32/riscv32 beyond what's already committed (still the plain generic non-x86_64 branch); this was purely a shared-header portability bug that real hardware/qemu testing surfaced and the hand-written smoke tests structurally could not, since they never touched timer.h.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix all the algos, and the build and test system.
document all the constraints and add fallbacks.
add formal verification tests.
add all the currently known new algos. (just not multi patterns matchers, this would be out of scope)