batch-groups: read a cold prompt beside the decoding slots, stall 11.9 -> 0.8 s, same work 6.4% sooner (opt-in) - #2326

batch-groups: read a cold prompt beside the decoding slots, stall 11.9 -> 0.8 s, same work 6.4% sooner (opt-in)#2326

SuperDistance wants to merge 7 commits into

Niko1221:mainNiko1221/Strata:mainfrom

SuperDistance:pr/piped-concurrent-readSuperDistance/Strata:pr/piped-concurrent-readCopy head branch name to clipboard