Simplify fkl-python: generic IO IOps + let C++ fuse_back fuse (Oscar review) - #1
Conversation
Point 1 (Oscar review): the initial/final IOps were special-cased by a
closed if/elif ladder in generate_cu() keyed on op .name, which silently
only covered a handful of memory ops and duplicated logic that belongs on
the IOp itself. Move each read/write op's host-side buffer construction
into emit_read()/emit_write() on the descriptor, so a Read/Write IOp is
self-contained (mirroring how the C++ side carries the buffer inside the
IOp type). generate_cu now just calls read_op.emit_read()/write_op.emit_write().
Also drops the dead TensorRead/Write.cpp() bodies (in_tensor/out_tensor /
input.ptr()) that codegen overrode and never called.
Verified emitted C++ is byte-identical to HEAD across all 9 IO variants
(Ptr2D, Tensor-planes, TensorSplit, TensorTSplit, SplitWrite, TensorPack,
ReadSet, BorderReader+Crop, batch Crop) x {gpu,cpu}. Full suite green:
operations, vertical/horizontal fusion, niche, roi, e2e, circular, dlpack,
torch. No CODEGEN_VERSION bump needed (output unchanged -> cache reused).
…splice) Point 2 (Oscar review): executeOperations() already runs BackFuser::fuse_back(iOps...) internally (executors.h), and BorderReader's value-less build() returns an IncompleteReadBack that fuse_back detects and fuses with the preceding Read via fk::fuse. So Python splicing the read expression into BorderReader was reimplementing fusion C++ does for free. Now replicate/reflect/wrap/reflect101 emit a plain IncompleteReadBack IOp in the flat list and let the library fuse it — verified numerically (niche replicate+OOB-crop still matches the CPU reference, with NO Python splice). CONSTANT stays on the read-splice path: FKL's incomplete-const builder BorderReader<CONSTANT>::build(value) does NOT compile (border_reader.h:72 passes NullType where a backIOp is required; reproduced in isolation), so the library only supports the complete build(readIOp, value) form. Kept a minimal _needs_read splice for that single mode, clearly scoped and documented. Renamed the generic _fuse_with_read marker to _needs_read to reflect that it is now a narrow FKL-limitation workaround, not the general border path. CODEGEN_VERSION 7 -> 8 (emitted C++ for value-less borders changed).
Extra: arreglados los 3 fallos pre-existentes de DivergentHFAprovechando el PR, arreglé también las 3 fallas de Causa: off-by-one en el selector de secuencias.
|
Contexto
Punto 1 — IO genérica en los descriptores de IOp
generate_cu()construía el buffer de entrada/salida con una escaleraif/elifcerrada keyed enop.name, que solo cubría un puñado de ops de memoria y duplicaba lógica que pertenece a la propia IOp._ReadOp) implementa ahoraemit_read(state, mem, n_inputs) -> (in_decl, read_expr): construye su propio objeto de buffer host (Ptr2D / Tensor /std::array<Ptr2D,B>/ TensorPack / ReadSet) y elbuild()que lo consume. Una Read IOp es autocontenida, igual que en C++ el buffer viaja dentro del tipo de la IOp._WriteOp) implementaemit_write(state, mem, pbase)(TensorWrite / TensorSplit / TensorTSplit / SplitWrite).generate_cu()ya no tiene escalera: solo llamaread_op.emit_read()/write_op.emit_write().TensorRead/Write.cpp()(in_tensor/out_tensor,input.ptr()) que el codegen sobreescribía y nunca llamaba.Verificación: el C++ emitido es byte-idéntico a
mainen las 9 variantes de IO (Ptr2D, Tensor-planes, TensorSplit, TensorTSplit, SplitWrite, TensorPack, ReadSet, BorderReader+Crop, batch Crop) × {gpu, cpu}. Como el output no cambia, la caché de compilación se reutiliza (sin recompile storm) y no hace falta bump deCODEGEN_VERSION.Punto 2 — Dejar que C++
fuse_backfusioneexecuteOperations(stream, iOps...)ya llama internamente aBackFuser::fuse_back(iOps...)(executors.h). Elbuild()sin valor de BorderReader devuelve unIncompleteReadBack, quefuse_backdetecta y fusiona con la Read previa víafk::fuse. Python splicear la expresión de read dentro del BorderReader estaba reimplementando una fusión que C++ hace gratis.replicate/reflect/wrap/reflect101→ ahora emiten una IOpIncompleteReadBackplana en la lista y la librería las fusiona. Verificado numéricamente (replicate + crop OOB sigue casando con la referencia CPU, sin splice en Python).BorderReader<CONSTANT>::build(value)no compila upstream (border_reader.h:72pasaNullTypedonde se requiere un backIOp; reproducido en aislado con nvcc). La FKL solo soporta la forma completabuild(readIOp, value). Para ese único modo se mantiene un splice mínimo (marcador_needs_read), claramente acotado y documentado. Todos los demás modos van por el camino limpio defuse_back.CODEGEN_VERSION7 → 8 (cambia el C++ emitido de los borders sin valor).Tests (venv del proyecto, GPU RTX PRO 6000 sm_120, nvcc 13.3)
Verde: operations, vertical/horizontal fusion, niche (ambos borders), roi, e2e, circular, dlpack, torch, flash_attention (23), cpu_backend, matrix, thread_fusion.
Las 3 fallas de
test_batch_divergent_hf(DivergentHF: …) son pre-existentes enmain(bug de selector upstreamDivergentBatchTransformDPP, camino de código distinto, no tocado en este PR). Idénticas antes y después.El tercer commit actualiza la skill
fkl-python-extending(arquitectura emit_read/emit_write + el matiz de fusión de BorderReader).