Skip to content

refactor(io): unify ionic-step markers and output layout for NAO/LCAO outputs - #8293

Merged
mohanchen merged 138 commits into
deepmodeling:developfrom
mohanchen:2026-11-05-a
Nov 21, 2026
Merged

mohanchen merged 138 commits into
deepmodeling:developfrom
mohanchen:2026-11-05-a

Conversation

@mohanchen

@mohanchen mohanchen commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator

Linked Issue

N/A — output-format unification tracked in this branch (no single upstream issue).

What's changed?

This PR unifies how ABACUS labels, headers, names and schedules its per-step
outputs, and makes the LCAO/NAO writers share one layout.

1. One ionic-step marker everywhere

  • Every step record now says Ionic_Step: # Ionic_Step N in scalar text
    files, bare Ionic_Step N in CSR, cube and dense matrix headers.
  • Replaces the previous mix of STEP: N, istep:, IONIC STEP, Step.
  • Each frame starts with a self-describing provenance header (step, quantity,
    spin count/index, basis size, number of non-empty R vectors, cell block),
    emitted through the new provenance_header / dense_text_body helpers.

2. One sparse-CSR layout

  • H(R), S(R), dH/dR, dS/dR, r(R) (per Cartesian axis), dH(k), T(k) and the
    angular-momentum matrices now share the same banner and block layout.
  • Text bodies are written row-major, fixing the transposed DM(k)/T(k) bodies.
  • Empty Bravais-lattice R blocks are skipped and reported in the header.
  • Spin/step suffixes fixed: s{spin} only for nspin = 2, g{step} only for
    out_app_flag = 2; redundant spin suffixes dropped from S(k) and dS/dR.

3. New, documented file names

  • vxcrs{spin}_nao.csr (real-space Vxc), vxcmok{ik}s{spin}_nao.txt
    (MO-basis Vxc, avoiding the old vxck collision), rrx/rry/rrz_nao.csr,
    dhk{k}{xyz}_ia{I}[s{spin}][g{step}]_nao.txt, xc_{rho,amag,sigma}*.cube.
  • eband_vxc*.txt moved next to the matching matrix output directory.

4. Output directory and cadence policy

  • out_freq_ion is the single cadence for bands, eig_occ.txt and
    eband*, including out_app_flag = 2.
  • out_app_flag modes are now consistent: 0 overwrites, 1 appends one frame
    per written step, 2 writes numbered files under the matching subdirectory
    (STRUC/, CHG/, POT/, ELF/, DOS/, DIP/, MAT/, BANDS/).
  • The final ionic step is always persisted; in RT-TDDFT the override is
    restricted to the last electronic substep.
  • New System_para::global_band_dir (BANDS/ under mode 2); MAT/ is created
    before get_s mode-2 writes; xc(r) cubes are routed through CHG/.
  • H(k) term files are truncated at the first ionic step and only announced
    after a successful write.

5. Restart and validation hardening

  • LCAO DM(k) restart reads the format that is actually written (streaming),
    rejects mixed ionic steps in mode 0, non-finite values, invalid stored k
    coordinates and non-positive text precision.
  • DMR presence is broadcast so all ranks follow the same collective path;
    UnitCell parse failures are propagated instead of being read as coordinates.

6. ASE interface

  • eig_occ frames are matched by their Ionic_Step ID (legacy markers still
    parsed), files are discovered for every out_app_flag mode, and a stale
    frame is never reported for an announced run.

7. Tests, docs and governance

  • New unit tests: write_dmk, dense_text_body, frame_append,
    libxc_r_suffix, dense H(k)/dS-dR and eig_occ coverage.
  • All touched legacy test sources moved into flat unittests/ directories
    (source_base, source_cell, module_hcontainer, source_io,
    module_lr); tmp_mocks.cpp became an inline tmp_mocks.h.
  • Integration references updated (02_NAO_Gamma, 03_NAO_multik,
    .ref fixtures) for the new markers and CSR layout.
  • Global dependency budget: net delta is now negative (-21) instead of +14.
  • Docs updated: dos.md, hs_matrix.md, hybrid_nscf.md,
    position_matrix.md, and the affected INPUT descriptions.
  • CI: README-only PRs are gated by a metadata workflow.

Migration notes for parsers

  • Match the Ionic_Step keyword case-sensitively; the old STEP:/istep:
    forms are no longer written.
  • vxck* is renamed to vxcmok*; vxcrs{spin}_nao.csr replaces the old
    Vxc(R) sparse names; dH(k) files now always carry the k index.
  • With out_app_flag = 2, band and matrix outputs live in BANDS/ and MAT/
    and carry a g{step} suffix.

Unit Tests and/or Case Tests for my changes

Build and run locally (BUILD_TESTING=ON, ENABLE_MPI=ON), all passing:

  • MODULE_IO_write_dmk (26), MODULE_IO_write_dmr_test (3),
    MODULE_IO_write_orb_info, MODULE_IO_hsr_writer_test (24),
    MODULE_IO_hs_sparse_io_test, MODULE_IO_dhs_sparse_writer_test,
    MODULE_IO_angmom_op_mat_test, MODULE_IO_pos_op_csr_test
  • MODULE_CELL_index_test (2), MODULE_BASE_csr_reader (68),
    MODULE_BASE_filename (10)
  • MODULE_LCAO_hcontainer_test (14) and the complex/folding/transfer/output/
    add_value hcontainer targets
  • MODULE_LR_dm_regressions (3) and the other module_lr targets
  • MODULE_IO_read_item_serial (33), MODULE_IO_input_domain_reader (19),
    MODULE_IO_rho_io (6), MODULE_IO_system_variable_test (3)
  • ASE: interfaces/ASE_interface/tests/test_eig_occ_files.py

Integration references refreshed for the NAO Gamma / multi-k cases.

Governance

  • agent_governance_check.py --base upstream/develop --head HEAD: 0 errors
    (previously 86); remaining findings are review-only warnings for global
    references (GlobalV::ofs_running, MY_RANK in output writers) and
    standard headers.

  • New/changed tests live in flat unittests/ directories with test_ names.

  • I have read the contributing guidelines.

  • My changes do not require additional documentation (docs updated above).

abacus_fixer added 12 commits November 5, 2026 15:08
…out_app_flag=2

The band output did not honor out_app_flag=2: the filename never received
a g{step} suffix and always wrote into global_out_dir, so multi-step runs
(MD/relax) would overwrite bands1.txt / bands2.txt and only keep the last
step.

Changes:
- Add global_band_dir to System_para, mapped to OUT/<suffix>/BANDS/ in mode 2
  and falling back to global_out_dir otherwise; includes MPI broadcast.
- write_bands: use global_band_dir as the output directory, add g{istep+1}
  suffix when out_app_flag==2, and lazily ensure the BANDS/ directory exists.
- Tests: sync io_sys_var_test directory assertions and test_write_bands
  TestParameters with the new global_band_dir.

Verified: MODULE_IO_write_bands (10/10), MODULE_IO_system_variable_test pass;
esolver builds cleanly; governance check has no new blockers.
Replace the legacy "RELAX STEP" keyword and pre-existing inconsistent
"Ionic Step" (with space) occurrences with "Ionic_Step" (underscore) to
match the established convention already used by write_cube.cpp and
related integration-test fixtures.

Production code:
- source_relax/relax_stru_io.cpp: STRU file provenance header
- source_io/module_output/print_info.cpp: stdout/ofs_running run logs
- source_io/module_hs/vxc_op_tools.cpp: eband decomposition energy header
- source_pw/module_pwdft/dftu_base_io.cpp: DFTU occ_mat provenance header
- source_io/module_dm/write_dmk.cpp: DM(k) frame header (historical)
- source_io/module_dm/write_dmr.cpp: DM(R) CSR frame header (historical)
- source_io/module_hs/hsr_writer.cpp: H(R)/S(R) CSR frame header (historical)
- source_io/module_mulliken/output_mulliken.cpp: Mulliken output frame header (historical)

Tests:
- source_relax/unittests/test_relax_stru_io.cpp
- source_io/unittests/test_print_info.cpp
- source_io/module_dm/test/write_dmk_test.cpp
- source_io/module_dm/test/test_write_dmr.cpp
- source_io/module_hs/unittests/test_hsr_writer.cpp

Comments / doc strings:
- source_base/module_out/csr_reader.h: CSR format example
- source_io/module_parameter/read_inp_out.cpp: out_stru parameter description

Verification:
- make -j 30 -> 100%
- ctest -R "(RELAX_relax_stru_io|IO_print_info|IO_vxc_op_tools|PW_dftu_base|
  IO_mulliken|IO_write_dmk|IO_write_dmr|IO_hsr_writer)" -> 10/10 PASSED
- grep -rn "RELAX STEP" / "Ionic Step " -> zero matches in source tree
Production code:
- source_io/module_wf/write_wfc_nao.cpp: wf{ki}s{spin}_nao.txt provenance header
- source_io/module_energy/write_bands.cpp: band eigenvalue provenance header

Tests:
- source_io/unittests/test_write_wfc_nao.cpp
- source_io/unittests/test_read_wfc_nao.cpp
- source_io/unittests/test_write_bands.cpp

Verification:
- make -j 30 -> 100%
- ctest -R "(IO_write_wfc_nao|IO_read_wfc_nao|IO_write_bands|
  RELAX_relax_stru_io|IO_print_info)" -> 8/8 PASSED
- grep -rn "RELAX STEP\|Ionic Step \|IONIC STEP" -> zero matches
Three source output files wrote "# ionic step" (or "X # ionic step") in
their per-ionic-step header comments, while STRU and cube outputs already
use "Ionic_Step". Rename to the unified keyword across:
- cal_dos.cpp      -> "X    # Ionic_Step" (doss*.txt / pdoss*.txt)
- write_eig_occ.cpp -> "X     # Ionic_Step" (eig_occ.txt)
- hs_dense_io.cpp  -> "# Ionic_Step X"  (HS dense matrix)

Sync test assertions and INPUT parameter description accordingly.
Also refines the INPUT file under examples/17_relax/04_relax_with_output_lcao
with end-of-line filename comments for each enabled out_* parameter.
The prior commit 4038796 unified "# ionic step" to "# Ionic_Step" in
hs_dense_io.cpp but missed test_hs_dense_io.cpp, causing TextHeaderContains
StepAndDimension and TextHeaderStepTwo to fail. Sync the EXPECT_NE find()
strings and a nearby comment.

Verification: MODULE_IO_hs_dense_io_test -> 11/11 PASSED.
Three writers produced per-frame headers as "STEP: N" or "IONIC_STEP: N":
- pos_op_csr.cpp       (rr_nao.csr, rx_nao.csr, ...)
- hs_sparse_io.cpp     (hrs*.csr, srg*.csr, trs*.csr, ...)
- dhs_sparse_writer.cpp (dhrx*.csr, dsrx*.csr, ...)
- overlap.cpp          (S_async(R) overlap)

Rename to "Ionic_Step N" (keyword-first, no colon) to match STRU cube,
doss, eig_occ, and HS dense-matrix headers unified in the previous commit.

Sync test assertions, fixture SR.csr, and comments accordingly.
PDOS text output (write_pdos_text.cpp) used "# istep:" as its per-step
header keyword while all other outputs had already been unified to
"Ionic_Step". This was the last remaining non-Ionic_Step keyword in
production output.

Production code:
- source_io/module_dos/write_pdos_text.cpp

Tests:
- source_io/module_dos/unittests/test_write_pdos_text.cpp (5 assertions)

Docs:
- docs/advanced/elec_properties/dos.md (PDOS format example and description)

Verification:
- make -j 30 -> 100%
- MODULE_DOS_write_pdos_text -> 5/5 PASSED
hs_dense_io.cpp writes "# Ionic_Step N" since the keyword unification, but
the hk/sk/vxc/dH reference files still contained "# ionic step". CompareFile.py
compares comment lines token by token, so those matrix comparisons would fail.
Update 31 references under tests/; value data is unchanged.
out_mat_r writes a single rr_nao.txt (or MAT/rrg{step}_nao.txt when
out_app_flag = 2) rather than separate rxrs/ryrs/rzrs CSR files. Update
the INPUT description, the position-matrix guide, and the example INPUT
comment, and align the documented frame header with the emitted
Ionic_Step keyword.
Tensor quantities (sigma, vsigma, v2rho2, v2rhosigma, v2sigma2, v3rho3,
v3rho2sigma, v3rhosigma2, v3sigma3, v4rho4, v4rho3sigma, v4rho2sigma2,
v4rhosigma3, v4sigma4) previously used _s1/_s2/_s3 suffixes that collide
with the real spin-index convention used by rho and vrho.  Replace them
with explicit Libxc-channel index tuples (_00, _01, _11 for 3-channel
tensors, etc.).  Scalar quantities (exc, amag) no longer carry a
redundant _s1 suffix.  Rho and vrho keep _sN since those map to actual
spin indices.  A set of small helper lambdas generates the suffix lists
so the channel ordering stays consistent with Libxc throughout.
Follow the repository NAO sparse-matrix naming convention: one-based
spin index, "_nao.csr" suffix, and the g{step} suffix plus append
behavior driven by out_app_flag in mode 2. Pass the ionic step and
out_app_flag explicitly into write_Vxc_R instead of hard-coding
istep = -1. Update the out_mat_xc2 help text, the example INPUT
comment, and the scf_out_hxc reference fixture (including its
Ionic_Step header).
@mohanchen mohanchen added Input&Output Suitable for coders without knowing too many DFT details Refactor Refactor ABACUS codes The Absolute Zero Reduce the "entropy" of the code to 0 Solve before 3.11 labels Nov 5, 2026

@ABACUS-PR-BOT ABACUS-PR-BOT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The output changes introduce mismatched derivative labels, a CSR step-reader regression, stale Vxc records on reruns, and unmigrated integration headers. The global-dependency budget and three non-blocking documentation/test items also need attention. Static source and API-contract review of 63431111; no builds or tests were run.

Comment thread source/source_io/module_hs/vxc_op_r.cpp Resolved
abacus_fixer added 3 commits November 6, 2026 14:44
The earlier commit 91465ba updated 31 .txt.ref files under tests/ for
the Ionic_Step unification, but left .csr.ref, rr_nao.txt.ref,
mulliken.txt.ref and cube.ref fixtures still carrying the legacy
"--- Ionic Step N ---", "STEP: N" or "IONIC_STEP: N" headers.
CompareFile.py does a token-by-token compare and those mismatches cause
the integrating tests (md_out_hk_syns, scf_out_hk_spin2, scf_out_mul,
scf_out_mul_spin2, ...) to fail. Update 41 remaining reference files
under tests/02_NAO_Gamma/ and tests/03_NAO_multik/ to the new unified
"Ionic_Step N" keyword. Data values are untouched.
The eig_occ.txt per-ionic-step header was unified from "X     # ionic
step" to "X     # Ionic_Step", but the ASE reader still counted frames
with endswith('# ionic step'), yielding nframe = 0 and failing the
assertion len(ik) == nframe * nspin * nk in read_band_from_eig_occ.

Match the header with a case-insensitive regex that accepts both the
old and the new spelling, and sync the nspin4-gamma-eigocc fixture to
the current format.
The out_band BANDS/ change read PARAM.globalv.global_band_dir inside
write_bands, which pushed the PR over the global dependency budget
(added=4, removed=3, net_delta=1) and blocked CI.

Resolve the directory once in ESolver_KS::after_scf and pass it as an
explicit argument, following the existing write_eig_file /
output_mat_sparse convention. This keeps the band writer free of PARAM
and brings the PR-wide delta to zero.

Verified: MODULE_IO_write_bands 2/2 passed; abacus_basic_para target
builds; governance diff counting gives net_delta=0.

@ABACUS-PR-BOT ABACUS-PR-BOT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reader now accepts both step-header spellings. The suggestion below covers the legacy format. Static incremental review; builds and tests were not run.

abacus_fixer added 8 commits November 7, 2026 11:33
The non-MPI-specific tests in test_hsr_writer.cpp performed file I/O
on all MPI ranks simultaneously, causing file corruption and race
conditions when the test binary was launched with mpirun -np 2.

- For the 7 tests that call the non-collective write_hcontainer_csr /
  write_hcontainer_csr_binary directly, add an early return on
  non-zero ranks so only rank 0 writes/reads/removes the files.
- For WriteHsrTextCsrCarriesPerSpinFermiFromEfermi, keep the collective
  write_hsr call (which internally uses gatherParallels) on all ranks,
  but guard the file read-back, assertions, and cleanup to rank 0 only.

Verified: single-process (21 passed, 1 skipped) and mpirun -np 2
(22 passed on both ranks), plus ctest serial+parallel both pass.
The MO-basis Vxc matrix written by out_mat_xc reused the "vxc" property,
which resolved to the same vxck{ik}s{spin}_nao.txt filename as the
NAO-basis matrix from out_mat_vxck; with out_app_flag 0/1 the later
writer silently overwrote the former. Give the MO-basis matrix its own
"vxcmo" property and pass the real ionic step and out_app_flag into
write_Vxc instead of locally shadowed constants, which restores the
one-based Ionic_Step header and the mode-2 g{step} suffix. Update the
two integration fixtures and props_mat.sh accordingly.
write_Vxc_R is called from ctrl_runner_lcao, which runs once at the end
of the whole calculation, so out_app_flag=2 only ever produced a single
vxcrs{spin}g{N}_nao.csr file. Drop the per-step suffix and always
truncate, matching the documented vxcrs{spin}_nao.csr name, and remove
the now unused out_app_flag parameter.
write_h_term looped both spin channels while write_hk_common enumerated
only the spin-up k indices, so Vnl(k)/Vl(k)/Vh(k) were written twice
into s1 files, and Vxc(k), which is spin dependent, placed both channels
in the same s1 file. Enumerate the local k points of each spin through
K_Vectors::isk as write_hsk does, pass nspin=1 to filename_output for
spin-independent terms so they carry no spin suffix, and apply the same
enumeration to the EXX writer and to T(k).
Emit the column header only once per file and stamp each frame with
"# Ionic_Step <1-based step>" so the angular-momentum matrices match the
step labelling of the other matrix outputs. Appending frames no longer
repeats the full header.
Mode 0 previously kept every written frame and mode 2 ignored the step
number. Open the file per the selected mode: overwrite for mode 0, append
after the first step for mode 1, and write mullikeng{step}.txt in mode 2.
write_eig_file always appended after the first step and ignored the ionic
output frequency, so mode 0 kept every frame and mode 2 had no step suffix.
Gate the write by out_freq_ion and open the file per out_app_flag: overwrite
in mode 0, append after the first step in mode 1, and write eig_occg{step}.txt
in mode 2.
out_band was dumped on every ionic step regardless of out_freq_ion, so a
run with out_freq_ion = 2 produced five band frames while the other
step-scoped outputs produced three. Gate write_bands with the same
frequency used by eig_occ.txt; a non-positive frequency still writes
every step.
abacus_fixer added 8 commits November 19, 2026 15:33
Change the dense dH/dk filename to put the 1-based k-point index right
after the prefix and use _ia for the atom index: dhk1x_ia1_nao.txt
replaces dhkx_iat1_ik0_nao.txt. The _nao suffix is kept. Update the
unit test and the scf_out_dh / scf_out_dh_t reference files and
result.ref keys to match.
Check strtod range errors and finite components so malformed restart
matrices cannot propagate infinity or NaN into the density.
The DM(k) text files produced by write_dmk carry D(i,j) = sum_n w_n
conj(c_n,i) c_n,j because dm_from_psi builds DM = conj(P) (the physical
one-body density matrix P with the complex conjugate on the first band
index). This convention was only implicit in a consumer-side note at
source/source_io/module_mulliken/output_mulliken.cpp ~L37 (added in PR
deepmodeling#8250), so restart readers and external post-processing tools could
silently mix conj-first and conj-last interpretations.

- Emit a '#' comment line inside the 11-line header of every DM(k) file
  stating the exact element convention. Placing it inside the header block
  lets read_dmk's read_data_line skip it transparently; put it after the
  UnitCell block would break the matrix-body stream extractor which does
  not skip comment lines.
- Document the same convention in write_dmk's Doxygen @details, including
  the relation to the physical density matrix sum_n w_n c_n,i conj(c_n,j).
- Add a unit test HeaderStatesConjFirstConvention that asserts the header
  comment is present and that a non-symmetric complex probe round-trips
  through write_dmk/read_dmk without losing off-diagonal imaginary signs.
…format upgrade

The hs_sparse_io writer was upgraded in c1ff9a6 ("align tr_nao.csr with
the self-describing CSR layout", 2026-11-09) to emit the header block with
#-comment annotation, lattice info, and CSR format banner. The commit updated
two integration-test refs but did not touch get_s/sr_nao.csr.ref, which was
still in the legacy "Matrix Dimension of S(R)" / "Matrix number of S(R)"
format. A follow-up update at 1500cec ("Align Vxc(R) CSR output with the
H(R) CSR layout") then changed the R count of the two tr_nao.csr.ref files
to values that do not match the case INPUT (125 and 9 vs the actual 9 and 1,
respectively). Both gaps are fixed here by running the three cases under the
current writer and capturing what it emits:

- get_s/sr_nao.csr.ref: legacy -> new self-describing header, R count 3
  (c1ff9a6 did not update this file at all).
- nscf_out_hsr_tr_rr/tr_nao.csr.ref: R count 125 -> 9 (matches c1ff9a6's
  correct value; 1500cec had changed it by hand).
- scf_out_dh_t/tr_nao.csr.ref: R count 27 -> 1 (the case INPUT emits one R
  block; previous ref was R=27 from c1ff9a6, already a case/INPUT mismatch).

Also remove the untracked KPT file left behind under tests/02_NAO_Gamma/ by a
local run. No INPUT changes are present in this commit — the refs now reflect
what the cases actually compute.
…rdinates as DM

UcellIO::read_ucell used to be a void line skipper: when the per-type
atom-count line in a DMK file was malformed (non-numeric, empty), the
std::istringstream loop yielded total_natom == 0, so the function only
consumed the "Direct" marker and left the coordinate rows on the stream.
A subsequent small-matrix body read (e.g. nlocal = 2 gamma) would then
happily consume those coordinate numbers and hand back a bogus density
matrix.  The init_chg = dm restart path trusts read_dmk's return value,
so a silently wrong density would seed the next SCF.

- Change UcellIO::read_ucell signature from void to bool.
- Every getline now checks the stream state; the atom-count line must
  yield at least one non-negative integer summing to a positive total,
  otherwise the block is indeterminate and we bail out.
- read_dmk handles the new return value: failure sets read_success,
  prints a diagnostic, and breaks out of the rank0 body so the existing
  bcast_bool cooperative-failure path propagates the error.
- Add a handwritten unit test ReadDMKRejectsMalformedAtomCount that
  rewrites the per-type count line to a non-numeric token and asserts
  read_dmk returns false without touching the caller buffer.

Test: DMKTest.* 24/24 pass (1 new, 23 existing no regression).
… 2026-11-05-a

Conflicts resolved:
- source/source_io/CMakeLists.txt: merged both sides' object lists, keeping
  parse_args.cpp + upstream's new parse_command_line.cpp + local libxc_r_suffix.cpp
- source/source_md/unittests/test_run_md.cpp: kept upstream's consolidate
  (deepmodeling#8322) RunMDTest/RunMDCadenceTest suite and appended local RunMdFinalStepTest
The old description compressed scalar, spin-resolved and tensor outputs
into one optional-suffix pattern, so users could not derive file names
such as xc_v2rhosigma_0_00.cube. List the three channel forms explicitly
and align the examples with libxc_r_channel_suffixes.

@ABACUS-PR-BOT ABACUS-PR-BOT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The merged MD test target is missing required link dependencies. Changed DMK findings and source-level fixes are updated in the original threads. This review used static source and reference-data analysis; no builds, tests or MPI/CI checks were run.

@AsTonyshment AsTonyshment left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still a few minor issues to address before merging.

abacus_fixer added 7 commits November 20, 2026 08:48
…ion unit

MODULE_MD_final_step was meant to test only the header-only loop predicates
(is_last_step / is_frame_write_step) and links without the heavy MD/cell
stack. After the source_md test consolidate, RunMdFinalStepTest ended up in
the same test_run_md.cpp as prepare_mdcell_from_unitcell, so the
dependency-free target failed to link with undefined references to
UnitCell / MDCell / DomainDecomposition / Run_MD::prepare_mdcell.

Move RunMdFinalStepTest into its own test_run_md_final_step.cpp (only
includes run_md.h + gtest) and point MODULE_MD_final_step at it. The heavy
prepare_mdcell and cadence tests stay in test_run_md.cpp, built by
MODULE_MD_run with the full dependency set.
… helpers

Replace raw MPI calls in test_write_dmk.cpp and test_hs_dense_io.cpp
with the project parallel helpers (ParaWorld::make_mpi + Parallel::barrier
/Parallel::reduce_min), following the world_barrier precedent in
test_read_wfc_nao.cpp. Encapsulate the MPI_Comm_split/free pair in a
test-local RAII guard (SplitCommGuard) since the parallel helpers
deliberately do not own communicator lifetimes. Add para_world.cpp and
para_mpi_func.cpp to MODULE_IO_hs_dense_io_test sources, which lists
files manually instead of linking base.
- Introduce a DMKTest fixture owning the Si UnitCell, replacing 24
  repeated create/delete boilerplate blocks and removing raw delete.
- Add expect_dmk_eq helpers (double/complex) to replace six handwritten
  element-wise EXPECT_NEAR loops in round-trip and mode tests.
- Collapse WriteDMK's six copy-pasted read/assert/remove blocks into a
  table-driven loop, keeping the trailing-newline check.
- Drop Mode1FixedNameAppend: its append semantics are fully covered by
  ReadDMKMode1FrameSelection.
- Compress the // ---- banner comments to one or two informative lines.

Net -238 lines (1817 -> 1579); serial and 2-rank MPI runs both pass.
…ame pattern

The writer embeds the 1-based k-point index right after the prefix
(dhk{k}{xyz}_ia{I}[s{spin}][g{step}]_nao.txt), but the out_mat_dh help
still documented dhk[xyz]_iat{I} with an optional _ik suffix, and the
write_dh.h comment used the same stale pattern. Update both to the real
naming, noting the spin suffix is always present for dH(R) (dhr_gen_fname)
but only for nspin=2 in dH(k) (dhk_gen_fname).
The hybrid-NSCF source resolver picked the latest g{step}/e{iter}
variant independently for each wavefunction file and for eig_occ.txt,
so an interrupted or partially written SCF could mix files from
different ionic steps or electronic iterations and silently build an
inconsistent EXX operator.

The resolver now reports the parsed snapshot tokens, and
prepare_exx_nscf locks the (ionic, electronic) pair from the first
wavefunction and rejects any subsequent wavefunction or eig_occ.txt
from a different snapshot. Step-less fixed names (modes 0/1) keep
working unchanged.
Mode 0 rewrites every k/spin density-matrix file under a fixed name at
each ionic step, so an interrupted write can leave files from different
steps on disk. The reader previously only validated the step marker for
mode 2 (per-step filenames) and would silently assemble a restart
density from mixed steps in mode 0.

read_dmk now records the first parsed Ionic_Step marker as the common
reference and rejects any later k/spin file whose marker disagrees.
Files written before the marker existed carry no step information and
skip the cross-check, keeping backward compatibility.

Add two unit tests: a mode-0 read with a tampered second file must fail
without touching the caller buffer, and a marker-less legacy file must
still be accepted.
The outer ionic driver sets UnitCell::last_ion_step for the whole final
ionic step, but RT-TDDFT runs several electronic substeps inside one
runner() call. ctrl_scf_lcao therefore bypassed out_freq_ion for every
substep of the final ionic step, e.g. md_nstep=1, estep_per_md=5 and
out_freq_ion=4 wrote all six electronic steps instead of 1, 5 and 6.

ESolver_KS_LCAO now combines last_ion_step with a final_electronic_step_
flag that defaults to true, so non-RT solvers are unchanged. The RT-TDDFT
runner sets it only when the current substep is the last one of the final
ionic step (estep == estep_max - 1), including the restart endpoint that
ends the ionic step right after the replay. Deriving the final cumulative
step from md_nstep * estep_per_md inside the output layer would have been
wrong for md_nstep > 1 and for restarted runs.

Verification: both touched translation units compile with the release
flags from build/source/source_esolver/CMakeFiles/esolver.dir/flags.make
(-O3 -DNDEBUG -std=gnu++14 -fopenmp, __LCAO/__MPI/__ELPA).
Docs: no user-facing INPUT or file-format change; runtime output cadence
only, so no documentation update is required.
Tests: not added here; the gating predicate is arithmetic on solver state
and would need a new unit test driver, which is left for a follow-up.

@AsTonyshment AsTonyshment left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few remaining issues have been documented in a separate issue (#8338) for future fixes. I think this PR can be merged once the conflicts are resolved.

@ABACUS-PR-BOT ABACUS-PR-BOT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The snapshot-token check catches unequal source steps, but the two EXX reader defects tracked in #8338 remain. The changed source fixes and remaining requests are updated in their existing threads.

Static review of 8f09ae614cf749468cf5e6f6e1503cc23fc6e9d7: no ABACUS builds, tests or CI verification were run.

abacus_fixer added 2 commits November 20, 2026 14:12
Resolve conflicts between the IO-output unifications on this branch and
upstream changes:

- source_relax: keep the new clang-format style and the Ionic_Step header,
  retaining the ofs_running parameter and write_notice calls added here.
- source_md: drop Run_MD::prepare_mdcell, which upstream moved into
  MDCellAdapter and driver_run.cpp now calls; keep the cadence predicates,
  the write_stru_frame helper, the final-ionic-step marking, and the
  supplementary final STRU frame.
…nges

- Migrate the touched legacy test sources into sibling `unittests/`
  directories (source_base, source_cell, source_hamilt/module_hcontainer,
  source_io, source_lcao/module_lr) and wire them into the matching
  CMakeLists. `module_hcontainer/test/tmp_mocks.cpp` becomes an inline
  header shared by the targets that need those mocks.
- The serial reader targets stay declared in `source_io/test_serial` so
  their translation units keep `__MPI` disabled while their sources live
  in `unittests/`.
- Remove new global dependencies: use the `inp.ks_solver` already passed
  to `ModuleIO::ctrl_scf_lcao`, and resolve the writing rank in the
  `write_dmk` unit test from MPI instead of `GlobalV::MY_RANK`.
- Forward-declare `UnitCell` in `write_orb_info.h` instead of including
  the full cell header.

Build and run: the 12 affected test targets pass.
@mohanchen mohanchen changed the title Unify ionic-step header keyword, cube channel suffixes, and Vxc(R) sparse output filename in IO outputs refactor(io): unify ionic-step markers and output layout for NAO/LCAO outputs Nov 20, 2026

@ABACUS-PR-BOT ABACUS-PR-BOT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The global-reference increase is addressed at 7892035; the new on_io_rank() helper still needs the project MPI wrapper. Static review covered the contributor changes, five merge resolutions and affected callers. Builds, runtime tests and CI were not verified.

abacus_fixer added 3 commits November 20, 2026 21:59
MODULE_LCAO_tddft_snap_psibeta_half_test aborted with
cudaErrorNoKernelImageForDevice (209) on the V100 runner while
MODULE_LCAO_radial_tables_gpu passed in the same run. The workflow wipes
build/ before configuring, so the only compiled artifact carried across
runs is the ccache directory. Its fingerprint covers the c++/nvcc/ccache
versions but not CMAKE_CUDA_ARCHITECTURES, and restore-keys are reused
across commits, so an object built for a higher architecture could be
served for the arch-70 request. The stale image cannot run on sm_70 and
its PTX cannot be JIT-compiled downwards, and WARNING_QUIT aborted the
whole test binary.

Include the architecture in the fingerprint, derive the cmake flag from a
single CUDA_ARCHITECTURES env var, and bump the cache key to v2 so the
already-polluted cache is dropped instead of restored.
@mohanchen
mohanchen merged commit 5ad50c1 into deepmodeling:develop Nov 21, 2026
18 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Input&Output Suitable for coders without knowing too many DFT details Refactor Refactor ABACUS codes Solve before 3.11 The Absolute Zero Reduce the "entropy" of the code to 0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants