Skip to content

fix(ase): backport bundled calculator and parser corrections - #8319

Open
QuantumMisaka wants to merge 22 commits into
deepmodeling:developfrom
QuantumMisaka:fix/ase-upstream-abacuslite
Open

QuantumMisaka wants to merge 22 commits into
deepmodeling:developfrom
QuantumMisaka:fix/ase-upstream-abacuslite

Conversation

@QuantumMisaka

@QuantumMisaka QuantumMisaka commented Nov 15, 2026 •

Copy link
Copy Markdown
Collaborator

Backports calculator and parser fixes to ABACUS's bundled ASE interface (abacuslite). This PR does not change the engine, its INPUT schema, or CI workflows.

Related issues and provenance

Related: #8165 and #8175. Synced with develop including #8293 (ionic-step markers and eigenvalue file layout). Adapted from QuantumMisaka/abacuslite@39ab1d5 and 5d3da5d; related standalone work is tracked in QuantumMisaka/abacuslite#16 and QuantumMisaka/abacuslite#17.

What's changed?

  • Result collection: SCF, native relaxation, and native MD use the final parsed frame. The readers restore ASE atom order. The calculator reports the log path if no frame was parsed. There is no SCF-only count check or STRU-to-log comparison. Result collection does not read abacus.json, and input writing leaves it untouched.
  • Output readers: Keep the LTS/develop reader choice local to each calculator. Accept paths and caller-owned named streams. Return Cartesian Å positions and cells with the simulation box's periodic boundaries. The develop reader locates current-run eigenvalue files from the log and matches spectra by ionic step. Missing spectra do not block energy or force results. Native MD still requires matching geometry; see the README for companion files and dump frequency.
  • ASE properties: Preserve signed occupations and noncollinear magnetic moments without changing ASE's global property schema. energy and free_energy retain ABACUS's E_KohnSham. energy_without_entropy is optional abacuslite metadata; efermi remains a dictionary alias for ASE's fermi_level.
  • Input and restart: Read species and KPT from the files named in the saved INPUT before applying new filenames. Restart does not parse per-atom STRU options. Full STRU parsing follows native defaults for omitted spin angles and per-atom magnitudes. Preserve public writer imports, representable species masses, velocity units, and fractional explicit KPT weights, including labelled rows. New mass or KPT validation leaves the corresponding existing file intact when it rejects input. Fixed-density calculations use the parent output directory unless a path is supplied.
  • Configuration: Omitted profile directories can fall back to ABACUS_PP_PATH and ABACUS_ORB_PATH. ASE 3.23 is the minimum version for the bundled calculator.

Unit and case tests

Verified 34c1dd89e against develop at c46ae1ede. The PR diff remains limited to seven existing files. The existing string-path test now uses a complete SCF log rather than requiring a missing-spectrum error; no regression files, fixtures, or workflows were added by this PR.

With Python 3.12 and each of ASE 3.23.0 and 3.29.0:

export PYTHONPATH="$PWD/interfaces/ASE_interface" OMP_NUM_THREADS=1
python -m pytest -q interfaces/ASE_interface/abacuslite/{core.py,io/generalio.py,io/latestio.py,io/legacyio.py,utils/ksampling.py} interfaces/ASE_interface/tests/test_eig_occ_files.py -k 'not test_calculator_results and not test_restart'
python3 tools/03_code_analysis/agent_governance_check.py --base c46ae1ede --head 34c1dd89e --format text
git diff --check c46ae1ede...34c1dd89e
  • 64 passed, 2 skipped, 2 deselected per ASE version. Governance and whitespace checks passed. The bundled wheel was built from a clean archive, installed outside the checkout, and used to replay the recorded develop runs.
  • Against the bundled source, ran python -m pytest -q tests/unit/{test_generalio,test_public_api,test_consumer_contract,test_frame_selection,test_reader_contract}.py -k 'not stru_accepts_signed_mobility' from QuantumMisaka/abacuslite@3ac69ef: 95 passed, 2 deselected per ASE version. The standalone-only signed mobility cases are not part of this backport. The replay covers fixed overwrite/append and numbered layouts, missing spectra (including the final step), an unannounced stale fixed file, named streams and signed occupations.

Engine checks:

  • Built abacus_pw_omp from c46ae1ede (v3.11.0-beta13). Ran native four-step Si PW MD (md_dumpfreq=1, out_freq_ion=2, OMP_NUM_THREADS=1) with out_app_flag=0/1/2. The engine wrote spectra for steps [3], [1, 3] and [1, 3], respectively. On both ASE versions the reader matched those steps; the calculator returned step 4's energy and forces without borrowing a spectrum. The installed bundled wheel also passed the same checks on ASE 3.29.0.
  • With LTS v3.10.1 and ASE 3.29.0, python -m unittest discover -s interfaces/ASE_interface/tests -p "$case" -v passed for scf.py, relax.py, and md.py. python -m pytest -q --tb=short interfaces/ASE_interface/abacuslite/core.py -k 'test_calculator_results or test_restart' passed restart; test_calculator_results reported the unchanged local energy difference (-194.9530533315 vs -194.953050937 eV). The reference was not changed. The earlier P100 runner issue was fixed separately in ci(cuda): detect runner GPU architectures before building #8347. No governance exception is requested.

@QuantumMisaka
QuantumMisaka marked this pull request as ready for review November 15, 2026 04:33

@ABACUS-PR-BOT ABACUS-PR-BOT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Source inspection identified two compatibility issues and two non-blocking follow-ups below. Reviewed the complete diff and affected callers, including ASE 3.23/3.29 source APIs. Governance and Python syntax checks passed; the full whitespace check flagged preserved fixture formatting only. Builds, tests and the engine were not run; author-reported results remain unverified, and CI diagnostics were unavailable.

# the parser; otherwise a malformed mapping leaks a bare IndexError.
reference = _read_stru_reference(
directory, stru_file, self.atomorder, log_path=log
)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new mandatory STRU reread rejects valid noncollinear input such as mag 1.0 -1.0 0.0: _parse_coord_line() recognizes a vector only when the y component starts with a digit, then rejects the remaining negative token. This prevents ordinary SCF result collection before either output reader runs. Please accept signed vector components and cover a negative-y moment through the public SCF handoff.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 770926e. _parse_coord_line() now parses all three Cartesian moment components with float(), including negative values. Added a public SCF handoff regression with mag 0.0 -1.0 0.0; the offline suite passes on ASE 3.23 and 3.29.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: signed Cartesian moments remain supported after the keyword-parser cleanup, including the negative-y public SCF handoff case. Its regression is maintained in the standalone suite and run externally against the bundled implementation; the imported test directory has been removed from this PR (329bdf8).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed in the current source: keyword-delimited parsing consumes signed vector components before the SCF STRU reread, including the original negative-y case.

if vector_moments:
results.setdefault('natoms', len(atoms))
if 'energy_without_entropy' in atoms.info:
results['energy_without_entropy'] = atoms.info['energy_without_entropy']

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After a successful calculation with Fermi or entropy data, inherited calc.export_properties() now raises: ASE 3.23/3.29 reconstructs Properties(self.results), which accepts neither efermi nor energy_without_entropy; vector magmoms also violates its shape schema. This affects export and the experimental atoms.get_properties(["energy"]) path, while ordinary energy getters remain usable. Please preserve the extra metadata while providing a schema-compatible export and regression.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 770926e. Abacus.export_properties() now filters unknown metadata and skips vector magnetic moments that do not match ASE’s versioned scalar schema; efermi, energy_without_entropy and vector magmoms remain in calc.results. Added coverage for both export_properties() and Atoms.get_properties(["energy"]).

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: export_properties still projects through ASE Properties, while read_results reuses the already-validated SinglePointDFTCalculator properties without projecting twice. Extra metadata and vector moments remain in calc.results. The public export/get_properties regressions pass; their source now stays in the standalone suite (329bdf8).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current export override validates a separate schema-compatible mapping while calc.results retains the extra metadata and vector moments.

Comment thread interfaces/ASE_interface/README.md Outdated
Both ASE `energy` and `free_energy` retain ABACUS's force-consistent
`E_KohnSham`. When an entropy term is present, `energy_without_entropy` is
provided separately; a missing term remains unknown. Noncollinear vector
moments are retained without modifying ASE's global property definitions.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: the recommended examples/soc.py and tests/magnetic.py still replace ASE’s global magnetic-property definitions. The new handoff no longer needs that workaround; removing it would align these consumers with the documented behavior and avoid rejecting later collinear results in the same process.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 770926e: removed the global ASE magnetic-property mutation from examples/soc.py and tests/magnetic.py. The calculator now handles vector moments locally.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The global magnetic-property mutations remain removed from both examples/soc.py and tests/magnetic.py. This consumer correction is retained in the reduced diff; it is not part of the test-directory removal (329bdf8).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both global magnetic-schema overrides are removed from the SOC example and magnetic integration test, so this consumer cleanup is addressed.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: tests/magnetic.py again overrides ASE's global magmom/magmoms schema at 83667f8f, while the SOC example retains the earlier cleanup. Please remove the test override too so the noncollinear case can catch regressions in the calculator's handling of ASE's standard schema.

Static source review only; no tests or CI results verified.


@pytest.mark.parametrize("scale", [1.234564321, 1.2345652])
def test_lts_rounded_cell_nonunit_alat_and_nonorthogonal_vectors(tmp_path, scale):
"""The precise Cartesian header, rather than its rounded Angstrom line, sets alat."""

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: this reader still takes alat from the rounded lattice constant (Angstrom) line, so this test establishes tolerance acceptance rather than precise-header selection. Likewise, lines 414–415 describe a second spin channel although their fixture declares nspin=1 and one E_Fermi. Please align both descriptions with the paths the assertions actually exercise.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Corrected in 770926e: the fixture is an nspin=1 log with one Fermi value; the comment now describes that value directly.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The imported test file has now been removed from the ABACUS PR. The standalone counterpart describes tolerance acceptance with rounded lattice output and the single Fermi value in its nspin=1 fixture; it does not claim precise-header selection or a second spin channel. The bundled README states the fixed-tolerance limitation (329bdf8).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The affected test file has been removed from this PR, so neither misleading description remains in the bundled change. The README states the fixed-tolerance limitation; the standalone wording remains unverified here.

@QuantumMisaka

Copy link
Copy Markdown
Collaborator Author

The compatibility findings are fixed in 770926e and the four inline replies document the changes. The standalone counterpart is tracked separately as QuantumMisaka/abacuslite#16 and remains draft pending the ABACUS-first maintenance decision.

@kirk0830 kirk0830 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have only reviewed a very limited of this PR, there are 7000 net increase line, a detailed review seems not plausible. However, as for as what I have found, there are still several lines, designs need to be re-considered, for a long-term maintainability.

values = list(atomorder)
except (TypeError, ValueError) as exc:
raise RuntimeError(
f"scf atomorder 无法解析(STRU: {stru_path};期望 {natoms} 个整数索引;"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please use English instead of Chinese, thanks

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Converted this diagnostic to English. The SCF errors now identify the relevant STRU/log and the rejected condition; the language cleanup is included in 329bdf8.

f"fail-closed)"
) from exc

valid_integer = all(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for this boolean is used only once, it is usually not recommended to explicitly create such a variable, instead, use the expression at the rhs of the = operator to keep the code clean

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the one-use valid_integer variable and placed the check directly in the permutation condition. The adjacent comment explains the integer, range and uniqueness requirements (329bdf8).

) as exc:
suffix = f";log: {log_path}" if log_path is not None else ""
raise RuntimeError(
f"scf STRU 身份信息无效(STRU: {stru_path}{suffix};{exc};fail-closed)"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use English please.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Converted the STRU/coordinate diagnostics here to English and kept the filename and failing condition in the error (329bdf8).

cell_inverse = _invert_cell(cell, f"STRU {stru_path}")

if coord_type.startswith('d'):
positions_species = coordinates @ cell

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the meaning of the variable "positions_species"? you use the "_" as "of"?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed this to species_positions: Cartesian positions in the species-grouped STRU order, before applying the mapping back to ASE order. The unused return fields and redundant array copies have also been removed (329bdf8).

'path': stru_path,
'positions': np.asarray(positions_species, dtype=float)[mapping].copy(),
'cell': np.asarray(cell, dtype=float).copy(),
'cell_inverse': np.asarray(cell_inverse, dtype=float).copy(),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the necessity to pass the inverse of the cell in this function? cell is also passed, the inverse should be calculated elsewhere

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed cell_inverse from the reference dictionary because no caller consumed it. The reference cell is still checked for singularity; the matching routine computes the candidate inverse where the periodic residual needs it (329bdf8).

scale = float(traj[0]['alat_in_angstrom'])
if not np.isfinite(scale) or scale <= 0:
raise ValueError(f"Invalid lattice constant in Angstrom: {scale!r}")
return direct, scale

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not what a commonly metadata like

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed the helper to _log_coordinate_system. Its two results are the logged fractional/Cartesian convention and the Cartesian scale in Angstrom. It reuses the already-validated raw trajectory during SCF collection (329bdf8).

raise
except (AssertionError, KeyError, OSError, TypeError, ValueError, IndexError) as exc:
raise RuntimeError(
f"scf running log 原始帧校验失败:{log_path}({exc};fail-closed)"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

English, please

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The frame-matching diagnostics are now in English, including the last-frame coordinate and cell differences. The cleanup is included in 329bdf8.

return _frame_coordinate_metadata(log_path, legacyio=legacyio)[0]


def _validate_raw_scf_trajectory(log_path, expected, *, legacyio=None):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this function has a non-trivial cognitive complexity due to too deep conditional statements...what does this function do?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check prevents atom reordering from silently dropping an extra atom in malformed output. I separated the per-frame checks from the trajectory loop and reduced duplicate matching bookkeeping and result conversion. The raw cardinality check remains before indexing; malformed-frame and atom-order regressions still pass (329bdf8).

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up in 4d24f6324: the raw-frame validation and SCF matching now live in private _scf_frames.py. The calculator selects the reader and passes it explicitly; validation still checks each raw frame before atom reordering can discard an extra atom. f31344ce0 adds coverage in the existing inline suite for eight engine coordinate modes and ensures valid per-atom options do not prevent geometry matching.

result['mag'] = ('Cartesian',
list(map(float, fields[idx+1:idx+4])))
idx += 4
elif idx + 3 < len(fields):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, this is a quite fragile parsing strategy if rely on the number of lines and this is where the ase-abacus project always failed in the past, which furtherly caused the endless bugfix... Do you have any better idea on it to enhance the robustness of these lines?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replaced the positional look-ahead with keyword-delimited option parsing. Counts now validate the declared grammar: three coordinates, mobility/velocity triplets, one or three magnetic components, and paired angle1/angle2 values. Signed/scientific values and valid trailing bare mobility triplets are covered, as are truncated or ambiguous records. This uses small standalone cases and existing fixtures (329bdf8).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The paired-angle check rejects the native-valid row 0 0 0 mag 2 angle1 90. The engine defaults the omitted angle2 to zero, but read_stru() now raises before Abacus.restart() can extract species metadata, whereas the base reader returned that metadata. Please preserve the native default and add a regression for this form.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up to the single-angle finding: ABACUS v3.10.1 produced the same initial moment with mag 2 angle1 90 and with angle2 0 added. The STRU reader now applies the native defaults; restart() reads only species and orbital records, so per-atom options such as sc do not block it. The retained tests are in QuantumMisaka/abacuslite#17. (3d6e51f5a)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed in 3d6e51f5a: either omitted spin angle defaults to zero, and an omitted per-atom magnitude uses the species value. restart() now reads species/orbital metadata without parsing per-atom options, so the reported single-angle failure is addressed in source.

trajectory = read_traj_from_running_log(abacus_lines) \
if fileobj.name != 'running_md.log' else \
read_traj_from_md_dump(fileobj.parent / 'MD_dump')
is_md_dump = fileobj.name == 'running_md.log'

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we do not always use the Java like "is_" as the boolean prefix in this project.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replaced is_md_dump with md_calculation in the latest reader. The legacy reader separately names geometry_from_dump, so the variable identifies the selected data source rather than suggesting that it defines periodicity (329bdf8).

positions=frame['coords'][ind],
cell=frame['cell'],
cell=cell,
pbc=is_md_dump,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

confusing in semantics: pbc or not depends on whether is MD dumped?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed: periodicity does not depend on the output file. Both readers now set pbc=True for the ABACUS periodic simulation cell, including SCF output. Existing SCF/MD fixtures verify this independently of whether MD_dump supplies the trajectory (329bdf8).

ind = ind or list(range(len(frame['elem'])))
positions = frame['coords'][ind]
if is_md_log:
if uses_md_dump:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

confusing semantics. it is not the md dump directly determines the unit. either use another variable name or add in-line annotations

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Separated md_calculation from geometry_from_dump and documented the format units at the conversion: MD_dump lattice vectors are in alat and its Cartesian positions are in Angstrom; running-log cells are already in Angstrom, while positions follow the logged coordinate convention. The conversion and non-unit-alat regressions pass (329bdf8).

cell=frame['cell'],
positions=positions,
cell=cell,
pbc=is_md_log,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

confusing.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pbc expression here no longer depends on the calculation/file branch: both readers return pbc=True for the periodic ABACUS cell. Geometry source selection and unit conversion are handled separately (329bdf8).

Comment thread interfaces/ASE_interface/pyproject.toml Resolved
@@ -0,0 +1,33 @@
INPUT_PARAMETERS

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't the testfiles in the integration tests folder satisfy the functionalities being tested? adding quite many additional files without maximizing the reuse is strongly discouraged.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the newly imported tests/unit and tests/fixtures trees and their added CI step. ABACUS keeps its existing inline and engine integration harness. The standalone suite can be run externally against the bundled source; its own duplicate tests/data were consolidated. The resulting PR changes 8 existing files, +760/-178, with no new recorded logs (329bdf8).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: please reuse the inline suites for numerical Direct/nonunit-alat position expectations in both public readers, plus signed/scientific moments and bare mobility after a complete option. The current reader mocks check magnetic ordering at alat=1, and the input tests cover leading bare flags but not those option combinations. A pinned external check against the bundled source would also cover this gap without restoring the fixture trees.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: please add compact rejection cases to the existing inline suite: an unmatched SCF frame, a malformed frame after an earlier valid match, and an incomplete option triplet through public read_stru(). The new eight-mode case and existing option round trips cover successful paths; these additions would protect the extracted validation paths.

Static review at 034d4fba; no runtime or CI results verified.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The geometry-rejection part of my earlier request is obsolete at 83667f8f. The updated standalone tests at QuantumMisaka/abacuslite@8910a348 now exercise empty/multiple SCF results, collection without STRU, and final relax/MD frames. Those sources address the revised contract; I have not verified their execution against this bundled revision.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: This revision selects the final SCF frame and changes restart filename overrides, labelled fractional KPT rows, and validation before file backup. Please retain matching checks in the reused inline suites or a pinned external test revision against this bundled source; the PR currently describes temporary checks and a temporary expectation change.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up on reproducibility: the regression tests are pinned at QuantumMisaka/abacuslite@9e6340e (QuantumMisaka/abacuslite#17). The PR body records the command for running them against the bundled source on ASE 3.23 and 3.29. No new test files or fixtures are added here. (3d6e51f5a)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pinned QuantumMisaka/abacuslite@3ac69ef contains the missing-spectrum export regression: test_develop_md_spectra_follow_ionic_steps exercises missing-final and missing-all spectra through the public calculator, retaining energy and forces while asserting that exported eigenvalues and occupations are absent. This addresses that specific regression through the accepted external-test route. I verified the test source; execution remains unverified.

@QuantumMisaka
QuantumMisaka marked this pull request as draft November 15, 2026 14:36
@QuantumMisaka

Copy link
Copy Markdown
Collaborator Author

Following the maintainability review, the PR is now limited to 8 existing files (+760/-178) at 329bdf8. The imported offline regression directories, recorded logs and added CI step are removed; ABACUS keeps its existing harness. Standalone tests and documentation were also consolidated in QuantumMisaka/abacuslite#16, and release artifacts contain no tests, logs or development history.

The follow-up removes redundant SCF matching state, array copies and property conversion, and leaves downstream abacus.json markers untouched. Keyword parsing, named streams, periodicity, units and the ASE minimum are addressed individually in the review threads. Verification: 130 external regressions against the bundled source on ASE 3.23 and 3.29, 55 existing inline tests, and one native SCF integration passed; the standalone source and rebuilt installed wheel each passed 210 tests. The PR description records skipped cases and the pre-existing energy-reference mismatch. Hosted checks for this new head are still running.

Threads are left open for reviewer confirmation.

@QuantumMisaka
QuantumMisaka marked this pull request as ready for review November 17, 2026 02:34

@ABACUS-PR-BOT ABACUS-PR-BOT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The earlier signed-moment and property-export issues are addressed in source. The consumer schema overrides are removed, and the misleading test descriptions are no longer in the bundled diff. A focused test-reuse suggestion remains non-blocking.

Static incremental review at cc3f2f91ca6934b684adac6e3eb9a1cf688af3f0: target-base governance, Python syntax and contributor whitespace checks passed. No candidate imports, builds, engine tests or CI results were verified; external test reports remain author-reported.

@mohanchen mohanchen added Interfaces Interfaces with other packages Refactor Refactor ABACUS codes labels Nov 17, 2026

@kirk0830 kirk0830 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution.
The lines of codes changed have been largely eliminated due to the reuse of reference files, that's good.
I have made my first complete scan of the code changes, now I have a better understanding on this PR.
This PR makes the ase interface more robust, but there are still comments regarding the readability and maintainability, as what I have pointed out inline and, another suggestion is, please consider split several helper functions out of the core.py, it may be good to have a new file to place those functions, because now the file is too long. A very long file can be harmful for both human and agentic programmer in the same way: occupying too much the context window.

Regards,
Yike

Comment on lines +19 to +37
def energy_components(ener: Mapping[str, float]) -> Dict[str, float]:
"""Return optional energy metadata from one ABACUS energy table.

``E_KohnSham`` already includes ``E_entropy(-TS)`` and remains the
force-consistent ASE energy. The separate no-entropy value is metadata,
not an ASE ``free_energy`` replacement. Missing entropy stays unknown.

Args:
ener: Frame-local ABACUS energies in eV.

Returns:
The no-entropy energy in eV when its entropy term is available.
"""
if 'E_entropy(-TS)' not in ener:
return {}
return {
'energy_without_entropy': float(ener['E_KohnSham']) - float(ener['E_entropy(-TS)']),
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the function name doesn't match its behavior

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed this helper to _energy_without_entropy_metadata() in 4d24f6324. It returns the single optional metadata field indicated by its name, rather than a breakdown of the energy components. Both readers use it; the ASE energy and free-energy values are unchanged.

@QuantumMisaka

Copy link
Copy Markdown
Collaborator Author

@kirk0830 Thanks for the review. The energy helper now names its single optional field, and the SCF validation and matching helpers have moved out of core.py into private _scf_frames.py. The full-diff recheck also found and fixed two compatibility issues: STRU matching now follows engine coordinate conventions, and valid per-atom options such as sc no longer prevent geometry matching. The existing inline suite covers these cases without new fixture directories or CI changes. Could you re-review f31344ce0 when convenient? The PR description has the updated verification.

@kirk0830 kirk0830 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only one suggestion in this round: rename the _scf_frames.py to something like atomorder_bridge.py, because the semantics of scf collides with the frame of MD, they two both are not expressive enough for the functions in the file.

@kirk0830

Copy link
Copy Markdown
Collaborator

@QuantumMisaka why do you choose the _self_geometry_check.py? what is the significance of the term "scf"?

@QuantumMisaka

Copy link
Copy Markdown
Collaborator Author

Thanks to @kirk0830 for raising this. scf referred to calculation=scf. The helper compared the single-point log geometry with the written STRU.

I removed the helper in 83667f8f7 because matching geometry alone cannot establish that the output came from the current run. The calculator now parses the output once, requires exactly one result for single-point SCF, and uses the existing mapping to restore ASE atom order. Native relaxation and MD still use the final parsed frame.

@ABACUS-PR-BOT ABACUS-PR-BOT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One STRU compatibility issue remains: valid single-angle input is rejected before restart can read species metadata. The existing parser discussion has the requested correction; the remaining compatibility, documentation, and focused test suggestions are non-blocking.

Static source review at dd8cd162c4d92baec2d75445e296af6b97152afa against eba09c937c734f3f4da52cbdbb9c55504846921a, including changes since 80d6f161. No product code, builds or tests were run; author-reported results and live CI remain unverified.


from abacuslite.io.generalio import (
_write_ase_stru,
_write_kpt,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: For external callers using from abacuslite import write_stru, write_kpt, these private imports remove the existing aliases because abacuslite/__init__.py re-exports public names from core. Retaining the public aliases alongside the private helpers would preserve those imports.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restored the top-level write_stru and write_kpt imports without changing their signatures. Verified both against the bundled source and the installed wheel. (3d6e51f5a)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed in 3d6e51f5a: core.py again imports both public writers, preserving the package-level write_stru and write_kpt aliases.

Comment thread interfaces/ASE_interface/README.md Outdated
native MD. The readers restore ASE atom order using the mapping created when
the calculator writes STRU. The legacy reader can use MD_dump when present,
and the develop reader reads native MD from MD_dump. Each calculator retains
its own LTS/develop reader.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: Please document that native-MD collection requires matching geometry and electronic/energy frames, normally with md_dumpfreq=1 and the companion files present. The readers compare frame counts directly without aligning sparse dumps, and disabling dumping leaves the develop reader without its required MD_dump.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The README now documents the matching-frame requirement, md_dumpfreq=1, and the companion files. Complete historical legacy logs still work without MD_dump; sparse dumps are not aligned. (3d6e51f5a)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The matching-frame requirement, md_dumpfreq=1, and companion files are now documented at 3d6e51f5a, including the exception for complete historical legacy logs. This addresses the documentation request for the current reader.

@kirk0830 kirk0830 left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find there are unnecessary deletion of annotations, along with the redundant naming of variables. I suspect highly the context of the agentic programmer has been severely polluted...
Please fix issues of this PR, thanks.

@QuantumMisaka

Copy link
Copy Markdown
Collaborator Author

Thanks to @kirk0830 for the follow-up. 3d6e51f5a addresses the naming and comment changes in the inline threads and documents the distinction between ASE properties and abacuslite metadata. It also fixes the native STRU defaults and makes restart read only the species information it needs.

One correction to my earlier SCF reply: the SCF-only frame-count check is gone too. SCF, native relaxation, and native MD now all use the final parsed result and report an empty result with the log path.

The PR description now records the local checks. The retained regression tests are in QuantumMisaka/abacuslite#17; no tests or fixtures were added to this PR.

@ABACUS-PR-BOT ABACUS-PR-BOT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The single-angle restart issue and public writer aliases are addressed in source, and the requested native-MD limits are documented. One integration note remains for the conflict with develop at c46ae1e.

Static review at 3d6e51f5a against eba09c937, with target compatibility checked through c46ae1e. Governance and whitespace checks passed; no product execution or merged candidate was validated, and runtime/CI results remain unverified.


# read the eigenvalues (nframe, nk, nbnd)
elecstate = read_band_from_eig_occ(fileobj.parent / 'eig_occ.txt')
elecstate = read_band_from_eig_occ(log_path.parent / 'eig_occ.txt')

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: the merge with develop at c46ae1e has a content conflict here. Please retain develop’s producer-header parsing, eigenvalue-file discovery and step alignment alongside this PR’s stream support, coordinate conversions and signed occupations, then align the MD documentation with the combined reader.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Synced with develop at c46ae1ede in 34c1dd89e. The reader uses the normalized log path with develop’s file discovery and ionic-step alignment. It retains named streams, coordinate conversions and signed occupations. Missing spectra now leave ASE’s kpts unset rather than empty, so energy and forces remain available. The README distinguishes sparse spectra from sparse geometry dumps.

Verified on native develop MD with all three out_app_flag layouts and a final step without a written spectrum, on ASE 3.23 and 3.29. The retained regressions are in QuantumMisaka/abacuslite#17.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Source review confirms that 34c1dd89e retains normalized-log discovery, ionic-step alignment, coordinate conversions and signed occupations. Missing spectra stay unavailable, and the README distinguishes sparse spectra from sparse geometry. This addresses the integration note. Builds, runtime tests and CI were not independently verified.

Keep step-aware eigenvalue discovery with named log sources. Represent missing spectra as unavailable for ASE while preserving energies and forces.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Interfaces Interfaces with other packages Refactor Refactor ABACUS codes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants