Document sub_group_size and correct its default to 1e9

[shravanibnikam]

Problem

sub_group_size has no description entry in the ZeRO section of config-json.md. It appears only as a bare value in the example JSON block, so nothing tells a user what it does or when to change it. That is the original report in #1145.

The default is also documented wrongly. deepspeed/runtime/zero/config.py:182 is authoritative:

sub_group_size: int = Field(pp_int(1e9), ge=0)

pp_int (deepspeed/runtime/config_utils.py:127) subclasses int and overrides only __repr__, so the default really is 1e9. @samyam's comment in #1145 says the same thing independently: "Most users can leave sub_group_size to its default value of 1e9."

Two places said 1e12:

• docs/_pages/config-json.md:381, the ZeRO example block.
• deepspeed/runtime/zero/config.py:39, inside ZERO_FORMAT. That one is not a comment — ZERO_FORMAT is interpolated into the logger.warning in read_zero_config_deprecated() (config.py:58-66), so the wrong value was printed at runtime to anyone on the deprecated config format.

docs/_tutorials/zero.md:139 already says 1e9, which is the inconsistency #1145 points at.

Fix

• Adds a ***sub_group_size*** entry to the ZeRO section, between stage3_param_persistence_threshold and stage3_gather_16bit_weights_on_model_save, matching the ordering in the example block above it. The ***key*** form matches every other entry in that section.
• Corrects config-json.md:381 from 1e12 to 1e9.
• Corrects ZERO_FORMAT in config.py:39 from 1000000000000 to 1000000000, keeping the full-digit style of its neighbours.

The prose is @samyam's explanation from #1145, relayed by @stas00, edited only for tense, to fix "each buckets" → "each bucket", and to drop the inline "1e9" now that the Default column carries it. It keeps the guidance on when to raise or lower the value, which seemed like the most useful part of that comment.

Where the 1e12 probably came from, and what I left alone

DeepSpeedZeroOptimizer_Stage3.__init__ declares sub_group_size=1000000000000 (deepspeed/runtime/zero/stage3.py:186). It is the only keyword default in that signature that disagrees with its config counterpart — reduce_bucket_size=500000000, max_reuse_distance=1000000000, max_live_parameters=1000000000 and param_persistence_threshold=100000 all match DeepSpeedZeroConfig. The sole in-tree caller, engine.py:2726, always passes sub_group_size=self.zero_sub_group_size() from the config, so the 1e12 is only reachable by constructing the optimizer directly, and no released behaviour depends on it via the normal path.

I have not touched it, because changing a public constructor's default is a behaviour change and does not belong in a docs fix. Flagging it in case you want it aligned separately — happy to send that as its own PR if so.

Also left alone for the same reason: deepspeed/autotuning/config_templates/template_zero3.json:15 and examples/sdma_allgather/ds_config_zero3.json:39, both of which also say 1e12. Those are real configs rather than documentation, so their values may be deliberate.

Test

No test. This is a documentation entry plus two literal corrections, so there is no contract to assert that a different correct implementation would not also satisfy. (Noting that #8242 added a docs/config consistency guard in this same file and @tohtana asked for it to be dropped as too fragile for CI.)

What was checked instead, on CPU:

• pre-commit run --files docs/_pages/config-json.md deepspeed/runtime/zero/config.py — all hooks pass, including yapf, flake8 and codespell.
• Rendered the added block through GitHub's GFM API and confirmed the output: <em><strong>sub_group_size</strong></em> for the heading, a well-formed <table> with Description/Default headers and <code>1e9</code> in the default cell, and an <ol> with both tuning cases. I did not build the Jekyll site locally; the addition uses no Minimal Mistakes-specific syntax, so plain GFM rendering covers it.
• Grepped the whole tree for sub_group_size. After this change, docs/ states the default in exactly one place and it agrees with config.py:182. The remaining occurrences are runtime code (zero/stage3.py, runtime/engine.py, superoffload/superoffload_stage3.py), tests, and the three non-doc configs noted above.
• I could not exercise the pydantic default at runtime — no GPU and no torch install here — so the 1e9 claim rests on reading config.py:182 plus pp_int, and on @samyam's own statement in the issue.

Fixes #1145