ISSUE

Specifying explicit GPU split crashes Strata.

Multi-GPU Documentation says Strata supports specifying split in strata-.json but Strata crashes on any value except "auto"

#strata-<model>.json

 "layer_split": "auto"  <-------works
 "layer_split": "24,16,12,12"  <-------fails
 "layer_split": [24,16,12,12]  <-------fails

Specifying Split Fails

[strata] layer split across GPUs [2, 0, 1, 3] (24,16,12,12)
[strata] starting the engine: reading the model's weights ...
Traceback (most recent call last):
File "/home/tonydiep/Strata/serve/server.py", line 2710, in
sys.exit(main())
~~~~^^
File "/home/tonydiep/Strata/serve/server.py", line 2587, in main
engine = StrataEngine(exe, engine_args(cfg), cwd=cfg.get("cwd"), log=cfg.get("log"), env=env, lazy=lazy)
File "/home/tonydiep/Strata/serve/server.py", line 291, in init
raise RuntimeError("the engine exited before it was ready" + (f" (see {log})" if log else "") +
start_failure_hint(log, log_start))
RuntimeError: the engine exited before it was ready (see /home/tonydiep/Strata/strata-iq3_xxs.log)
tonydiep@tiny:~/Projects/optimise-llms$ tail /home/tonydiep/Strata/strata-iq3_xxs.log)
bash: syntax error near unexpected token `)'

"auto" Layer Works

[strata] layer split across GPUs [2, 0, 1, 3] (auto)