Skip to content

Adopt src layout for the agent and server packages - #7

Draft
messa wants to merge 20 commits into
mainfrom
202606_messa_src
Draft

Adopt src layout for the agent and server packages#7
messa wants to merge 20 commits into
mainfrom
202606_messa_src

Conversation

@messa

@messa messa commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

Summary

Move the logline_agent and logline_server packages under a src/ directory (the conventional src layout), so tests and tools run against the installed package instead of accidentally importing straight from the working tree.

Stacked on #6. The base of this PR is 202606_messa_minor_updates, so the diff shows only the changes specific to this PR. Merge #6 first, then this can be retargeted to main.

Changes

  • Move agent/logline_agent/agent/src/logline_agent/ and server/logline_server/server/src/logline_server/ (via git mv, so the file history is preserved).
  • Point the uv build backend at the new location: module-root = "src" (was "") in both pyproject.toml files.
  • Update the server Dockerfile to COPY src ./src.
  • e2e_tests is left unchanged — it ships no package of its own.

No changes were needed to the Makefiles, the CI workflows, the script entry points, or uv.lock.

Testing

  • make check is green for all three components: agent (2), server (12), e2e (7 — including the real TLS round-trip and the --help smoke tests). The e2e suite installs both packages and runs their CLIs, so it exercises the build/install from the new layout end to end.
  • make -C server image — podman build plus the entrypoint --help smoke check passes.

🤖 Generated with Claude Code

messa and others added 20 commits June 2, 2026 13:09
Convert agent and server to standalone uv projects using the hatchling
build backend, with metadata moved from setup.cfg into pyproject.toml
([project] table) and committed uv.lock files. Remove setup.py/setup.cfg.

Add e2e_tests as a third uv project that pulls in agent and server via
editable path sources, and move its test modules into e2e_tests/tests/
to match the agent/server layout.

Add per-component Makefiles (check = lint + test) and make the root
Makefile delegate via $(MAKE) -C; add a server `image` target that builds
the container with podman and smoke-checks the entrypoint. Keep
run_e2e_tests as a backwards-compatible alias.

Rewrite the Dockerfile and the python-tests/e2e-tests workflows to use uv
(astral-sh/setup-uv, uv sync --frozen).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Bundle the MIT LICENSE into the agent and server wheels by symlinking
  the root LICENSE into each component (hatchling picks it up via PEP 639).
- server/Dockerfile: bump base image to python:3.14 and install
  dependencies in their own cached layer (uv sync --no-install-project)
  before copying the source, so source changes no longer invalidate the
  dependency install.
- CLAUDE.md: add an "Agent workflow" note to review changes via a
  background subagent.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Martin Hanzík <martin@hanzik.com>
The debug log in test_rotate_log_file referenced expected_dst_second_file,
which is only defined in test_new_log_file_gets_copied. Use expected_dst_file
instead: the destination path actually in scope, and the same variable the
analogous first wait-loop already logs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a lint target to e2e_tests/Makefile, identical to the agent and server
ones, and make it part of the default check (check: lint test). Recurse into
e2e_tests from the root lint target so the aggregate lint now covers all
three components, matching the existing check and test aggregates.

Previously the e2e test code was the only Python in the repo not covered by
flake8.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Point the "project readme file" link at leadhub-code/logline instead of the
old messa/logline location, matching the homepage URLs already corrected in
the pyproject files.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Switch both components from hatchling to uv's own uv_build backend:
- requires = ["uv_build>=0.11,<0.12"], build-backend = "uv_build"
- [tool.uv.build-backend] module-root = "" for the flat layout, since the
  package dir lives in the project root (not under src/) and uv_build
  defaults module-root to "src"
- license-files = ["LICENSE"] because uv_build, unlike hatchling, does not
  auto-glob license files

Replace the agent/server LICENSE symlinks with real copies of the repo-root
MIT license. uv_build validates license-files while building the project,
and the upward ../LICENSE symlink escaped the build root, which broke both
the server Docker build (context is server/) and isolated sdist builds.
Real copies fix every build path (wheel, sdist, Docker).

Copy LICENSE into the server image so uv_build's metadata validation passes
during uv sync.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace flake8 with ruff across all three components (agent, server,
e2e_tests). The ruff configuration lives in each component's
pyproject.toml: line length 150, rule sets E/F/W/I, and isort with two
blank lines after the import block. force-sort-within-sections keeps the
existing "sort by module name" import style, so reordering churn is small.

Update the Makefiles and GitHub Actions workflows to lint with
`uvx ruff check .`. The e2e_tests component is now linted as well
(previously it had no lint step), which surfaced a latent bug: a debug
log in test_rotate_log_file referenced an undefined name
(expected_dst_second_file instead of expected_dst_file).

Apply ruff's autofixes (import sorting, unused imports, redundant f-string
prefix, unused exception variable) and fix the remaining findings by hand:
- re-export the entry points via __all__ in the package __init__ modules
- reference the imported module in the smoke tests
- use `is not None` / `not ...` instead of `!= None` / `== False`
- fix the undefined name in the e2e rotate test

Document the Python style conventions (line length, two blank lines after
imports, preferring `from X import Y`) in CLAUDE.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Hoist function-local imports to module level where there is no reason to
defer them:
- logging name imports in setup_logging/setup_log_file (agent and server)
- the logging import in the e2e conftest
- the lazy ssl imports in the TLS branches (agent client, server)
- the lazy yaml imports in the configuration loaders (agent and server)

Also switch hashlib usage to `from hashlib import sha1` to match the
`from X import Y` convention.

Conditional compatibility shims (try/except ImportError in asyncio_helpers
and util), the optional zstandard/zstd fallbacks, and the smoke-test
imports are intentionally left local.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The minimum supported Python version is 3.9, where asyncio.run,
asyncio.create_task and asyncio.get_running_loop (all 3.7+) and
asyncio.to_thread (3.9+) are always available, so the fallback shims are
dead code.

Delete the agent's asyncio_helpers module and import run/create_task/
to_thread directly from asyncio. Drop the to_thread shim from the
server's util module the same way.

The zstandard/zstd fallback in util is intentionally kept: it selects
between two optional libraries at runtime and is not a Python-version shim.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
In the agent's obfuscate_secrets(), re.ASCII was passed as the 4th
positional argument to re.sub(), which is `count`, not `flags`. That made
the flag a no-op (256 used as the replacement count) and triggered a
DeprecationWarning on Python 3.13+. Pass it as `flags=re.ASCII`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
datetime.utcnow() returns a naive datetime and is deprecated since Python
3.12. Use datetime.now(timezone.utc) instead. The rendered string is
unchanged: the strftime format ('%Y%m%dT%H%M%SZ') has no %z/%Z field and
the trailing Z is a literal, so the rotated-file suffix stays identical.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The TLS certificate path from the YAML config was resolved relative to
the config file's directory, but the key path was used verbatim (i.e.
relative to the current working directory). Resolve the key the same way
as the certificate and the key password file for consistent behaviour.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The agent ran its event loop with debug=True, which was left over from
development. The asyncio debug mode adds runtime overhead (slow-callback
logging, extra thread-safety checks, ...) that is undesirable in
production. Run the loop with the default settings.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A freshly connected client that never sends its initial command kept a
server connection slot (and file descriptor) occupied indefinitely, which
could be abused in a slowloris-style fashion. Wrap the receipt of the
initial command in a timeout; established connections are still allowed to
stay idle for as long as needed, since the agent only sends data when the
watched log grows.

On the agent side, open_connection() had no timeout either, so a silent or
unreachable server would make the connection attempt hang forever. Wrap it
in a connect timeout so the follow loop can retry.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The handler used assert statements to validate data coming from the
client (required header fields, the data payload type and the expected
write offset). Assertions are stripped when Python runs with -O, which
would silently disable this validation, and they also produce confusing
AssertionErrors. Replace them with explicit checks that raise
ProtocolError, and read the offset via .get() so a missing field is
reported as a protocol error rather than a KeyError.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a "Pull requests" section that prescribes a single blockquote format
for flagging a stacked PR (one based on another PR's branch rather than
main), so these notes look the same across all PRs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The TLS certificate and key paths were Path objects when read from the
YAML config but plain strings when passed via CLI args. Wrap the CLI
values in Path() so tls_cert_file and tls_key_file have a consistent type
regardless of source, matching how the agent already handles its cert
path. Purely cosmetic: ssl.load_cert_chain accepts both.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move the logline_agent and logline_server packages under a src/ directory
and point the uv build backend at it (module-root = "src" instead of "").
This is the conventional src layout: the project root is no longer
importable, so tests and tools exercise the installed package rather than
accidentally importing straight from the working tree.

The server Dockerfile now copies src/ instead of the old top-level package
directory. e2e_tests is left unchanged since it ships no package of its own.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@messa messa self-assigned this Jun 2, 2026
@martinhanzik
martinhanzik force-pushed the 202606_messa_minor_updates branch from b880dba to 11dc1c1 Compare June 3, 2026 13:01
Base automatically changed from 202606_messa_minor_updates to main June 3, 2026 13:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants