Web server cannot serve binary content (fonts, images) — response body and file reads are text-only

[logbie]

Summary

WFL's built-in web server cannot serve binary files (fonts, images, etc.) losslessly. Both the file-read path and the HTTP response body are String/UTF-8-based, so any non-UTF-8 bytes are corrupted before they reach the client. This blocks natural static-asset serving — e.g. self-hosting a .ttf font or serving a .png/.ico directly from a WFL web server.

Impact / motivation

Building the WFL website in WFL (serving its own marketing site via listen on port ...) requires serving a self-hosted Alegreya .ttf, a favicon, and potentially raster images. Today that's impossible without a workaround (base64 data: URIs embedded in a text CSS file), which inflates payloads and isn't how anyone expects static serving to work.

Root cause (two text-only chokepoints)

All references are in src/interpreter/mod.rs.

1. File read is text-only. The WFL read content path uses read_to_string:

   // src/interpreter/mod.rs:739
   match AsyncReadExt::read_to_string(&mut file_clone, &mut contents).await {

   A .ttf/.png is corrupted the moment it is loaded. Note that binary-capable readers already exist internally but aren't wired to a WFL keyword or to the serve path:

   // src/interpreter/mod.rs:842
   async fn read_binary(&self, handle_id: &str) -> Result<Vec<u8>, String> { ... }
   // src/interpreter/mod.rs:869
   async fn read_binary_n(&self, handle_id: &str, count: usize) -> Result<Vec<u8>, String> { ... }

2. The HTTP response body is a String. The response (and request) content is typed as String:

   // src/interpreter/mod.rs:75  (WflHttpRequest)
   pub body: String,
   // src/interpreter/mod.rs:82  (WflHttpResponse)
   pub content: String,

   So even a correctly-read Vec<u8> can't be carried through respond to req with <bytes> without a lossy String conversion.

Good news — the outbound warp path already accepts bytes

The warp reply is already built from bytes, so only the WFL-facing plumbing needs to change, not the HTTP layer:

// src/interpreter/mod.rs:5199 (approx)
match reply_builder.body(content_bytes) { ... }

Reproduction

1. Write a WFL web server that reads a .ttf and responds with it:

   listen on port 8080 as server
   main loop:
       wait for request comes in on server as req
       open file at "public/assets/fonts/Alegreya-Variable.ttf" for reading as f
       store data as read content from f
       close file f
       respond to req with data and content_type "font/ttf"
   end loop
   

2. curl -s localhost:8080/ -o out.ttf and compare with the source file.
3. Expected: byte-identical font. Actual: size/bytes differ (UTF-8 lossy replacement) and the font fails to load in a browser.

Proposed fix

Keep bytes end-to-end from read → respond → warp:

• Expose a binary read at the WFL level (e.g. read binary content from f) backed by the existing read_binary (mod.rs:842), yielding a bytes-capable value.
• Allow respond to req with <bytes> ... to carry those bytes into warp's existing .body(content_bytes) (mod.rs:5199) without routing through the String content field — e.g. add a bytes variant to WflHttpResponse (content: Bytes | String) so text responses stay unchanged and binary responses stay lossless.
• Bonus: a small built-in static-file/MIME helper so .ttf/.woff2/.png/.ico/.svg serve correctly out of the box (extension → content type, binary when appropriate).

Backward compatibility

Text responses (respond to req with "...") and text read content must keep working exactly as-is — the change is additive (a new binary read form + a bytes-capable response variant), preserving the backward-compatibility invariant.

Workaround until fixed

Base64-embed binary assets as data: URIs inside a text CSS file (which WFL serves fine as UTF-8). Works, but bloats payloads and defeats normal static serving.

[logbie]

Assessment: confirmed, well-scoped enhancement — line references verified against main.

Reproduced the two chokepoints in src/interpreter/mod.rs on current main (numbers have drifted a few lines since you filed, but every reference checks out):

• Text-only file read: AsyncReadExt::read_to_string(...) at mod.rs:745 — a .ttf/.png is UTF-8-lossily corrupted on load.
• Binary readers exist but are unwired: async fn read_binary at :848, read_binary_n at :875.
• Bodies are String: struct WflHttpRequest at :70, struct WflHttpResponse at :84.
• Outbound warp path already takes bytes: reply_builder.body(content_bytes) at :5209 — so only the WFL-facing plumbing is text-bound, not the HTTP layer, exactly as you note.

Your proposed fix is the right shape and preserves the backward-compat invariant: (1) a binary read form (read binary content from f) backed by the existing read_binary, and (2) a bytes-capable variant on WflHttpResponse (e.g. content: Bytes | String) so text responses stay byte-for-byte unchanged while binary responses reach .body(content_bytes) losslessly. I'd split the extension→MIME static-serve helper into its own issue so the core binary-plumbing fix can land first. TDD: serve a known .ttf/.png and assert the response bytes are byte-identical to the source. Suggested labels: enhancement, interpreter, web-server.

Posted by the WFL repo warden (automated triage pass).

[logbie]

Verified resolved on current main — recommend closing.

PR #574 ("binary file I/O and MIME type support") landed this. WFL now has a binary read at the language level (open file at … for reading binary + read binary from <handle>), binary write, lossless byte round-trip, and MIME helpers for static assets — covered by tests/web_server_binary_test.rs, tests/binary_io_test.rs, and TestPrograms/binary_file_and_mime_test.wfl. The text read content / text response paths are unchanged, preserving backward compatibility as requested.

---

Generated by Claude Code