truncated compressed source files read as empty instead of raising an error #68064

Bug Report

1. Minimal reproduce step (Required)

Create a zstd-compressed CSV that has been truncated mid-frame (any bytes after the zstd magic + frame header but before the last block boundary), upload it to the import source, then run estimate / IMPORT INTO size sampling. For example with the public Import SDK:

scanner, _ := importsdk.NewFileScanner(ctx, "s3://bucket/case-data-5/", db, defaultSDKConfig())
estimate, err := scanner.EstimateImportDataSize(ctx)
// err is nil; estimate.TotalSourceSize == 0; estimate.TotalTiKVSize == 0

Or use a synthetic input directly through mydump:

var compressed bytes.Buffer
zw, _ := zstd.NewWriter(&compressed)
zw.Write([]byte("a,b,c\n1,2,3\n4,5,6\n"))
zw.Close()
truncated := compressed.Bytes()[:compressed.Len()-4]
// feed `truncated` through mydump.OpenReader(... CompressionZStd ...) into a CSVParser; ReadRow returns io.EOF

zstd -dc on the same bytes correctly reports Read error (39): premature end.

2. What did you expect to see? (Required)

Either an error explaining that the source file is truncated, or — at minimum — a non-zero RealSize/sample size that reflects the bytes actually read. Silently treating a truncated compressed stream as a zero-byte empty file is a data-correctness hazard: the size estimate is reported as 0 and downstream size-based sizing decisions are wrong.

3. What did you see instead (Required)

EstimateImportDataSize returns nil error with TotalSourceSize=0, TotalTiKVSize=0. The Lightning CSV/SQL parser layer also silently terminates as if it had read a clean EOF.

Root cause is in pkg/lightning/mydump/parser.go:blockParser.readBlock:

n, err := parser.reader.ReadFull(parser.blockBuf)
switch err {
case io.ErrUnexpectedEOF, io.EOF:
    parser.isLastChunk = true
    fallthrough
case nil:
    ...

io.ReadFull returns io.ErrUnexpectedEOF in two situations:

• A regular short read at the tail of the stream (n > 0).
• The underlying reader actively returned io.ErrUnexpectedEOF (n == 0) — e.g. the zstd / gzip / snappy decoder hit a truncated frame.

Treating the second case as a clean last chunk is what masks the truncation. The (n=0, ErrUnexpectedEOF) pair is the load-bearing signal that the decompressor saw the stream end mid-frame, and pkg/objstore/... reader implementations (s3like, gcs, azblob, local, memstore) all return (0, io.EOF) for normal EOF, so this signal is unambiguous on the source-file reader chain.

A second related symptom: pkg/lightning/mydump/loader.go:calculateFileBytes swallows the same io.ErrUnexpectedEOF during compression-ratio sampling, which causes EstimateRealSizeForFile to compute a NaN ratio and report RealSize=0 for the truncated file. That part is out of scope for the immediate fix but worth tracking separately.

4. What is your TiDB version? (Required)

master (around 4a6574e456, v9.0.0-this-is-a-placeholder).