Skip to content

fix(cli): 在 zip 压缩阶段应用 --ignore-dirs 参数过滤指定目录 - #2070

Merged
qin-ctx merged 3 commits into
volcengine:mainfrom
sponge225:fix/ignore-dirs-zip-filter
May 15, 2026
Merged

qin-ctx merged 3 commits into
volcengine:mainfrom
sponge225:fix/ignore-dirs-zip-filter

Conversation

@sponge225

@sponge225 sponge225 commented May 15, 2026

Copy link
Copy Markdown
Collaborator

Description

ov add-resource --ignore-dirs 参数此前仅被发送到服务端用于目录扫描阶段的过滤,但客户端 Rust CLI 在 zip 压缩打包阶段仍然将所有目录(包括 .gitnode_modules 等)打包进 zip 文件,导致上传时可能超过 512MB 限制。

本次修改在客户端压缩阶段即通过 walkdir::filter_entry 剪枝忽略目录,从源头减小 zip 体积。

Related Issue

Fixes #2038

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional changes)
  • Performance improvement
  • Test update

Changes Made

  • 新增 parse_ignore_dirs() 函数,解析逗号分隔的忽略模式字符串
  • 新增 ignore_dirs_filter() 函数,生成 walkdir::filter_entry 闭包,支持 basename 匹配和相对路径匹配(含 / 的模式)
  • zip_directoryzip_directory_with_progress 增加 ignore_dirs: Option<&str> 参数
  • 在三处 walkdir 遍历中应用 filter_entry 剪枝忽略目录
  • 添加根目录保护,防止根目录名与忽略模式匹配时误剪枝整棵目录树
  • add_skill 调用时传入 None,不影响 skill 添加流程
  • 新增 12 个单元测试覆盖解析和过滤逻辑

Testing

  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have tested this on the following platforms:
    • Linux
    • macOS
    • Windows

验收结果:

  • 测试目录含 .gitnode_modules,原始大小约 32MB
  • 使用 --ignore-dirs ".git,node_modules" 后,zip 体积缩减至 868B
  • 不使用 --ignore-dirs 时,服务端兜底过滤仍生效(IGNORE_DIRS 默认列表)

测试过滤掉约一半的文件后,打包上传耗时约减少一半
image

image

Checklist

  • My code follows the project's coding style
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

Screenshots (if applicable)

Additional Notes

  • 客户端过滤与服务端过滤互补:客户端在压缩阶段剪枝可减小上传体积,服务端在解析阶段过滤作为兜底
  • 相对路径匹配语义与服务端 directory_scan.py 中的 _should_skip_directory 保持一致

sponge225 added 2 commits May 15, 2026 15:21
Previously, the --ignore-dirs parameter was only sent to the server-side
for filtering during directory scanning, but the client-side zip
compression still included all directories. This caused large dirs like
.git and node_modules to be packaged into the zip, potentially exceeding
the 512MB upload limit.

Changes:
- Add parse_ignore_dirs() to parse comma-separated ignore patterns
- Add ignore_dirs_filter() to generate walkdir filter_entry closures
  supporting both basename matching and relative path matching
- Add ignore_dirs parameter to zip_directory/zip_directory_with_progress
- Pass ignore_dirs from add_resource through to zip functions
- Protect root directory from being accidentally pruned
- Add 12 unit tests for parse_ignore_dirs and ignore_dirs_filter

Fixes: volcengine#2038
@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis ✅

2038 - Fully compliant

Compliant requirements:

  • ov add-resource --ignore-dirs now filters directories during zip compression
  • .git directories are excluded from the zip
  • add_skill uses None for ignore_dirs, maintaining existing behavior
⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🏅 Score: 95
🧪 PR contains tests
🔒 No security concerns identified
✅ No TODO sections
🔀 No multiple PR themes
⚡ No major issues detected

@github-actions

Copy link
Copy Markdown

PR Code Suggestions ✨

No code suggestions found for the PR.

@qin-ctx
qin-ctx merged commit 008816e into volcengine:main May 15, 2026
9 of 10 checks passed
@github-project-automation github-project-automation Bot moved this from Backlog to Done in OpenViking project May 15, 2026
@sponge225
sponge225 deleted the fix/ignore-dirs-zip-filter branch July 7, 2026 09:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

[Bug]: ov add-resource Error: API error: [INVALID_ARGUMENT] Upload exceeds size limit (536870912 bytes)

2 participants