Skip to content

ignore: speed up Gitignore::empty - #895

Merged
BurntSushi merged 1 commit into
masterfrom
ag/faster-gitignore-empty
Apr 24, 2018
Merged

BurntSushi merged 1 commit into
masterfrom
ag/faster-gitignore-empty

Conversation

@BurntSushi

Copy link
Copy Markdown
Owner

This commit makes Gitignore::empty a bit faster by avoiding allocation
and manually specializing the implementation instead of routing it through
the GitignoreBuilder.

This helps improve uses of ripgrep that traverse many directories, and
in particular, when the use of ignores is disabled via command line
switches.

Fixes #835, Closes #836

@BurntSushi

Copy link
Copy Markdown
Owner Author

@sharkdp I think this should resolve the performance regression. I think my initial analysis here wasn't quite the whole story. In addition to the overhead of create_gitignore, it turns out that Gitignore::empty() had a fair bit of cost associated with it as well.

Here are the benchmarks (first is current master, second is your PR on top of current master and third is this PR):

$ hyperfine -s basic --warmup 10 'rg-baseline -uu --files -g "README.rst"'
Benchmark #1: rg-baseline -uu --files -g "README.rst"
  Time (mean ± σ):     596.9 ms ±   4.7 ms    [User: 3.815 s, System: 3.200 s]
  Range (min … max):   589.8 ms … 604.1 ms

$ hyperfine -s basic --warmup 10 'rg-sharkdp -uu --files -g "README.rst"'
Benchmark #1: rg-sharkdp -uu --files -g "README.rst"
  Time (mean ± σ):     544.0 ms ±   2.8 ms    [User: 3.287 s, System: 3.112 s]
  Range (min … max):   540.3 ms … 548.5 ms

$ hyperfine -s basic --warmup 10 'rg -uu --files -g "README.rst"'
Benchmark #1: rg -uu --files -g "README.rst"
  Time (mean ± σ):     554.9 ms ±   6.8 ms    [User: 3.365 s, System: 3.160 s]
  Range (min … max):   547.7 ms … 566.3 ms

One important observation here is that because -u is used, the calls to matched on the Gitignore matchers never actually take place since has_any_ignore_rules returns false.

Your PR is still a hair faster, but I'm not inclined to it since it spreads case analysis out too much. The likely cause of the performance difference is the additional function calls/copying when calling Gitignore::empty as opposed to just using None. One possible middle ground is to change Gitignore to be pub struct Gitignore(Option<GitignoreInner>) which would make the implementation of Gitignore::empty() equivalent to your PR without spreading out the case analysis. I'm not inclined to think it is worth it though.

This commit makes Gitignore::empty a bit faster by avoiding allocation
and manually specializing the implementation instead of routing it through
the GitignoreBuilder.

This helps improve uses of ripgrep that traverse *many* directories, and
in particular, when the use of ignores is disabled via command line
switches.

Fixes #835, Closes #836
@BurntSushi
BurntSushi force-pushed the ag/faster-gitignore-empty branch from 25e0e21 to e68cc86 Compare April 24, 2018 14:53
@BurntSushi
BurntSushi merged commit ab64da7 into master Apr 24, 2018
@BurntSushi
BurntSushi deleted the ag/faster-gitignore-empty branch April 24, 2018 15:23
@sharkdp

sharkdp commented Apr 24, 2018

Copy link
Copy Markdown
Contributor

Very cool - thank you very much for looking into this!

I've run the benchmark again and see no significant difference between my PR and yours.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants