Reducing queries in ScanFiles.php#40019
Conversation
szaimen
requested review from
a team,
ArtificialOwl,
CarlSchwan,
icewind1991 and
nfebe
and removed request for
a team
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
|
||
| if ($lastUser === $user) { | ||
| $this->logger->warning("User $user still has unscanned files after running background scan, background scan might be stopped prematurely"); | ||
| if ($this->config->getSystemValue('loglevel') > ILogger::WARN) { |
Check notice
Code scanning / Psalm
DeprecatedClass
There was a problem hiding this comment.
Checking the code there are not good alternatives.
There is a Psr\Log\LogLevel but it is a string, not a int.
There was a problem hiding this comment.
this is fine, the exception just need to be added in ./build/psalm-baseline.xml
|
Do you think the Samba's failure is my fault? I think it just not sleeped enough or something like that. https://drone.nextcloud.com/nextcloud/server/39214/7/3 A question, I see it usually fails the "Cypress" test in every PR. I'm doing something wrong? |
Signed-off-by: FedericoHeichou <federicoheichou@gmail.com>
Signed-off-by: FedericoHeichou <federicoheichou@gmail.com>
AndyScherzinger
force-pushed
the
master
branch
from
ff1c7ae to
6740a19
Compare
|
I'm unsure about the scalability of this approach. pre-fetching the data into memory will become unfeasible on instances with very large numbers of users or with many mounts per user (e.g. with heavy sharing). The current approach is in large parts the way it is for being fully incremental. |
|
Hello 👋 Moreover this might resolve the query problem but will run into memory issues on large instances, so at least some kind of chunking would be required. If you think this is still actual feel free to reopen the pull request and resolve the conflicts and adjust the approach to the review comments so we can incorporate it into Nextcloud. |
|
I'm using this patch from years and it fixed all the problems in my instances with low number of users but hug number of files. But I understand your point of view |
11 participants
Summary
By changing the approach to the problem, it will guarantee that all users are scanned and it will only perform 2 (or 3) fast queries without performing joins.
TODO
Checklist