Repository navigation
Revert using objectId as tie breaker in infinite scrolling #1799
Description
New Issue Checklist
- I am not disclosing a vulnerability.
- I am not just asking a question.
- I have searched through existing issues.
- I can reproduce the issue with the latest versions of Parse Server and Parse Dashboard.
Issue Description
Since #1706, creating an index to optimize dashboard queries requires compound indices including objectId. For example, when browsing a class sorted by createdAt a compound index createdAt, objectId is required.
parse-dashboard/src/dashboard/Data/Browser/Browser.react.js
Lines 650 to 656 in cb96799
| if (field !== 'objectId') { | |
| if (sortDir === '-') { | |
| query.addDescending('objectId'); | |
| } else { | |
| query.addAscending('objectId'); | |
| } | |
| } |
This approach requires special indices just for dashboard browsing. If an app is using a complex or resource intensive index, dashboard requires the same index plus the additional objectId in the index compound. I don't think this is a feasible solution, given the high cost of indices on a DB. Maybe we need to look for an alternative solution.
Additionally, #1706 broke existing "query optimization" and should have been treated as a breaking change. The reason this was released as non-breaking was that we didn't have the policy that a changelog entry has to be written as part of the PR back then. It was not noticeable without reading the PR discussion in detail that it was a breaking change. This should not happen anymore as we are now requiring a changelog entry with the PR.
Steps to reproduce
- Create new call
- Browse class, sorted by
createdAt - Query
createdAt, objectIdsent to server
Actual Outcome
Dashboard currently paginates by issuing an entirely new query which requires to construct a complicated pagination mechanism (#1706), which creates the indexing issue described here.
Expected Outcome
In MongoDB at least, the proper and resource efficient way to paginate is to use the query cursor, see getMore. That may require some changes on the server side, for example sending the cursor ID to dashboard as query results meta data. And not sure whether Postgres supports cursor based pagination.
In any case, I think we should revert #1706 ASAP and release as a fix, or at least the part that adds the object ID to the query, because that can render the dashboard unusable or cause stability issues for some deployments.
@sadakchap @davimacedo @Klaitos What do you think?
Environment
n/a
Dashboard
- Parse Dashboard version:
2.2.0,master
Logs
n/a
Activity
parse-github-assistant commentedon Sep 14, 2021
Thanks for opening this issue!
[-]Default indices missing for sort by createdAt[/-][+]docs: add indexing considerations to optimize dashboard queries[/+][-]docs: add indexing considerations to optimize dashboard queries[/-][+]Add indexing considerations to optimize dashboard queries[/+][-]Add indexing considerations to optimize dashboard queries[/-][+]Revert using objectId as tie breaker in infinite scrolling[/+]sadakchap commentedon Sep 14, 2021
@mtrezza, yeah.. we should revert #1706 . Could you please do that? And then we can tackle #1551 later separately (probably using getMore).
Klaitos commentedon Sep 14, 2021
I'm glad you changed your mind about it 🙂.
mtrezza commentedon Sep 14, 2021
It's unfortunate that we need to revert it, because that opens the original scrolling issue again, and @sadakchap put some good work into fixing this. However, I think the implications we discovered now do not justify the type of fix.
@Klaitos Would you want to give it a try to open a PR to revert this?
Klaitos commentedon Sep 15, 2021
Done here #1800
mtrezza commentedon Sep 16, 2021
Closing via #1800