Skip to content

Add MongoDB 5 compatibility #7670

Closed
Closed
Add MongoDB 5 compatibility#7670
Labels
bounty:$50Bounty applies for fixing this issue (Parse Bounty Program)type:featureNew feature or improvement of existing feature

Description

@mtrezza
mtrezza
opened on Oct 30, 2021

New Feature / Enhancement Checklist

Current Limitation

PR to upgrade to mongodb node driver 4.x currently doesn't pass, needs some refactoring.
See #7586.

❗️ This is a critical upgrade for MongoDB 5 compatibility. All M0/M2/M5 Atlas clusters will automatically be upgraded to MongoDB 5.0 in early February 2022. That means after that, Parse Server will be incompatible with these and all free MongoDB Atlas plans.

A separate question is whether we want / can easily port this back to the 4.x branch. If not, then users of free or small (non-dedicated) MongoDB Atlas plans will be forced to upgrade to Parse Server 5.0. With that comes the necessity to adapt the deployment for the breaking changes in Parse Server 5.0.

Feature / Enhancement Description

Upgrade to the latest mongodb node driver.

Example Use Case

n/a

Alternatives / Workarounds

n/a

Activity

  1. parse-github-assistant commented on Oct 30, 2021

    @parse-github-assistant
    parse-github-assistantBot
    on Oct 30, 2021

    Thanks for opening this issue!

    • 🎉 We are excited about your ideas for improvement!
  2. mtrezza
    added
    bounty:$10Bounty applies for fixing this issue (Parse Bounty Program)
    type:improvement
    on Oct 30, 2021
  3. mtrezza
    linked a pull request that will close this issuerefactor: upgrade mongodb from 3.6.11 to 4.1.1 #7586on Oct 30, 2021
  4. mtrezza
    changed the title [-]Upgrade to mongodb 4.x[/-] [+]Upgrade to mongodb node driver 4.x[/+] on Oct 30, 2021
  5. mtrezza
    removed
    bounty:$10Bounty applies for fixing this issue (Parse Bounty Program)
    on Nov 3, 2021
  6. mtrezza
    changed the title [-]Upgrade to mongodb node driver 4.x[/-] [+]Upgrade mongodb node driver to 4.x for MongoDB 5 compatibility[/+] on Nov 3, 2021
  7. mtrezza
    added
    bounty:$50Bounty applies for fixing this issue (Parse Bounty Program)
    on Nov 3, 2021
  8. mtrezza
    pinned this issue on Nov 3, 2021

  9. mtrezza commented on Nov 3, 2021

    @mtrezza
    mtrezza
    on Nov 3, 2021
    MemberAuthor

    @parse-community/server Anyone wants to take on this critical upgrade?
    Given its time-criticality we'll also merge this into beta.

  10. mtrezza
    changed the title [-]Upgrade mongodb node driver to 4.x for MongoDB 5 compatibility[/-] [+]Add MongoDB 5 compatibility (critical)[/+] on Nov 3, 2021
  11. mtrezza
    changed the title [-]Add MongoDB 5 compatibility (critical)[/-] [+]Add MongoDB 5 compatibility[/+] on Nov 3, 2021

  12. Moumouls commented on Nov 4, 2021

    @Moumouls
    Moumouls
    on Nov 4, 2021
    Member

    Hi @mtrezza

    All M0/M2/M5 Atlas clusters will automatically be upgraded to MongoDB 5.0 in early February 2022. That means after that, Parse Server will be incompatible with these and all free MongoDB Atlas plans.

    Do you have the Mongo DB announcement link, just to track the direct source of the issue ?

    A separate question is whether we want / can easily port this back to the 4.x branch. If not, then users of free or small (non-dedicated) MongoDB Atlas plans will be forced to upgrade to Parse Server 5.0. With that comes the necessity to adapt the deployment for the breaking changes in Parse Server 5.0.

    Here Mongo Node JS driver v4 migration guide: https://github.com/mongodb/node-mongodb-native/blob/4.0/docs/CHANGES_4.0.0.md

    From my point of view and knowledge of parse-server, here the list of BK changes that can affect Parse:

    As we saw in https://github.com/parse-community/parse-server/pull/7626/files, since Parse Mongo driver is globally well written, i think it's feasible to backport the mongo v5 PR to parse-server 4 without too many effort.

    Here i'm also sorry i can't tell if i could work on this asap.

  13. mtrezza commented on Nov 4, 2021

    @mtrezza
    mtrezza
    on Nov 4, 2021
    MemberAuthor

    Do you have the Mongo DB announcement link, just to track the direct source of the issue ?

    It's standard MongoDB Atlas procedure for free and shared clusters. These are always kept on the latest version and users have no control over the MongoDB version. We are actually late on upgrading because the 4.x driver has been released back in July 2021 already.

    It's interesting though that we are currently running CI tests with MongoDB 5.0.3 but according to the compatibility table, MongoDB 5 requires the 4.x driver. Maybe this will be an Atlas specific restriction, or Parse Server is currently just not using or not testing features that are incompatible.

    @dariakp do you have any insight to what extend the 3.x driver works with MongoDB 5 or should we open a ticket?

  14. dariakp commented on Nov 4, 2021

    @dariakp
    dariakp
    on Nov 4, 2021

    @mtrezza I'm not sure I understand the question. The same features that work in the 3.x driver against pre-5.0 MongoDB servers should work against 5.0 MongoDB servers. However, the 3.x driver will not necessarily support the new features available in MongoDB 5.0. To get the new features, the 4.x driver should be used.

  15. mtrezza commented on Nov 4, 2021

    @mtrezza
    mtrezza
    on Nov 4, 2021
    MemberAuthor

    Thanks @dariakp that was my question. The compatibility table wasn't clear to me because it says that the 3.x driver is incompatible with MongoDB 5.0 but it doesn't specify what that means. That confirms our priority on this upgrade.

  16. Moumouls commented on Nov 4, 2021

    @Moumouls
    Moumouls
    on Nov 4, 2021
    Member

    @dariakp here can you confirm that parse server instances using the Mongo v3 driver will work with a Mongo DB v5.

    How many time MongoDB v5 will support v3 driver ?
    Following the recent mongo blog post about new version release calendar, a major MongoDB version each year, does MongoDB v6 in 2023 will drop the support of mongo driver v3 ?

    If v3 driver is fully compatible with MongoDB v5, this issue is not as critical as it seems, and parse servers using mongo atlas will just work normally.

    The #7586 could be developed/fixed later.

  17. mtrezza commented on Nov 4, 2021

    @mtrezza
    mtrezza
    on Nov 4, 2021
    MemberAuthor

    @Moumouls I think your questions have been answered already. This PR stays a priority for us, because we are already too far behind with the driver. Let's please keep this thread focused on the driver upgrade to 4.x. If you or someone else could reserve some time for it, that would be great 🙂

  18. Moumouls commented on Nov 4, 2021

    @Moumouls
    Moumouls
    on Nov 4, 2021
    Member

    @Moumouls I think your questions have been answered already

    Yes, but i think it could help us to start to prevent the community if the v3 driver will be not supported on MongoDB v6 (should be released in 2023 following the new mongo release rhythm). So developers on atlas will "only" have 1 year to upgrade there parse server before the old v3 driver will be dropped. @mtrezza

  19. mtrezza commented on Nov 4, 2021

    @mtrezza
    mtrezza
    on Nov 4, 2021
    MemberAuthor

    1 year to upgrade should be more than enough, we also don't give more time when we deprecate features. There are many other reasons why developers should upgrade to Parse Server 5.0. We will begin an upgrade campaign across the community once Parse Server 5.0 is released in Jan 2022 to facilitate that.

    We cannot say that Parse Server is compatible with MongoDB 5.0 if we don't know whether it supports all of the 5.0 features that are customizable via Parse Server, think aggregation pipeline or settings that Parse Server does not validate before being sent to the DB. In that sense, the current MongoDB 5.0 compatibility indicated in the Parse Server README may be incorrect. Upgrading the driver will give us assurance.

  20. mtrezza
    added
    type:featureNew feature or improvement of existing feature
    and removed
    type:improvement
    on Dec 6, 2021

  21. jonas-db commented on Jan 11, 2022

    @jonas-db
    jonas-db
    on Jan 11, 2022

    I just received an email about this:

    Your M0/M2/M5 Atlas cluster(s) will automatically be upgraded to MongoDB 5.0 in mid February 2022.

    What is the status of this ticket? Despite its importance, it seems almost nothing has been done in the past 2 months.
    Can someone of the head contributors make a clear checklist of what needs to be done? Also which versions of 4.x (only 4.1.1?) need to be supported?

  22. mtrezza commented on Jan 12, 2022

    @mtrezza
    mtrezza
    on Jan 12, 2022
    MemberAuthor
    1. Update MongoDB driver dependency to the latest version
    2. Open PR and see which tests fail in the CI
    3. Make changes to Parse Server to make tests pass

  23. Moumouls commented on Jan 30, 2022

    @Moumouls
    Moumouls
    on Jan 30, 2022
    Member

    @jonas-db i sent a PR to upgrade mongo driver to v4. But if you run on a recent parse server version like > 4.5.X driver v3 should works with mongo 5 in all major use cases.

    Here results of 4.5.X on Mongo 5

    2683 specs, 6 failures, 71 pending specs

    Failures

    1) OAuth Should fail a GET request
  Message:
    Uncaught exception: SyntaxError: Unexpected end of JSON input
  Stack:
        at <Jasmine>
        at IncomingMessage.<anonymous> (/Users/antoinecormouls/Desktop/OpenSource/parse-server/lib/Adapters/Auth/OAuth1Client.js:31:21)
        at IncomingMessage.emit (events.js:327:22)
        at IncomingMessage.EventEmitter.emit (domain.js:486:12)
        at endReadableNT (_stream_readable.js:1327:12)
        at processTicksAndRejections (internal/process/task_queues.js:80:21)

2) Parse.Query hint query aggregate with hint string
  Message:
    Expected 'PROJECTION_SIMPLE' to be 'COLLSCAN'.
  Stack:
    Error: Expected 'PROJECTION_SIMPLE' to be 'COLLSCAN'.
        at <Jasmine>
        at UserContext.<anonymous> (/Users/antoinecormouls/Desktop/OpenSource/parse-server/spec/ParseQuery.hint.spec.js:66:44)
        at <Jasmine>
        at processTicksAndRejections (internal/process/task_queues.js:93:5)
  Message:
    Expected 'PROJECTION_SIMPLE' to be 'FETCH'.
  Stack:
    Error: Expected 'PROJECTION_SIMPLE' to be 'FETCH'.
        at <Jasmine>
        at UserContext.<anonymous> (/Users/antoinecormouls/Desktop/OpenSource/parse-server/spec/ParseQuery.hint.spec.js:73:44)
        at <Jasmine>
        at processTicksAndRejections (internal/process/task_queues.js:93:5)
  Message:
    Expected undefined to be '_id_'.
  Stack:
    Error: Expected undefined to be '_id_'.
        at <Jasmine>
        at UserContext.<anonymous> (/Users/antoinecormouls/Desktop/OpenSource/parse-server/spec/ParseQuery.hint.spec.js:74:59)
        at <Jasmine>
        at processTicksAndRejections (internal/process/task_queues.js:93:5)

3) Parse.Query hint query aggregate with hint object
  Message:
    Expected 'PROJECTION_SIMPLE' to be 'COLLSCAN'.
  Stack:
    Error: Expected 'PROJECTION_SIMPLE' to be 'COLLSCAN'.
        at <Jasmine>
        at UserContext.<anonymous> (/Users/antoinecormouls/Desktop/OpenSource/parse-server/spec/ParseQuery.hint.spec.js:86:44)
        at <Jasmine>
        at processTicksAndRejections (internal/process/task_queues.js:93:5)
  Message:
    Expected 'PROJECTION_SIMPLE' to be 'FETCH'.
  Stack:
    Error: Expected 'PROJECTION_SIMPLE' to be 'FETCH'.
        at <Jasmine>
        at UserContext.<anonymous> (/Users/antoinecormouls/Desktop/OpenSource/parse-server/spec/ParseQuery.hint.spec.js:93:44)
        at <Jasmine>
        at processTicksAndRejections (internal/process/task_queues.js:93:5)
  Message:
    Expected undefined to equal Object({ _id: 1 }).
  Stack:
    Error: Expected undefined to equal Object({ _id: 1 }).
        at <Jasmine>
        at UserContext.<anonymous> (/Users/antoinecormouls/Desktop/OpenSource/parse-server/spec/ParseQuery.hint.spec.js:94:60)
        at <Jasmine>
        at processTicksAndRejections (internal/process/task_queues.js:93:5)

4) Parse.Query hint query aggregate with hint (rest)
  Message:
    Expected 'PROJECTION_SIMPLE' to be 'COLLSCAN'.
  Stack:
    Error: Expected 'PROJECTION_SIMPLE' to be 'COLLSCAN'.
        at <Jasmine>
        at UserContext.<anonymous> (/Users/antoinecormouls/Desktop/OpenSource/parse-server/spec/ParseQuery.hint.spec.js:134:44)
        at <Jasmine>
        at processTicksAndRejections (internal/process/task_queues.js:93:5)
  Message:
    Expected undefined to equal Object({ _id: 1 }).
  Stack:
    Error: Expected undefined to equal Object({ _id: 1 }).
        at <Jasmine>
        at UserContext.<anonymous> (/Users/antoinecormouls/Desktop/OpenSource/parse-server/spec/ParseQuery.hint.spec.js:146:60)
        at <Jasmine>
        at processTicksAndRejections (internal/process/task_queues.js:93:5)

5) Parse.Query testing Use a regex that requires all modifiers
  Message:
    Failed: {"message":{"ok":0,"code":51091,"codeName":"Location51091","name":"MongoError"},"code":1}
  Stack:
    Error: Failed: {"message":{"ok":0,"code":51091,"codeName":"Location51091","name":"MongoError"},"code":1}
        at <Jasmine>
        at global.jfail (/Users/antoinecormouls/Desktop/OpenSource/parse-server/spec/helper.js:393:3)
        at /Users/antoinecormouls/Desktop/OpenSource/parse-server/spec/ParseQuery.spec.js:2062:11
        at <Jasmine>
        at processTicksAndRejections (internal/process/task_queues.js:93:5)

6) Parse.Query testing select keys query
  Message:
    Expected false to be truthy 'other error: {"message":{"ok":0,"code":40352,"codeName":"Location40352","name":"MongoError"},"code":1}'.
  Stack:
    Error: Expected false to be truthy 'other error: {"message":{"ok":0,"code":40352,"codeName":"Location40352","name":"MongoError"},"code":1}'.
        at <Jasmine>
        at ok (/Users/antoinecormouls/Desktop/OpenSource/parse-server/spec/helper.js:265:16)
        at /Users/antoinecormouls/Desktop/OpenSource/parse-server/spec/ParseQuery.spec.js:3207:11
        at <Jasmine>

    @jonas-db i can suggest to test asap your (staging/pre prod) parse app with a mongo 5 (locally or atlas) then you will be able to verify if you are safe regarding your usage of parse 🙂

  24. mtrezza commented on Feb 6, 2022

    @mtrezza
    mtrezza
    on Feb 6, 2022
    MemberAuthor

    Closing via #7794; great work @Moumouls!

  25. mtrezza
    closed this as completedon Feb 6, 2022
  26. mtrezza
    unpinned this issue on Feb 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bounty:$50Bounty applies for fixing this issue (Parse Bounty Program)type:featureNew feature or improvement of existing feature

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions