[MongoDB] Handle out-of-range BSON datetimes to prevent sync failures - #4148
Merged
Merged
Conversation
Documents containing dates outside the Python datetime range (years 1-9999) caused pymongo to raise InvalidBSON during decode, aborting the entire sync. Configure the client with DATETIME_AUTO so such values are returned as DatetimeMS, and serialize them as raw milliseconds since epoch. Relates to elastic/sdh-search#1924 Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Switch datetime_conversion from DATETIME_AUTO to DATETIME_CLAMP so out-of-range BSON dates are clamped to datetime.min/max and stay valid dates. This avoids the mixed date/long Elasticsearch mapping conflict that DATETIME_AUTO could cause, and removes the now-unneeded DatetimeMS serialization branch. Co-authored-by: Cursor <cursoragent@cursor.com>
Member
|
Just checking - how is it related to the support case? |
Contributor
Author
Not directly. Looking into the support case just triggered agent to look into related issues and this was found. I decided it would be a good easy drive by fix. |
erikcurrin-elastic
approved these changes
Jul 10, 2026
Jan-Kazlouski-elastic
added
enhancement
Contributor
Author
|
As discussed offline with @artem-shelkovnikov , it was decided to make the logic of handling the out of bounds dates configurable. |
Add a "datetime_conversion" configuration option exposing every pymongo DatetimeConversion mode (DATETIME, DATETIME_CLAMP, DATETIME_AUTO, DATETIME_MS). Default is the legacy DATETIME behavior (raise on out-of-range dates), so users must opt in to clamping or epoch-ms conversion. Restore DatetimeMS serialization for the AUTO/MS modes and validate the configured value. Co-authored-by: Cursor <cursoragent@cursor.com>
Jan-Kazlouski-elastic
requested review from
artem-shelkovnikov and
erikcurrin-elastic
July 10, 2026 16:27
Contributor
Author
|
@erikcurrin-elastic thanks for the approval. We've had a chat with @artem-shelkovnikov today and decided it would be a good thing to have this behavior configurable. |
erikcurrin-elastic
approved these changes
Jul 10, 2026
erikcurrin-elastic
left a comment
erikcurrin-elastic
left a comment
Contributor
There was a problem hiding this comment.
The configuration option is definitely much better. Good test coverage on what threw errors and no longer does
This was referenced Jul 13, 2026
artem-shelkovnikov
approved these changes
Jul 13, 2026
artem-shelkovnikov
left a comment
artem-shelkovnikov
left a comment
Member
There was a problem hiding this comment.
LGTM! Imo this needs to be backported as far as possible
Jan-Kazlouski-elastic
enabled auto-merge (squash)
July 13, 2026 10:12
Jan-Kazlouski-elastic
added a commit
to elastic/elasticsearch
that referenced
this pull request
Jul 13, 2026
* Docs: MongoDB out-of-range date handling option Document the new datetime_conversion configuration field in the MongoDB connector reference, describing each mode and when to use clamping over epoch milliseconds. See elastic/connectors#4148. * Docs: align MongoDB date option with config keys Use the datetime_conversion config key and its raw enum values (DATETIME, DATETIME_CLAMP, DATETIME_AUTO, DATETIME_MS) so the field matches the other MongoDB settings and API/config.yml usage. * Docs: use "might" instead of "may" per Vale rule Satisfies the Elastic.WordChoice style rule in the MongoDB out-of-range date handling note.
This was referenced Jul 13, 2026
This was referenced Jul 13, 2026
Jan-Kazlouski-elastic
added a commit
that referenced
this pull request
Jul 13, 2026
…ilures (#4148) (#4156) Backports the following commits to 9.5: - [MongoDB] Handle out-of-range BSON datetimes to prevent sync failures (#4148) Co-authored-by: Jan-Kazlouski-elastic <jan.kazlouski@elastic.co> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Jan-Kazlouski-elastic
added a commit
that referenced
this pull request
Jul 13, 2026
…ilures (#4148) (#4155) Backports the following commits to 9.4: - [MongoDB] Handle out-of-range BSON datetimes to prevent sync failures (#4148) Co-authored-by: Jan-Kazlouski-elastic <jan.kazlouski@elastic.co> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Jan-Kazlouski-elastic
added a commit
that referenced
this pull request
Jul 13, 2026
…ailures (#4148) (#4157) Backports the following commits to 8.19: - [MongoDB] Handle out-of-range BSON datetimes to prevent sync failures (#4148) The automatic backport failed because 8.19 predates the connector package refactor and still uses the single-file `connectors/sources/mongo.py` layout (no `app/connectors_service/` prefix, `connectors/sources/mongo/` package, or `connectors_sdk` imports). This is a manual backport adapting the same changes to that layout. ## Changes - Add a configurable `datetime_conversion` option exposing every pymongo `DatetimeConversion` mode (DATETIME, DATETIME_CLAMP, DATETIME_AUTO, DATETIME_MS). Default is the legacy `DATETIME` behavior (raise on out-of-range dates), so users must opt in. - `get_client()`: pass the configured `datetime_conversion` to the MongoDB client. - `serialize()`: serialize `DatetimeMS` values (from AUTO/MS modes) as raw epoch milliseconds. - `validate_config()`: reject unknown `datetime_conversion` values. - Ported unit tests for serialization, client config, out-of-range decoding, and validation. ## Test plan - `.venv/bin/ruff check`/`format` on `connectors` and `tests`: pass - Full `pytest` suite: 2412 passed, coverage 92.40% (>= 92% required) Made with [Cursor](https://cursor.com) Co-authored-by: Cursor <cursoragent@cursor.com>
Jan-Kazlouski-elastic
added a commit
to elastic/kibana
that referenced
this pull request
Jul 14, 2026
…277685) ## Summary Adds the `datetime_conversion` ("Out-of-range date handling") configuration field to the MongoDB native connector definition in `native_connectors.ts`, mirroring the new field added to the connectors service in [elastic/connectors#4148](elastic/connectors#4148). This lets users choose how MongoDB dates outside the supported range (years 1–9999) are handled instead of aborting the sync: - `DATETIME` – Raise an error (legacy default behavior) - `DATETIME_CLAMP` – Clamp to the min/max date - `DATETIME_AUTO` – Out-of-range dates as epoch milliseconds - `DATETIME_MS` – All dates as epoch milliseconds The field is a dropdown under the advanced options, defaults to the legacy `DATETIME` behavior, and follows the existing dropdown configuration pattern used elsewhere in the file (i18n `translate` labels/tooltip). ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) - [ ] Review the [backport guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing) and apply applicable `backport:*` labels. ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ... Co-authored-by: Cursor <cursoragent@cursor.com>
kibanamachine
added a commit
to elastic/kibana
that referenced
this pull request
Jul 14, 2026
…ion (#277685) (#278014) # Backport This will backport the following commits from `main` to `9.4`: - [[Search][Connectors] Add MongoDB datetime_conversion configuration (#277685)](#277685) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Jan-Kazlouski-elastic","email":"jan.kazlouski@elastic.co"},"sourceCommit":{"committedDate":"2026-07-14T09:15:24Z","message":"[Search][Connectors] Add MongoDB datetime_conversion configuration (#277685)\n\n## Summary\n\nAdds the `datetime_conversion` (\"Out-of-range date handling\")\nconfiguration field to the MongoDB native connector definition in\n`native_connectors.ts`, mirroring the new field added to the connectors\nservice in\n[elastic/connectors#4148](https://github.com/elastic/connectors/pull/4148).\n\nThis lets users choose how MongoDB dates outside the supported range\n(years 1–9999) are handled instead of aborting the sync:\n\n- `DATETIME` – Raise an error (legacy default behavior)\n- `DATETIME_CLAMP` – Clamp to the min/max date\n- `DATETIME_AUTO` – Out-of-range dates as epoch milliseconds\n- `DATETIME_MS` – All dates as epoch milliseconds\n\nThe field is a dropdown under the advanced options, defaults to the\nlegacy `DATETIME` behavior, and follows the existing dropdown\nconfiguration pattern used elsewhere in the file (i18n `translate`\nlabels/tooltip).\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [ ] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [ ]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas added for features that require explanation or tutorials\n- [ ] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [ ] If a plugin configuration key changed, check if it needs to be\nallowlisted in the cloud and added to the [docker\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\n- [ ] This was checked for breaking HTTP API changes, and any breaking\nchanges have been approved by the breaking-change committee. The\n`release_note:breaking` label should be applied in these situations.\n- [ ] [Flaky Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\nused on any tests changed\n- [ ] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n- [ ] Review the [backport\nguidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)\nand apply applicable `backport:*` labels.\n\n### Identify risks\n\nDoes this PR introduce any risks? For example, consider risks like hard\nto test bugs, performance regression, potential of data loss.\n\nDescribe the risk, its severity, and mitigation for each identified\nrisk. Invite stakeholders and evaluate how to proceed before merging.\n\n- [ ] [See some risk\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)\n- [ ] ...\n\nCo-authored-by: Cursor <cursoragent@cursor.com>","sha":"fa6f6a4c7f6067e72c38154a2b0ca3a3659b664e","branchLabelMapping":{"^v9.6.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:enhancement","backport:version","v9.5.0","v9.4.4","v9.6.0","v8.19.19"],"title":"[Search][Connectors] Add MongoDB datetime_conversion configuration","number":277685,"url":"https://github.com/elastic/kibana/pull/277685","mergeCommit":{"message":"[Search][Connectors] Add MongoDB datetime_conversion configuration (#277685)\n\n## Summary\n\nAdds the `datetime_conversion` (\"Out-of-range date handling\")\nconfiguration field to the MongoDB native connector definition in\n`native_connectors.ts`, mirroring the new field added to the connectors\nservice in\n[elastic/connectors#4148](https://github.com/elastic/connectors/pull/4148).\n\nThis lets users choose how MongoDB dates outside the supported range\n(years 1–9999) are handled instead of aborting the sync:\n\n- `DATETIME` – Raise an error (legacy default behavior)\n- `DATETIME_CLAMP` – Clamp to the min/max date\n- `DATETIME_AUTO` – Out-of-range dates as epoch milliseconds\n- `DATETIME_MS` – All dates as epoch milliseconds\n\nThe field is a dropdown under the advanced options, defaults to the\nlegacy `DATETIME` behavior, and follows the existing dropdown\nconfiguration pattern used elsewhere in the file (i18n `translate`\nlabels/tooltip).\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [ ] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [ ]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas added for features that require explanation or tutorials\n- [ ] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [ ] If a plugin configuration key changed, check if it needs to be\nallowlisted in the cloud and added to the [docker\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\n- [ ] This was checked for breaking HTTP API changes, and any breaking\nchanges have been approved by the breaking-change committee. The\n`release_note:breaking` label should be applied in these situations.\n- [ ] [Flaky Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\nused on any tests changed\n- [ ] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n- [ ] Review the [backport\nguidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)\nand apply applicable `backport:*` labels.\n\n### Identify risks\n\nDoes this PR introduce any risks? For example, consider risks like hard\nto test bugs, performance regression, potential of data loss.\n\nDescribe the risk, its severity, and mitigation for each identified\nrisk. Invite stakeholders and evaluate how to proceed before merging.\n\n- [ ] [See some risk\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)\n- [ ] ...\n\nCo-authored-by: Cursor <cursoragent@cursor.com>","sha":"fa6f6a4c7f6067e72c38154a2b0ca3a3659b664e"}},"sourceBranch":"main","suggestedTargetBranches":["9.5","9.4","8.19"],"targetPullRequestStates":[{"branch":"9.5","label":"v9.5.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.4","label":"v9.4.4","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.6.0","branchLabelMappingKey":"^v9.6.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/277685","number":277685,"mergeCommit":{"message":"[Search][Connectors] Add MongoDB datetime_conversion configuration (#277685)\n\n## Summary\n\nAdds the `datetime_conversion` (\"Out-of-range date handling\")\nconfiguration field to the MongoDB native connector definition in\n`native_connectors.ts`, mirroring the new field added to the connectors\nservice in\n[elastic/connectors#4148](https://github.com/elastic/connectors/pull/4148).\n\nThis lets users choose how MongoDB dates outside the supported range\n(years 1–9999) are handled instead of aborting the sync:\n\n- `DATETIME` – Raise an error (legacy default behavior)\n- `DATETIME_CLAMP` – Clamp to the min/max date\n- `DATETIME_AUTO` – Out-of-range dates as epoch milliseconds\n- `DATETIME_MS` – All dates as epoch milliseconds\n\nThe field is a dropdown under the advanced options, defaults to the\nlegacy `DATETIME` behavior, and follows the existing dropdown\nconfiguration pattern used elsewhere in the file (i18n `translate`\nlabels/tooltip).\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [ ] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [ ]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas added for features that require explanation or tutorials\n- [ ] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [ ] If a plugin configuration key changed, check if it needs to be\nallowlisted in the cloud and added to the [docker\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\n- [ ] This was checked for breaking HTTP API changes, and any breaking\nchanges have been approved by the breaking-change committee. The\n`release_note:breaking` label should be applied in these situations.\n- [ ] [Flaky Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\nused on any tests changed\n- [ ] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n- [ ] Review the [backport\nguidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)\nand apply applicable `backport:*` labels.\n\n### Identify risks\n\nDoes this PR introduce any risks? For example, consider risks like hard\nto test bugs, performance regression, potential of data loss.\n\nDescribe the risk, its severity, and mitigation for each identified\nrisk. Invite stakeholders and evaluate how to proceed before merging.\n\n- [ ] [See some risk\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)\n- [ ] ...\n\nCo-authored-by: Cursor <cursoragent@cursor.com>","sha":"fa6f6a4c7f6067e72c38154a2b0ca3a3659b664e"}},{"branch":"8.19","label":"v8.19.19","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Jan-Kazlouski-elastic <jan.kazlouski@elastic.co> Co-authored-by: Cursor <cursoragent@cursor.com>
kibanamachine
added a commit
to elastic/kibana
that referenced
this pull request
Jul 14, 2026
…ion (#277685) (#278015) # Backport This will backport the following commits from `main` to `9.5`: - [[Search][Connectors] Add MongoDB datetime_conversion configuration (#277685)](#277685) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Jan-Kazlouski-elastic","email":"jan.kazlouski@elastic.co"},"sourceCommit":{"committedDate":"2026-07-14T09:15:24Z","message":"[Search][Connectors] Add MongoDB datetime_conversion configuration (#277685)\n\n## Summary\n\nAdds the `datetime_conversion` (\"Out-of-range date handling\")\nconfiguration field to the MongoDB native connector definition in\n`native_connectors.ts`, mirroring the new field added to the connectors\nservice in\n[elastic/connectors#4148](https://github.com/elastic/connectors/pull/4148).\n\nThis lets users choose how MongoDB dates outside the supported range\n(years 1–9999) are handled instead of aborting the sync:\n\n- `DATETIME` – Raise an error (legacy default behavior)\n- `DATETIME_CLAMP` – Clamp to the min/max date\n- `DATETIME_AUTO` – Out-of-range dates as epoch milliseconds\n- `DATETIME_MS` – All dates as epoch milliseconds\n\nThe field is a dropdown under the advanced options, defaults to the\nlegacy `DATETIME` behavior, and follows the existing dropdown\nconfiguration pattern used elsewhere in the file (i18n `translate`\nlabels/tooltip).\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [ ] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [ ]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas added for features that require explanation or tutorials\n- [ ] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [ ] If a plugin configuration key changed, check if it needs to be\nallowlisted in the cloud and added to the [docker\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\n- [ ] This was checked for breaking HTTP API changes, and any breaking\nchanges have been approved by the breaking-change committee. The\n`release_note:breaking` label should be applied in these situations.\n- [ ] [Flaky Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\nused on any tests changed\n- [ ] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n- [ ] Review the [backport\nguidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)\nand apply applicable `backport:*` labels.\n\n### Identify risks\n\nDoes this PR introduce any risks? For example, consider risks like hard\nto test bugs, performance regression, potential of data loss.\n\nDescribe the risk, its severity, and mitigation for each identified\nrisk. Invite stakeholders and evaluate how to proceed before merging.\n\n- [ ] [See some risk\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)\n- [ ] ...\n\nCo-authored-by: Cursor <cursoragent@cursor.com>","sha":"fa6f6a4c7f6067e72c38154a2b0ca3a3659b664e","branchLabelMapping":{"^v9.6.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:enhancement","backport:version","v9.5.0","v9.4.4","v9.6.0","v8.19.19"],"title":"[Search][Connectors] Add MongoDB datetime_conversion configuration","number":277685,"url":"https://github.com/elastic/kibana/pull/277685","mergeCommit":{"message":"[Search][Connectors] Add MongoDB datetime_conversion configuration (#277685)\n\n## Summary\n\nAdds the `datetime_conversion` (\"Out-of-range date handling\")\nconfiguration field to the MongoDB native connector definition in\n`native_connectors.ts`, mirroring the new field added to the connectors\nservice in\n[elastic/connectors#4148](https://github.com/elastic/connectors/pull/4148).\n\nThis lets users choose how MongoDB dates outside the supported range\n(years 1–9999) are handled instead of aborting the sync:\n\n- `DATETIME` – Raise an error (legacy default behavior)\n- `DATETIME_CLAMP` – Clamp to the min/max date\n- `DATETIME_AUTO` – Out-of-range dates as epoch milliseconds\n- `DATETIME_MS` – All dates as epoch milliseconds\n\nThe field is a dropdown under the advanced options, defaults to the\nlegacy `DATETIME` behavior, and follows the existing dropdown\nconfiguration pattern used elsewhere in the file (i18n `translate`\nlabels/tooltip).\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [ ] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [ ]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas added for features that require explanation or tutorials\n- [ ] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [ ] If a plugin configuration key changed, check if it needs to be\nallowlisted in the cloud and added to the [docker\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\n- [ ] This was checked for breaking HTTP API changes, and any breaking\nchanges have been approved by the breaking-change committee. The\n`release_note:breaking` label should be applied in these situations.\n- [ ] [Flaky Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\nused on any tests changed\n- [ ] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n- [ ] Review the [backport\nguidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)\nand apply applicable `backport:*` labels.\n\n### Identify risks\n\nDoes this PR introduce any risks? For example, consider risks like hard\nto test bugs, performance regression, potential of data loss.\n\nDescribe the risk, its severity, and mitigation for each identified\nrisk. Invite stakeholders and evaluate how to proceed before merging.\n\n- [ ] [See some risk\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)\n- [ ] ...\n\nCo-authored-by: Cursor <cursoragent@cursor.com>","sha":"fa6f6a4c7f6067e72c38154a2b0ca3a3659b664e"}},"sourceBranch":"main","suggestedTargetBranches":["9.5","9.4","8.19"],"targetPullRequestStates":[{"branch":"9.5","label":"v9.5.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.4","label":"v9.4.4","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.6.0","branchLabelMappingKey":"^v9.6.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/277685","number":277685,"mergeCommit":{"message":"[Search][Connectors] Add MongoDB datetime_conversion configuration (#277685)\n\n## Summary\n\nAdds the `datetime_conversion` (\"Out-of-range date handling\")\nconfiguration field to the MongoDB native connector definition in\n`native_connectors.ts`, mirroring the new field added to the connectors\nservice in\n[elastic/connectors#4148](https://github.com/elastic/connectors/pull/4148).\n\nThis lets users choose how MongoDB dates outside the supported range\n(years 1–9999) are handled instead of aborting the sync:\n\n- `DATETIME` – Raise an error (legacy default behavior)\n- `DATETIME_CLAMP` – Clamp to the min/max date\n- `DATETIME_AUTO` – Out-of-range dates as epoch milliseconds\n- `DATETIME_MS` – All dates as epoch milliseconds\n\nThe field is a dropdown under the advanced options, defaults to the\nlegacy `DATETIME` behavior, and follows the existing dropdown\nconfiguration pattern used elsewhere in the file (i18n `translate`\nlabels/tooltip).\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [ ] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [ ]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas added for features that require explanation or tutorials\n- [ ] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [ ] If a plugin configuration key changed, check if it needs to be\nallowlisted in the cloud and added to the [docker\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\n- [ ] This was checked for breaking HTTP API changes, and any breaking\nchanges have been approved by the breaking-change committee. The\n`release_note:breaking` label should be applied in these situations.\n- [ ] [Flaky Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\nused on any tests changed\n- [ ] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n- [ ] Review the [backport\nguidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)\nand apply applicable `backport:*` labels.\n\n### Identify risks\n\nDoes this PR introduce any risks? For example, consider risks like hard\nto test bugs, performance regression, potential of data loss.\n\nDescribe the risk, its severity, and mitigation for each identified\nrisk. Invite stakeholders and evaluate how to proceed before merging.\n\n- [ ] [See some risk\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)\n- [ ] ...\n\nCo-authored-by: Cursor <cursoragent@cursor.com>","sha":"fa6f6a4c7f6067e72c38154a2b0ca3a3659b664e"}},{"branch":"8.19","label":"v8.19.19","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Jan-Kazlouski-elastic <jan.kazlouski@elastic.co> Co-authored-by: Cursor <cursoragent@cursor.com>
biamalveiro
pushed a commit
to biamalveiro/kibana
that referenced
this pull request
Jul 14, 2026
…lastic#277685) ## Summary Adds the `datetime_conversion` ("Out-of-range date handling") configuration field to the MongoDB native connector definition in `native_connectors.ts`, mirroring the new field added to the connectors service in [elastic/connectors#4148](elastic/connectors#4148). This lets users choose how MongoDB dates outside the supported range (years 1–9999) are handled instead of aborting the sync: - `DATETIME` – Raise an error (legacy default behavior) - `DATETIME_CLAMP` – Clamp to the min/max date - `DATETIME_AUTO` – Out-of-range dates as epoch milliseconds - `DATETIME_MS` – All dates as epoch milliseconds The field is a dropdown under the advanced options, defaults to the legacy `DATETIME` behavior, and follows the existing dropdown configuration pattern used elsewhere in the file (i18n `translate` labels/tooltip). ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) - [ ] Review the [backport guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing) and apply applicable `backport:*` labels. ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ... Co-authored-by: Cursor <cursoragent@cursor.com>
Jan-Kazlouski-elastic
added a commit
that referenced
this pull request
Jul 15, 2026
…ilures (#4148) (#4166) Backports the following commits to 9.3: - [MongoDB] Handle out-of-range BSON datetimes to prevent sync failures (#4148) Manual backport: the original PR was labeled for `8.19`, `9.4`, `9.5`, `9.6` but not `9.3`, so the automatic backport never ran for this branch. 9.3 is still receiving patch releases (9.3.8), so this bug fix belongs here too. Cherry-picked cleanly from `62aa206` (`git cherry-pick -x`). Made with [Cursor](https://cursor.com) --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
kibanamachine
added a commit
to elastic/kibana
that referenced
this pull request
Jul 15, 2026
…tion (#277685) (#278013) # Backport This will backport the following commits from `main` to `8.19`: - [[Search][Connectors] Add MongoDB datetime_conversion configuration (#277685)](#277685) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Jan-Kazlouski-elastic","email":"jan.kazlouski@elastic.co"},"sourceCommit":{"committedDate":"2026-07-14T09:15:24Z","message":"[Search][Connectors] Add MongoDB datetime_conversion configuration (#277685)\n\n## Summary\n\nAdds the `datetime_conversion` (\"Out-of-range date handling\")\nconfiguration field to the MongoDB native connector definition in\n`native_connectors.ts`, mirroring the new field added to the connectors\nservice in\n[elastic/connectors#4148](https://github.com/elastic/connectors/pull/4148).\n\nThis lets users choose how MongoDB dates outside the supported range\n(years 1–9999) are handled instead of aborting the sync:\n\n- `DATETIME` – Raise an error (legacy default behavior)\n- `DATETIME_CLAMP` – Clamp to the min/max date\n- `DATETIME_AUTO` – Out-of-range dates as epoch milliseconds\n- `DATETIME_MS` – All dates as epoch milliseconds\n\nThe field is a dropdown under the advanced options, defaults to the\nlegacy `DATETIME` behavior, and follows the existing dropdown\nconfiguration pattern used elsewhere in the file (i18n `translate`\nlabels/tooltip).\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [ ] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [ ]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas added for features that require explanation or tutorials\n- [ ] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [ ] If a plugin configuration key changed, check if it needs to be\nallowlisted in the cloud and added to the [docker\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\n- [ ] This was checked for breaking HTTP API changes, and any breaking\nchanges have been approved by the breaking-change committee. The\n`release_note:breaking` label should be applied in these situations.\n- [ ] [Flaky Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\nused on any tests changed\n- [ ] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n- [ ] Review the [backport\nguidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)\nand apply applicable `backport:*` labels.\n\n### Identify risks\n\nDoes this PR introduce any risks? For example, consider risks like hard\nto test bugs, performance regression, potential of data loss.\n\nDescribe the risk, its severity, and mitigation for each identified\nrisk. Invite stakeholders and evaluate how to proceed before merging.\n\n- [ ] [See some risk\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)\n- [ ] ...\n\nCo-authored-by: Cursor <cursoragent@cursor.com>","sha":"fa6f6a4c7f6067e72c38154a2b0ca3a3659b664e","branchLabelMapping":{"^v9.6.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:enhancement","backport:version","v9.5.0","v9.4.4","v9.6.0","v8.19.19"],"title":"[Search][Connectors] Add MongoDB datetime_conversion configuration","number":277685,"url":"https://github.com/elastic/kibana/pull/277685","mergeCommit":{"message":"[Search][Connectors] Add MongoDB datetime_conversion configuration (#277685)\n\n## Summary\n\nAdds the `datetime_conversion` (\"Out-of-range date handling\")\nconfiguration field to the MongoDB native connector definition in\n`native_connectors.ts`, mirroring the new field added to the connectors\nservice in\n[elastic/connectors#4148](https://github.com/elastic/connectors/pull/4148).\n\nThis lets users choose how MongoDB dates outside the supported range\n(years 1–9999) are handled instead of aborting the sync:\n\n- `DATETIME` – Raise an error (legacy default behavior)\n- `DATETIME_CLAMP` – Clamp to the min/max date\n- `DATETIME_AUTO` – Out-of-range dates as epoch milliseconds\n- `DATETIME_MS` – All dates as epoch milliseconds\n\nThe field is a dropdown under the advanced options, defaults to the\nlegacy `DATETIME` behavior, and follows the existing dropdown\nconfiguration pattern used elsewhere in the file (i18n `translate`\nlabels/tooltip).\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [ ] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [ ]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas added for features that require explanation or tutorials\n- [ ] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [ ] If a plugin configuration key changed, check if it needs to be\nallowlisted in the cloud and added to the [docker\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\n- [ ] This was checked for breaking HTTP API changes, and any breaking\nchanges have been approved by the breaking-change committee. The\n`release_note:breaking` label should be applied in these situations.\n- [ ] [Flaky Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\nused on any tests changed\n- [ ] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n- [ ] Review the [backport\nguidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)\nand apply applicable `backport:*` labels.\n\n### Identify risks\n\nDoes this PR introduce any risks? For example, consider risks like hard\nto test bugs, performance regression, potential of data loss.\n\nDescribe the risk, its severity, and mitigation for each identified\nrisk. Invite stakeholders and evaluate how to proceed before merging.\n\n- [ ] [See some risk\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)\n- [ ] ...\n\nCo-authored-by: Cursor <cursoragent@cursor.com>","sha":"fa6f6a4c7f6067e72c38154a2b0ca3a3659b664e"}},"sourceBranch":"main","suggestedTargetBranches":["9.5","9.4","8.19"],"targetPullRequestStates":[{"branch":"9.5","label":"v9.5.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.4","label":"v9.4.4","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.6.0","branchLabelMappingKey":"^v9.6.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/277685","number":277685,"mergeCommit":{"message":"[Search][Connectors] Add MongoDB datetime_conversion configuration (#277685)\n\n## Summary\n\nAdds the `datetime_conversion` (\"Out-of-range date handling\")\nconfiguration field to the MongoDB native connector definition in\n`native_connectors.ts`, mirroring the new field added to the connectors\nservice in\n[elastic/connectors#4148](https://github.com/elastic/connectors/pull/4148).\n\nThis lets users choose how MongoDB dates outside the supported range\n(years 1–9999) are handled instead of aborting the sync:\n\n- `DATETIME` – Raise an error (legacy default behavior)\n- `DATETIME_CLAMP` – Clamp to the min/max date\n- `DATETIME_AUTO` – Out-of-range dates as epoch milliseconds\n- `DATETIME_MS` – All dates as epoch milliseconds\n\nThe field is a dropdown under the advanced options, defaults to the\nlegacy `DATETIME` behavior, and follows the existing dropdown\nconfiguration pattern used elsewhere in the file (i18n `translate`\nlabels/tooltip).\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [ ] Any text added follows [EUI's writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\nsentence case text and includes [i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n- [ ]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas added for features that require explanation or tutorials\n- [ ] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [ ] If a plugin configuration key changed, check if it needs to be\nallowlisted in the cloud and added to the [docker\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\n- [ ] This was checked for breaking HTTP API changes, and any breaking\nchanges have been approved by the breaking-change committee. The\n`release_note:breaking` label should be applied in these situations.\n- [ ] [Flaky Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\nused on any tests changed\n- [ ] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n- [ ] Review the [backport\nguidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)\nand apply applicable `backport:*` labels.\n\n### Identify risks\n\nDoes this PR introduce any risks? For example, consider risks like hard\nto test bugs, performance regression, potential of data loss.\n\nDescribe the risk, its severity, and mitigation for each identified\nrisk. Invite stakeholders and evaluate how to proceed before merging.\n\n- [ ] [See some risk\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)\n- [ ] ...\n\nCo-authored-by: Cursor <cursoragent@cursor.com>","sha":"fa6f6a4c7f6067e72c38154a2b0ca3a3659b664e"}},{"branch":"8.19","label":"v8.19.19","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> --------- Co-authored-by: Jan-Kazlouski-elastic <jan.kazlouski@elastic.co> Co-authored-by: Cursor <cursoragent@cursor.com>
1 of 3 tasks
patrykkopycinski
pushed a commit
to patrykkopycinski/kibana
that referenced
this pull request
Aug 5, 2026
…lastic#277685) ## Summary Adds the `datetime_conversion` ("Out-of-range date handling") configuration field to the MongoDB native connector definition in `native_connectors.ts`, mirroring the new field added to the connectors service in [elastic/connectors#4148](elastic/connectors#4148). This lets users choose how MongoDB dates outside the supported range (years 1–9999) are handled instead of aborting the sync: - `DATETIME` – Raise an error (legacy default behavior) - `DATETIME_CLAMP` – Clamp to the min/max date - `DATETIME_AUTO` – Out-of-range dates as epoch milliseconds - `DATETIME_MS` – All dates as epoch milliseconds The field is a dropdown under the advanced options, defaults to the legacy `DATETIME` behavior, and follows the existing dropdown configuration pattern used elsewhere in the file (i18n `translate` labels/tooltip). ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) - [ ] Review the [backport guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing) and apply applicable `backport:*` labels. ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ... Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Documents containing dates outside the Python
datetimerange (years 1–9999) cause pymongo to raiseInvalidBSONwhile decoding, which aborts the entire MongoDB sync:This PR configures the MongoDB client with
DatetimeConversion.DATETIME_CLAMP, so out-of-range values are clamped todatetime.min/datetime.maxand remain valid dates. They then serialize to normal ISO date strings, exactly like in-range dates.Why DATETIME_CLAMP (and not DATETIME_AUTO)
The earlier (closed) PR #3476 used
DATETIME_AUTO, which keeps in-range values as dates but converts out-of-range values tolong(epoch millis). That makes a single field hold mixed types (ISO string vs. long). Because Elasticsearch infers a field's type from the first document it sees, a later document of the other type is then rejected — turning the hard crash into an order-dependent ingestion failure.DATETIME_CLAMPavoids this entirely: every date serializes as a date string, so there is one consistent mapping, no crash, and no breaking change for existing users. The trade-off is that extreme dates lose their exact value (clamped to the datetime bounds) — an acceptable outcome for what are almost always sentinel/garbage dates.Context
This revives and supersedes the work from the earlier (closed) PR #3476, which was never merged and predates the connector package refactor (it targeted the old single-file
connectors/sources/mongo.py). This PR ports the fix to the currentconnectors/sources/mongo/package layout, switches the strategy to clamping, and adds unit test coverage.Changes
get_client(): setdatetime_conversion=DatetimeConversion.DATETIME_CLAMP.serialize(): no special date handling needed — clamped values are ordinarydatetimeobjects and go through the existingdatetime -> isoformat()branch.DATETIME_CLAMP, and an end-to-end test that encodes an out-of-range BSON date, decodes it with the client codec, and confirms it becomes a clampeddatetimeserialized as an ISO string (never aDatetimeMSor a number).Test plan
make clean install autoformat lint test PYTHON=python3.11(all MongoDB tests pass; unrelated pre-existing flaky failures only)Release Note
MongoDB connector: set default
datetime_conversiontoDatetimeConversion.DATETIME_CLAMPso out-of-rangedatetimevalues from MongoDB are clamped to valid dates instead of aborting the sync. See https://www.mongodb.com/docs/languages/python/pymongo-driver/current/data-formats/dates-and-times/#handling-out-of-range-datetimes for additional information.