Update CONTRIBUTE.rst - #4
Conversation
Minor improvements to CONTRIBUTE.rst: - add hyperlink to Ingestion Team - add a comment about pinning dependencies - add a comment about 10,001 documents
| 7. when possible, provide a docker image that runs the backend service, so we can test the connector | ||
| 8. if you can't provide a docker image, provide the credentials needed to run against a service | ||
| 9. The backend needs to return 10,001 documents. | ||
| 9. the test backend needs to return 10,001 documents due to internal implementation details of the connector - it batches the data into chunks 0f 10,000 items |
There was a problem hiding this comment.
this is not the reason actually. I added 10,001 because 10k is the default size limit for pagination, so having the whole flow work for +1 seems like a decent default for folks building and trying out their connectors https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html#index-max-result-window
There was a problem hiding this comment.
Oh right! I've changed the comment to:
the test backend needs to return 10,001 documents due to 10,000 being a default size limit for Elasticsearch pagination. Having 10,001 documents returned from the test backend will help testing connector more deeply
Does it look better?
| and `requirements.txt <requirements.txt>`_ will be rejected. | ||
|
|
||
| If you need changes in the framework, reach out the Ingestion team. | ||
| If you need changes in the framework, reach out to the `Ingestion team <https://github.com/orgs/elastic/teams/ingestion-team/members>`_. |
There was a problem hiding this comment.
maybe they can add an issue here? and we can provide a template?
There was a problem hiding this comment.
Hmmmm I think it makes sense. We have an issue template in connectors-ruby: https://github.com/elastic/connectors-ruby/issues/new?assignees=&labels=enhancement&template=enhancement.md&title=
Will this one work in this repo as well?
There was a problem hiding this comment.
I think it's an excellent start
Update the comment about 10,000 documents
## Summary Adds `release-eng: BUILD_AND_READ` to the `connectors` pipeline's teams block so release-eng team members can trigger builds. ## Why The `connectors-version-bump` pipeline (introduced in #3986) executes a `trigger:` step that starts a DRA build on the target release branch on the main `connectors` pipeline. Buildkite propagates the original triggering user's identity through trigger chains, so the downstream build on `connectors` is created as that user -- and the user needs Build access on `connectors` for the trigger to succeed. We already granted release-eng `BUILD_AND_READ` on `connectors-version-bump` (so they can trigger our pipeline at all), but missed adding it on the main `connectors` pipeline. This caused failures in [build #3](https://buildkite.com/elastic/connectors-version-bump/builds/3) (9.3.5) and [build #4](https://buildkite.com/elastic/connectors-version-bump/builds/4) (8.19.16) when triggered by Nina Lee, while [build #2](https://buildkite.com/elastic/connectors-version-bump/builds/2) (9.4.1) triggered by Julien Mailleret succeeded -- presumably via individual grants. This grants team-level access so any release-eng member can complete the version bump end-to-end. ## Test plan - [ ] After merge and Backstage sync (~few minutes), re-trigger build #3 or #4 from the centralized pipeline and confirm the `Trigger DRA build on ${BRANCH}` step succeeds --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2 participants
Minor improvements to CONTRIBUTE.rst: