Skip to content

Requirements: Model-building CI #487

Closed

Description

@jastram

WP4 requested continuous integration for the Scade model, and indicated that WP3 is likely interested as well. Here are the assumptions and requirements for such a system:

  • At DLR, there is already a build computer that allow the automated building of the Scade Model. The job of the CI system would be to trigger the build, "when required".
  • As a build system would always require a Scade license, the build computer will reside at DLR.
  • Making the build computer visible to the Internet would be cumbersome. Therefore, the build should be triggered via a "pull", rather than a "push".
  • The build would be triggered by observing certain files in the git repository.

Specification

Status file

  • The file grows continuously, appending new status at the end of this file.
  • Each time the status is updated, two lines are appended:
    ** A timestamp, in the form of an integer (seconds passed since 1970)
    ** A number, indicating the number of files changes in the /model/Scade folder

Reasoning

Just updating the status would be problematic: imagine two subsequent changes that are picked up by Jenkins: the first one with changes in the Scade folder, the second without. If we'd just update the status, the change could be missed. Using timestamps allows the requesting script to memorize how far the status file had been processed last time. Any non-zero value since that timestamp would count as a significant change.

Activity

  1. self-assigned this
    on Apr 17, 2015
  2. jastram commented on Apr 17, 2015

    @jastram
    MemberAuthor

    Possible solution discussed by @MarcBehrens and @jastram :

    We already have Jenkins (via CloudBees) in place. Jenkins is capable of observing a git repository. Therefore, the process could work as follows:

    • We set up a dedicated Jenkins project for the model building
    • This project is configured to observe the model files in the git repository
    • If a relevant change is detected, Jenkins will push a new file (the trigger-file) to the git repository (e.g. /build-pricess/build-requested.txt)
    • The build computer (at DLR) runs a cron job that checks for the existence of that file on a regular basis.
    • If the file is found, it triggers the building locally
    • Upon completion, the build artifacts are processed (pushed to git, uploaded somewhere, etc.)
    • Upon completion, the trigger-file is deleted and the deletion pushed back to git.
  3. MarcBehrens commented on May 20, 2015

    @MarcBehrens
    Collaborator

    The environment and automation for the build computer is already available.
    First results of a complete code generation and compilation is available at openETCS/modeling#384 .
    Is there an upate on the CI?

  4. jastram commented on May 21, 2015

    @jastram
    MemberAuthor

    Hello Marc,

    Unfortunately, nothing new yet, it's in the pipeline.

  5. jastram commented on Jun 4, 2015

    @jastram
    MemberAuthor

    Hello Marc,

    We're making some progress here and started to set up the infrastructure. While we do this, a few questions:

    1. What changes shall we monitor, exactly: Which repository (validation or model)? Which files?
    2. We can put a marker file in the repository, as proposed above. But keep in mind that this will create a lot of commits that are not really meaningful. Is this really what you want? It's probably very convenient for you. If you want Jenkins to write a marker file, then Jenkins needs write permissions to the repository. We'll have to set up a corresponding account.
    3. If you decide against a marker file in the repository, we can put a marker file on the cloudbees server. You could then simply check the content of that file in a regular interval by retrieving it with wget or similar.

    So: Which route would you like to go, 2 or 3? My recommendation is 3.

    • Michael
  6. MarcBehrens commented on Jun 4, 2015

    @MarcBehrens
    Collaborator

    Hello Michael,

    To 1) since we are working on SCADE code the entrance point is the folder

    https://github.com/openETCS/modeling/tree/master/model/Scade

    and below.

    To 2) and 3) A marker file would be a good way for the second step.

    I think the first step would be to get a nightly automatic code generation and compilation running with the aim to provide a nightly executable of the OBU.

    This means that we would provide the nightly code which is then further processed.

    Should we also provide the executable or is it possible to process this on cloudbee?

    In a second step we could then talk about a marker or summary within the cloudbees server providing only an update of code/executable when changes occur.

    I would like to keep the commits to the modeling repository as slim as possible.

    Kind regards

    Marc

  7. jastram commented on Jun 4, 2015

    @jastram
    MemberAuthor

    Hello Marc,

    Please check out the updated description of this issue, where I propose a new implementation. Would that work for you?

  8. jastram commented on Jun 9, 2015

    @jastram
    MemberAuthor

    @MarcBehrens - CloudBees is now connected to the modeling repository and checks once an hour for changes. As far as I am concerned, you can start using this for your automation.

    So essentially, we are done with this. I will therefore close this issue.

    I filed #492 as a follow-up, as you requested at the Backlog grooming session.

  9. RobertoKretschmer commented on Jun 10, 2015

    @RobertoKretschmer

    I will be out of the office starting 05.06.2015 and will not return until 20.06.2015

    Ich werde mich sobald wie m�glich bei Ihnen melden.
    Bei dringenden Angelegenheiten wenden Sie sich bitte an:
    Carsten.Kuebler@twt-gmbh.de

    I will get back to you as soon as possible.
    In case of urgent matters please contact:
    Carsten.Kuebler@twt-gmbh.de

    Hinweis: Dies ist eine automatische Antwort auf Ihre Nachricht  "Re: [toolchain] Requirements: Model-building CI (#487)" gesendet am 09.06.2015 23:00:27.

    Diese ist die einzige Benachrichtigung, die Sie empfangen werden, w�hrend diese Person abwesend ist.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions