Improvement: Fix audit 4 findings #1085

During Exida audit 4 on May-12 we recorded the following MoM (now reworked with Exida Audit report input):

Problem Mgt:

• Deviation_1: The Status “in review” needs to be defined. The use of an “ISO 26262 Impact Analysis” needs to be defined. The target must be to analyze which ISO work products are affected by the change. The activities to perform an ISO 26262 compliant Impact Analysis need to be described.
• Action_13: implement Problem Template in tool
• Deviation_2: The ASIL classification of the Problem is required, but it remains unclear how this will be done. Probably by “Labels”, but no clear definition is available. It remains unclear how the classification is done if e.g., safety and security is affected.
• Deviation_3: The ASIL classification shall be verified by a competent person, as not all contributors are necessarily competent safety engineers. This shall be explicitly mentioned in the process definition. The verification shall be documented.
• Action_14: The list of blocker causes shall be formatted as “list” as otherwise it can be misinterpreted. Make clear that safety relevant finding is always blocker.

Management of Safety Anomalies:

• Deviation_4: For safety classified issues (problems), it is not defined how affected versions/ released products are identified and potential product recalls are identified. It is imperative to identify the exact moment when the problem occurs. The process of notification of all users needs to be described. The responsibility of the user of the product shall be clarified: the user is responsible for checking for known safety issues (this requires a process of publishing known safety issues (with version information).
• Deviation_5: The Safety Manager shall be able to bypass the Technical Lead for escalation of safety anomalies.
  Document: https://eclipse-score.github.io/score/main/platform_management_plan/safety_management.html
• Action_19: The activities / arguments when a Safety Anomaly can be considered as managed are missing. It might be that the future Problem Resolution Process covers the requirements of a Safety Anomaly process

Change Mgt:

• Deviation_17: The Impact Analysis on Element Level is not defined by the process definition. It shall be identified which ISO 26262 work products are affected by the change.
• Deviation_18: To define that the committer is a competent person because he got training for 16h or got any training by a Safety Person will not guarantee an adequate level of safety knowledge to fulfill the responsibilities. Several years of professional safety projects experience with evidence shall be required.

Config Mgt:

• Small finding: github should be termed “version management tool”
• Deviation_15: There is no description for the baselining process which includes several GitHub repositories.
• Deviation_16: It remains unclear how the baselines, caused by the needs of different companies, are controlled. Companies might require changes of an Eclipse baseline for their SOP. Is it really realistic that changes are made in components where the component owner is not part of the specific company (key message: BMW would never change a Vector delivery)?

Document Mgt Plan:

• Deviation_20: The Documentation Plan (not a mirror of the existing documents) shall refer to the relevant Safety Plans of the platform and the exact module safety plan. The intention is to have a single source for the definition of all documents (Configuration items).
• Action_1: Documents do not surface a version number nor a date in Eclipse (Sphinx-Needs).
• Action_2: It’s not clear how the surfaced version in Sphinx-Needs links to GitHub.
• Finding: how to know how many modules are planned? - Improvement: doc mgt fix feature plan #1105

Tool Mgt:

• Action: Tool use case detail – create a use case description example
• Finding: upper arrow in TCL figure can be removed
• Action_3: The tool evaluation is still in progress.
• Action_4: A tool list needs to be created, there are no specific processes yet defined, no templates are created.

This comes from Audit Report analysis (all other Actions/Deviations are in #537:

Still open from #1&2:

• Action_17: Cybersecurity is not yet addressed. - Improvement: Document (Cyber)-Security process #709

New (also from audit number 3):

Quality Mgt:

• Deviation_6: Quality Management Manager roles are lacking details as processes are not yet defined. - Improvement: Document Quality Management process #316

Safety Management:

• Deviation_7: The escalation for the Safety Responsible shall not be the Technical Lead (project management) (but to the Project Lead, as the project lead is the ultimate decision point).
• Deviation_8: There is no role defined, which covers the Software Safety Analysis. The committer is intended to cover the Software Safety Analysis, but this is not part of the role description yet.
• Deviation_9: The statement “There will be HW / SW integration tests of feature requirements, as required by ISO 26262 part 6.” is wrong. The PMP needs to be changed as there are no such requirements in part 6. HW/ SW integration is covered by part 4.

Release Mgt:

• Action_23: It shall be defined, to which release (major release, or patch release) the delta needs to be shown. The originating release tag needs to be documented.
• Action_24: The module Verification report(s) shall be referenced.
• Action_25: The Safety Package (as part of the users’ Safety Case) shall be prepared.
• Action_26: It needs to be described, how the Project Management circle “approves” the release with “signature”. (it was discussed that this is a tick mark in the release ticket, and that this tick with a clear allocation to a person is equivalent to a signature).

Requirements:

• Action_45: ASIL D shall be removed from the requirements process description.
• Deviation_14: The definition of the participants of the inspection reviews needs to be done. Several roles shall be defined for the inspectors. At least the content responsible and the verification responsible shall be defined for the requirements. Other review properties might be done automatically.
• Action_46: The Configuration Management of the requirements needs to be audited. Baselines of a complete set of requirements shall be described, not only the version management of single requirements. - The release process and the Branching concept for release candidates are not yet described. The above action remains.

Architecture:

• Action_32: Is there a traceability between the architectural models? (not only on requirements level).

Implementation:

• Action_27: No coding guideline for RUST exists. Coding standards need to be defined.
• Action_28: Coding Standard for C++ is currently C++17. May need to be revised to a newer standard.
• Action_41: A Code Guideline for RUST shall be referenced. (in the PMP)
• Action_42: The acceptance criteria for the static code analysis (static, dynamic) shall be defined.

Verification:

• Deviation_13: The method selection for the unit tests seems to be too theoretical or wrong. How is a control flow and data flow analysis made on Unit level? Why is requirements based testing and interface based testing not selected? The table should be reworked in accordance to the mapping of the specified test levels to the ISO 26262 test levels.
• Action_43: The mapping of the test levels to the ISO 26262-6 test levels shall be shown e.g., in the PMP.
• Action_44: Provide an argumentation about which tests have been selected for a feature.
  • Base for this is done. Further improvement planed in Pending actions for verfication process to reach complete ML2 process_description#489