Improvement: Document Verification Process #320

Document in score repository (until the process_description repo is available)

• migrate verification parts from incubator to s-core process: initial documentation of verification #441
• Create documentation according to "template".
  • this includes work products definition and linking to standard.
  • this includes guidance definition and linking to standard.

Address audit no. 2 findings

Test environment

• An argument shall be provided which covers the ISO 26262 requirement that all test levels are executed in a target near environment.
  • We have a middleware stack. Most things are covered by logical level.
  • The hardware is mostly abstracted by the underlying OS handling specifics of the SoC and architecture x86 and ARM.
  • ARM and x86 can both be simulated. HSI would be a task for the OEM or handed to some part to the integrator.
  • This argument can be put in the verification plan. This comes in justification for safety case and test strategy.
  • Addressed with commit 4ff71d1
• It needs to be clarified where the reference system is located physically (affecting e.g. access rights).
  - Added a section to the verification plan explaining the need for the reference hardware. As it is not yet defined and will not be defined by the time of the next audit, issue Location of reference hardware #608 was created.

Test specification

• The test shall have an attribute which level the test case belongs to.
  - This is defined by the linking to requirements and test type. Coverage may not have a requirement linked.
  - Put this in concept / verification plan
  - Create/Generate a view based on linking.

• It‘s not clear which test belongs to which level.

  • Addressed in d087d84

• The methods of ISO 26262 shall be explained #497

• ❌ The application of the test methods must be recognizable #498

  • A proposal is made in d087d84
  • Follow up in Bugfix: Practical execution of verificaiton activities #1091 (not part of this issue due to additional external dependency outside process community)

• The Coverage target values as listed by table 37 need to described in more detail #499

• Integrate the test specification into the respective tests rather than having them as an independent item.

• It remains unclear how much Verification reports will be created (one per module?) (Report part is with ASchemmel)
  - Current text in workflow describes one Verification report per module and one for a platform configuration (feature selection). A report is valid for a version. This is written in wf__verification__<mod|platform>_ver_report as part of "verification_workflows"

Audit related items here also have to be ticked in the 2nd audit issue: eclipse-score/process_description#2

Clean up

• Implement review findings.
• Prepare presentation in audit session

Optional items for discussion:

• The tool Bazel needs to be evaluated as it may not be able to see the dependencies of changes in requirements, code and tests and may skip some tests. Define the counter measurements.
  • Checked as this is part of tool qualification and not directly related to the verification process.
• The Verification Plan does not define the required review activities (for reuse of OSS existing tests).
  - Handled in commit f4ef54a describing the basics how it was handled in the incubator project with a json lib.
• The independence of the verification team is not yet described in the Verification Plan
  - this is set as a tool requirement in commit https://github.com/eclipse-score/score/pull/441/files#diff-7b5efbb585d7758abd6712db9577217ee1627ae8ac8c9874694a4b1630ff3550