- Summary
- add test dependencies; Owner: Tim; Reporter: Tim, AGL, Seimens; Status: in progress; Priority: very high
|
; Summary: add test dependencies; Owner: Tim; Reporter: Tim, AGL, Seimens; Status: in progress; Priority: very high
|
|
|
= Description =Dependencies are important to avoid running tests that have problems.
|
|
This feature allows the test creator to specify a set of dependencies thatare checked before a test is invoked. It is intended that this be usedfor high-level sanity-checks - that is, to specify resources or configurationsthat, if missing, invalidate the entire test. If these are missing itmakes it impossible to run the test.
|
This feature allows the test creator to specify a set of dependencies thatare checked before a test is invoked. It is intended that this be usedfor high-level sanity-checks - that is, to specify resources or configurationsthat, if missing, invalidate the entire test. If these are missing itmakes it impossible to run the test.
|
|
Some tests check for features themselves, and abort the test or skip sub-testsif the features cannot be found. Individual tests use their own mechanismsfor testing for these types of conditions.
|
Some tests check for features themselves, and abort the test or skip sub-testsif the features cannot be found. Individual tests use their own mechanismsfor testing for these types of conditions.
|
|
These are checks intended to be used by the base script to validate whethera particular test even applies to a certain target (or target configuration).
|
These are checks intended to be used by the base script to validate whethera particular test even applies to a certain target (or target configuration).
|
|
The dependency checks are processed during the pre_test phase, and if the dependencies are not met, the test is aborted with a failure.
|
The dependency checks are processed during the pre_test phase, and if the dependencies are not met, the test is aborted with a failure.
|
check_capability- check for the presence of a CAP_FEATURE environment variable * this is used to check free-form variables (defined in the board or distro files) * assert_define
- check for the presence of an environment variable * this is used to check for test_spec variables * is_on_target - check for the presence of a file on the target system
|
== Existing support ==Fuego already supports the following mechanisms for detecting dependencies: * [[function_check_capability|check_capability]] - check for the presence of a CAP_''FEATURE'' environment variable * this is used to check free-form variables (defined in the board or distro files) * [[function_assert_define|assert_define]] - check for the presence of an environment variable * this is used to check for test_spec variables * [[function_is_on_target|is_on_target]] - check for the presence of a file on the target system
|
|
|
== Types of dependencies ==Here are different types of dependencies that may be present: * dependency on a specific hardware feature (such as touchpad, video, audio) * dependency on a system capability (such as networking, file system, user accounts, etc.) * dependency on a particular kernel configuration * these are often used as proxies for system capabilities or hardware * dependency on a file or executable on the target
|
|
|
== Outcomes ==Should dependency failures result in test failures, or in some otherfailure result - like "test skipped"? Is this a particular type of failurethat should be reported differently from a regular test failure?
|
|
|
== Caching dependency information ==It may save time to be able to cache the dependency information about a particular target board, if detecting the information is time-consuming.For example, the
|
|
check_capability
|
== Industry survey ==How to other systems allow for checking for test dependencies? * what does 0-day do for test dependencies? * uses 'need_xxx' directives (which are declarative) * the following are supported: * need_kconfig * examples: * need_kconfig: CONFIG_ * need_x * need_kernel_headers * example: need_kernel_headers: true * need_memory * example: need_memory 2G * need_cpu * number of cpus, not used * what does fuego do? (imperative) * [[function_check_capability|check_capability]] * [[function_assert_define|assert_define]] * [[function_is_on_target|is_on_target]] * what does kernelci * what does LTP do? *
|
|
|
= proposal = * declare dependencies as environment variables: * need_memory=2G * need_kconfig=CONFIG_MD_RAID ... * space-separated * need_env_vars= * need_kernel_headers=1
|
|
|
= Notes =This was originally suggested in July of 2016 (probably in discussions atLinuxCon Japan).
|
- backlink
Fuego Issues List, Fuego To Do List
|
; backlink: [[Fuego Issues List]], [[Fuego To Do List]]
|