Test Dependency
Note
This feature is subject to change
Buildtest can support test dependencies which allows one to specify condition before running a test. Let’s take a look
at this next example, we have a buildspec with three tests jobA
, jobB
, and jobC
. The test jobA will run immediately
but now we introduce a new keyword needs
which is a list of test names as dependency. We want test jobB to run after jobA is
complete, and jobC to run once jobA and jobB is complete.
buildspecs:
jobA:
type: script
executor: generic.local.bash
description: no job dependency
run: |
echo "This job has no dependency"
sleep 5
jobB:
type: script
executor: generic.local.bash
description: job dependency on jobA
needs: [jobA]
run: |
echo "This job depends on jobA"
sleep 2
jobC:
type: script
executor: generic.local.bash
description: job dependency on jobA and jobB
needs: [jobA, jobB]
run: |
echo "This job depends on jobA and jobB"
sleep 2
The needs
property expects a list of strings, and values must match name of test. If you specify an invalid
test name in needs property then buildtest will ignore the value. If multiple tests are specified in needs property then
all test must finish prior to running test.
Let’s run this test, and take a note that buildtest will run test jobA, followed by jobB then jobC.
buildtest build -b tutorials/job_dependency/ex1.yml
$ buildtest build -b tutorials/job_dependency/ex1.yml
╭───────────────────────────── buildtest summary ──────────────────────────────╮
│ │
│ User: docs │
│ Hostname: build-24413852-project-280831-buildtest │
│ Platform: Linux │
│ Current Time: 2024/05/17 18:01:01 │
│ buildtest path: /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ buildtest version: 2.0 │
│ python path: /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ python version: 3.9.18 │
│ Configuration File: /tmp/tmpqzxycqdh/config.yml │
│ Test Directory: /tmp/tmpqzxycqdh/var/tests │
│ Report File: /tmp/tmpqzxycqdh/var/report.json │
│ Command: /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ │
╰──────────────────────────────────────────────────────────────────────────────╯
─────────────────────────── Discovering Buildspecs ────────────────────────────
Discovered buildspecs
╔══════════════════════════════════════════════════════════════════════════════╗
║ buildspec ║
╟──────────────────────────────────────────────────────────────────────────────╢
║ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/devel/t ║
║ utorials/job_dependency/ex1.yml ║
╟──────────────────────────────────────────────────────────────────────────────╢
║ Total: 1 ║
╚══════════════════════════════════════════════════════════════════════════════╝
Total Discovered Buildspecs: 1
Total Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
────────────────────────────── Parsing Buildspecs ──────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/devel/tutorials/job_dependency/ex1.yml: VALID
Total builder objects created: 3
Builders by type=script
┏━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┓
┃ ┃ ┃ ┃ ┃ ┃ ┃ descript ┃ buildsp ┃
┃ builder ┃ type ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ ion ┃ ecs ┃
┡━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━┩
│ jobA/ea5 │ script │ generic. │ None │ None │ None │ no job │ /home/d │
│ 66444 │ │ local.ba │ │ │ │ dependen │ ocs/che │
│ │ │ sh │ │ │ │ cy │ ckouts/ │
│ │ │ │ │ │ │ │ readthe │
│ │ │ │ │ │ │ │ docs.or │
│ │ │ │ │ │ │ │ g/user_ │
│ │ │ │ │ │ │ │ builds/ │
│ │ │ │ │ │ │ │ buildte │
│ │ │ │ │ │ │ │ st/chec │
│ │ │ │ │ │ │ │ kouts/d │
│ │ │ │ │ │ │ │ evel/tu │
│ │ │ │ │ │ │ │ torials │
│ │ │ │ │ │ │ │ /job_de │
│ │ │ │ │ │ │ │ pendenc │
│ │ │ │ │ │ │ │ y/ex1.y │
│ │ │ │ │ │ │ │ ml │
├──────────┼────────┼──────────┼──────────┼───────┼───────┼──────────┼─────────┤
│ jobB/40b │ script │ generic. │ None │ None │ None │ job │ /home/d │
│ 14674 │ │ local.ba │ │ │ │ dependen │ ocs/che │
│ │ │ sh │ │ │ │ cy on │ ckouts/ │
│ │ │ │ │ │ │ jobA │ readthe │
│ │ │ │ │ │ │ │ docs.or │
│ │ │ │ │ │ │ │ g/user_ │
│ │ │ │ │ │ │ │ builds/ │
│ │ │ │ │ │ │ │ buildte │
│ │ │ │ │ │ │ │ st/chec │
│ │ │ │ │ │ │ │ kouts/d │
│ │ │ │ │ │ │ │ evel/tu │
│ │ │ │ │ │ │ │ torials │
│ │ │ │ │ │ │ │ /job_de │
│ │ │ │ │ │ │ │ pendenc │
│ │ │ │ │ │ │ │ y/ex1.y │
│ │ │ │ │ │ │ │ ml │
├──────────┼────────┼──────────┼──────────┼───────┼───────┼──────────┼─────────┤
│ jobC/75a │ script │ generic. │ None │ None │ None │ job │ /home/d │
│ ce308 │ │ local.ba │ │ │ │ dependen │ ocs/che │
│ │ │ sh │ │ │ │ cy on │ ckouts/ │
│ │ │ │ │ │ │ jobA and │ readthe │
│ │ │ │ │ │ │ jobB │ docs.or │
│ │ │ │ │ │ │ │ g/user_ │
│ │ │ │ │ │ │ │ builds/ │
│ │ │ │ │ │ │ │ buildte │
│ │ │ │ │ │ │ │ st/chec │
│ │ │ │ │ │ │ │ kouts/d │
│ │ │ │ │ │ │ │ evel/tu │
│ │ │ │ │ │ │ │ torials │
│ │ │ │ │ │ │ │ /job_de │
│ │ │ │ │ │ │ │ pendenc │
│ │ │ │ │ │ │ │ y/ex1.y │
│ │ │ │ │ │ │ │ ml │
└──────────┴────────┴──────────┴──────────┴───────┴───────┴──────────┴─────────┘
──────────────────────────────── Building Test ─────────────────────────────────
jobA/ea566444: Creating Test Directory: /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex1/jobA/ea566444
jobB/40b14674: Creating Test Directory: /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex1/jobB/40b14674
jobC/75ace308: Creating Test Directory: /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex1/jobC/75ace308
──────────────────────────────── Running Tests ─────────────────────────────────
Spawning 1 processes for processing builders
───────────────────────────────── Iteration 1 ──────────────────────────────────
jobC/75ace308 Skipping job because it has job dependency on jobA/ea566444 which is in state PENDING
jobB/40b14674 Skipping job because it has job dependency on jobA/ea566444 which is in state PENDING
jobA/ea566444 does not have any dependencies adding test to queue
Builders Eligible to Run
┏━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━┩
│ jobA/ea566444 │
└───────────────┘
jobA/ea566444: Current Working Directory : /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex1/jobA/ea566444/stage
jobA/ea566444: Running Test via command: bash jobA_build.sh
─────────────────────── Output Message for jobA/ea566444 ───────────────────────
This job has no dependency
jobA/ea566444: Test completed in 5.00834 seconds with returncode: 0
jobA/ea566444: Writing output file - /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex1/jobA/ea566444/jobA.out
jobA/ea566444: Writing error file - /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex1/jobA/ea566444/jobA.err
jobA/ea566444: Running Post Run Script: /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex1/jobA/ea566444/stage/jobA_postrun.sh
jobA/ea566444: Post run script exit code: 1
──────────────────── jobA/ea566444: Post Run Script Output ─────────────────────
───────────────────── jobA/ea566444: Post Run Script Error ─────────────────────
/tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex1/jobA/ea566444/stage/jobA_postrun.sh not found.
───────────────────────────────── Iteration 2 ──────────────────────────────────
jobC/75ace308 Skipping job because it has job dependency on jobB/40b14674 which is in state PENDING
Builders Eligible to Run
┏━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━┩
│ jobB/40b14674 │
└───────────────┘
jobB/40b14674: Current Working Directory : /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex1/jobB/40b14674/stage
jobB/40b14674: Running Test via command: bash jobB_build.sh
─────────────────────── Output Message for jobB/40b14674 ───────────────────────
This job depends on jobA
jobB/40b14674: Test completed in 2.007945 seconds with returncode: 0
jobB/40b14674: Writing output file - /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex1/jobB/40b14674/jobB.out
jobB/40b14674: Writing error file - /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex1/jobB/40b14674/jobB.err
jobB/40b14674: Running Post Run Script: /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex1/jobB/40b14674/stage/jobB_postrun.sh
jobB/40b14674: Post run script exit code: 1
──────────────────── jobB/40b14674: Post Run Script Output ─────────────────────
───────────────────── jobB/40b14674: Post Run Script Error ─────────────────────
/tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex1/jobB/40b14674/stage/jobB_postrun.sh not found.
───────────────────────────────── Iteration 3 ──────────────────────────────────
Builders Eligible to Run
┏━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━┩
│ jobC/75ace308 │
└───────────────┘
jobC/75ace308: Current Working Directory : /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex1/jobC/75ace308/stage
jobC/75ace308: Running Test via command: bash jobC_build.sh
─────────────────────── Output Message for jobC/75ace308 ───────────────────────
This job depends on jobA and jobB
jobC/75ace308: Test completed in 2.009665 seconds with returncode: 0
jobC/75ace308: Writing output file - /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex1/jobC/75ace308/jobC.out
jobC/75ace308: Writing error file - /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex1/jobC/75ace308/jobC.err
jobC/75ace308: Running Post Run Script: /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex1/jobC/75ace308/stage/jobC_postrun.sh
jobC/75ace308: Post run script exit code: 1
──────────────────── jobC/75ace308: Post Run Script Output ─────────────────────
───────────────────── jobC/75ace308: Post Run Script Error ─────────────────────
/tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex1/jobC/75ace308/stage/jobC_postrun.sh not found.
Test Summary
┏━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder ┃ executor ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ jobB/40b14674 │ generic.local.bash │ PASS │ 0 │ 2.008 │
├───────────────┼────────────────────┼────────┼────────────┼─────────┤
│ jobC/75ace308 │ generic.local.bash │ PASS │ 0 │ 2.010 │
├───────────────┼────────────────────┼────────┼────────────┼─────────┤
│ jobA/ea566444 │ generic.local.bash │ PASS │ 0 │ 5.008 │
└───────────────┴────────────────────┴────────┴────────────┴─────────┘
Passed Tests: 3/3 Percentage: 100.000%
Failed Tests: 0/3 Percentage: 0.000%
Adding 3 test results to report file: /tmp/tmpqzxycqdh/var/report.json
Writing Logfile to /tmp/tmpqzxycqdh/var/logs/buildtest_bsd6k1mu.log
Test Dependency by returncode
In this next example, we can control behavior of job dependency based on returncode for a given test. This test has three
tests: test1
, test2
and test3
. The first test will exit with returncode 1 but this test will pass because we have
set state: PASS
to override the status check. The next test test2
requires test1 to have a returncode of 1 in order
to satisfy dependency. The returncode
property expects a valid returncode and it can be a list of returncode similar to
how one specify returncode
under the status property see returncode: Return Code Matching. The needs
property can support multiple
test with returncode, in test3
we require test1
to have returncode 1 while test2
has a returncode of 2. We expect test2
to return a returncode of 2 because of exit 2
statement so we expect all three tests to run.
buildspecs:
test1:
type: script
executor: generic.local.bash
description: This test will pass with exit 1
run: exit 1
status:
state: PASS
test2:
type: script
executor: generic.local.bash
description: This test will run if test1 has returncode 1
run: exit 2
status:
state: PASS
needs:
- test1:
returncode: 1
test3:
type: script
executor: generic.local.bash
description: This test will run if test1 has returncode 1 and test2 has returncode 2
run: exit 1
status:
state: PASS
needs:
- test1:
returncode: 1
- test2:
returncode: 2
Let’s build this test and take note of execution order of test.
buildtest build -b tutorials/job_dependency/ex2.yml
$ buildtest build -b tutorials/job_dependency/ex2.yml
╭───────────────────────────── buildtest summary ──────────────────────────────╮
│ │
│ User: docs │
│ Hostname: build-24413852-project-280831-buildtest │
│ Platform: Linux │
│ Current Time: 2024/05/17 18:01:11 │
│ buildtest path: /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ buildtest version: 2.0 │
│ python path: /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ python version: 3.9.18 │
│ Configuration File: /tmp/tmpqzxycqdh/config.yml │
│ Test Directory: /tmp/tmpqzxycqdh/var/tests │
│ Report File: /tmp/tmpqzxycqdh/var/report.json │
│ Command: /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ │
╰──────────────────────────────────────────────────────────────────────────────╯
─────────────────────────── Discovering Buildspecs ────────────────────────────
Discovered buildspecs
╔══════════════════════════════════════════════════════════════════════════════╗
║ buildspec ║
╟──────────────────────────────────────────────────────────────────────────────╢
║ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/devel/t ║
║ utorials/job_dependency/ex2.yml ║
╟──────────────────────────────────────────────────────────────────────────────╢
║ Total: 1 ║
╚══════════════════════════════════════════════════════════════════════════════╝
Total Discovered Buildspecs: 1
Total Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
────────────────────────────── Parsing Buildspecs ──────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/devel/tutorials/job_dependency/ex2.yml: VALID
Total builder objects created: 3
Builders by type=script
┏━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┓
┃ ┃ ┃ ┃ ┃ ┃ ┃ descript ┃ buildsp ┃
┃ builder ┃ type ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ ion ┃ ecs ┃
┡━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━┩
│ test1/dd │ script │ generic. │ None │ None │ None │ This │ /home/d │
│ 8a01df │ │ local.ba │ │ │ │ test │ ocs/che │
│ │ │ sh │ │ │ │ will │ ckouts/ │
│ │ │ │ │ │ │ pass │ readthe │
│ │ │ │ │ │ │ with │ docs.or │
│ │ │ │ │ │ │ exit 1 │ g/user_ │
│ │ │ │ │ │ │ │ builds/ │
│ │ │ │ │ │ │ │ buildte │
│ │ │ │ │ │ │ │ st/chec │
│ │ │ │ │ │ │ │ kouts/d │
│ │ │ │ │ │ │ │ evel/tu │
│ │ │ │ │ │ │ │ torials │
│ │ │ │ │ │ │ │ /job_de │
│ │ │ │ │ │ │ │ pendenc │
│ │ │ │ │ │ │ │ y/ex2.y │
│ │ │ │ │ │ │ │ ml │
├──────────┼────────┼──────────┼──────────┼───────┼───────┼──────────┼─────────┤
│ test2/f5 │ script │ generic. │ None │ None │ None │ This │ /home/d │
│ a7979e │ │ local.ba │ │ │ │ test │ ocs/che │
│ │ │ sh │ │ │ │ will run │ ckouts/ │
│ │ │ │ │ │ │ if test1 │ readthe │
│ │ │ │ │ │ │ has │ docs.or │
│ │ │ │ │ │ │ returnco │ g/user_ │
│ │ │ │ │ │ │ de 1 │ builds/ │
│ │ │ │ │ │ │ │ buildte │
│ │ │ │ │ │ │ │ st/chec │
│ │ │ │ │ │ │ │ kouts/d │
│ │ │ │ │ │ │ │ evel/tu │
│ │ │ │ │ │ │ │ torials │
│ │ │ │ │ │ │ │ /job_de │
│ │ │ │ │ │ │ │ pendenc │
│ │ │ │ │ │ │ │ y/ex2.y │
│ │ │ │ │ │ │ │ ml │
├──────────┼────────┼──────────┼──────────┼───────┼───────┼──────────┼─────────┤
│ test3/d7 │ script │ generic. │ None │ None │ None │ This │ /home/d │
│ e39c7c │ │ local.ba │ │ │ │ test │ ocs/che │
│ │ │ sh │ │ │ │ will run │ ckouts/ │
│ │ │ │ │ │ │ if test1 │ readthe │
│ │ │ │ │ │ │ has │ docs.or │
│ │ │ │ │ │ │ returnco │ g/user_ │
│ │ │ │ │ │ │ de 1 and │ builds/ │
│ │ │ │ │ │ │ test2 │ buildte │
│ │ │ │ │ │ │ has │ st/chec │
│ │ │ │ │ │ │ returnco │ kouts/d │
│ │ │ │ │ │ │ de 2 │ evel/tu │
│ │ │ │ │ │ │ │ torials │
│ │ │ │ │ │ │ │ /job_de │
│ │ │ │ │ │ │ │ pendenc │
│ │ │ │ │ │ │ │ y/ex2.y │
│ │ │ │ │ │ │ │ ml │
└──────────┴────────┴──────────┴──────────┴───────┴───────┴──────────┴─────────┘
──────────────────────────────── Building Test ─────────────────────────────────
test1/dd8a01df: Creating Test Directory: /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex2/test1/dd8a01df
test2/f5a7979e: Creating Test Directory: /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex2/test2/f5a7979e
test3/d7e39c7c: Creating Test Directory: /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex2/test3/d7e39c7c
──────────────────────────────── Running Tests ─────────────────────────────────
Spawning 1 processes for processing builders
───────────────────────────────── Iteration 1 ──────────────────────────────────
test1/dd8a01df does not have any dependencies adding test to queue
test2/f5a7979e Skipping job because it has job dependency on test1/dd8a01df
test3/d7e39c7c Skipping job because it has job dependency on test1/dd8a01df
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━┩
│ test1/dd8a01df │
└────────────────┘
test1/dd8a01df: Current Working Directory : /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex2/test1/dd8a01df/stage
test1/dd8a01df: Running Test via command: bash test1_build.sh
────────────────────── Output Message for test1/dd8a01df ───────────────────────
test1/dd8a01df: failed to submit job with returncode: 1
─────────────────────── Error Message for test1/dd8a01df ───────────────────────
test1/dd8a01df: Detected failure in running test, will attempt to retry test: 1 times
test1/dd8a01df: Run - 1/1
test1/dd8a01df: Running Test via command: bash test1_build.sh
test1/dd8a01df: failed to submit job with returncode: 1
test1/dd8a01df: Test completed in 0.014761 seconds with returncode: 1
test1/dd8a01df: Writing output file - /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex2/test1/dd8a01df/test1.out
test1/dd8a01df: Writing error file - /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex2/test1/dd8a01df/test1.err
test1/dd8a01df: Running Post Run Script: /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex2/test1/dd8a01df/stage/test1_postrun.sh
test1/dd8a01df: Post run script exit code: 1
──────────────────── test1/dd8a01df: Post Run Script Output ────────────────────
──────────────────── test1/dd8a01df: Post Run Script Error ─────────────────────
/tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex2/test1/dd8a01df/stage/test1_postrun.sh not found.
───────────────────────────────── Iteration 2 ──────────────────────────────────
test3/d7e39c7c Skipping job because it has job dependency on test2/f5a7979e
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━┩
│ test2/f5a7979e │
└────────────────┘
test2/f5a7979e: Current Working Directory : /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex2/test2/f5a7979e/stage
test2/f5a7979e: Running Test via command: bash test2_build.sh
────────────────────── Output Message for test2/f5a7979e ───────────────────────
test2/f5a7979e: failed to submit job with returncode: 2
─────────────────────── Error Message for test2/f5a7979e ───────────────────────
test2/f5a7979e: Detected failure in running test, will attempt to retry test: 1 times
test2/f5a7979e: Run - 1/1
test2/f5a7979e: Running Test via command: bash test2_build.sh
test2/f5a7979e: failed to submit job with returncode: 2
test2/f5a7979e: Test completed in 0.014311 seconds with returncode: 2
test2/f5a7979e: Writing output file - /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex2/test2/f5a7979e/test2.out
test2/f5a7979e: Writing error file - /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex2/test2/f5a7979e/test2.err
test2/f5a7979e: Running Post Run Script: /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex2/test2/f5a7979e/stage/test2_postrun.sh
test2/f5a7979e: Post run script exit code: 1
──────────────────── test2/f5a7979e: Post Run Script Output ────────────────────
──────────────────── test2/f5a7979e: Post Run Script Error ─────────────────────
/tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex2/test2/f5a7979e/stage/test2_postrun.sh not found.
───────────────────────────────── Iteration 3 ──────────────────────────────────
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━┩
│ test3/d7e39c7c │
└────────────────┘
test3/d7e39c7c: Current Working Directory : /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex2/test3/d7e39c7c/stage
test3/d7e39c7c: Running Test via command: bash test3_build.sh
────────────────────── Output Message for test3/d7e39c7c ───────────────────────
test3/d7e39c7c: failed to submit job with returncode: 1
─────────────────────── Error Message for test3/d7e39c7c ───────────────────────
test3/d7e39c7c: Detected failure in running test, will attempt to retry test: 1 times
test3/d7e39c7c: Run - 1/1
test3/d7e39c7c: Running Test via command: bash test3_build.sh
test3/d7e39c7c: failed to submit job with returncode: 1
test3/d7e39c7c: Test completed in 0.014088 seconds with returncode: 1
test3/d7e39c7c: Writing output file - /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex2/test3/d7e39c7c/test3.out
test3/d7e39c7c: Writing error file - /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex2/test3/d7e39c7c/test3.err
test3/d7e39c7c: Running Post Run Script: /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex2/test3/d7e39c7c/stage/test3_postrun.sh
test3/d7e39c7c: Post run script exit code: 1
──────────────────── test3/d7e39c7c: Post Run Script Output ────────────────────
──────────────────── test3/d7e39c7c: Post Run Script Error ─────────────────────
/tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex2/test3/d7e39c7c/stage/test3_postrun.sh not found.
Test Summary
┏━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder ┃ executor ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ test3/d7e39c7c │ generic.local.bash │ PASS │ 1 │ 0.014 │
├────────────────┼────────────────────┼────────┼────────────┼─────────┤
│ test2/f5a7979e │ generic.local.bash │ PASS │ 2 │ 0.014 │
├────────────────┼────────────────────┼────────┼────────────┼─────────┤
│ test1/dd8a01df │ generic.local.bash │ PASS │ 1 │ 0.015 │
└────────────────┴────────────────────┴────────┴────────────┴─────────┘
Passed Tests: 3/3 Percentage: 100.000%
Failed Tests: 0/3 Percentage: 0.000%
Adding 3 test results to report file: /tmp/tmpqzxycqdh/var/report.json
Writing Logfile to /tmp/tmpqzxycqdh/var/logs/buildtest_hpm527wn.log
Test Dependency by state
You can specify state
as a property to check for test state when specify test dependency. In this next example, we have
have four tests pass_test, fail_test, pass_and_fail_test, and final_test. The first test will be a PASS because
we have state: PASS
. The test fail_test
depends on pass_test only if it has state: PASS
, if value is mismatch then test
will be skipped. Note that buildtest will skip test until next iteration if test is not executed, however if test is complete then buildtest
will cancel dependent test. We can specify multiple test dependencies with state property such as test pass_and_fail_test which expects
pass_test
to have state: PASS and fail_test
to have state: FAIL. In test final_test
, shows how you can combine the format, the
needs
property is a list of object where each element is name of test. If no properties are associated with test name then buildtest will
wait until job is complete to execute test. In this example, the test expects both pass_test
and fail_test
to run while pass_and_fail_test
must have returncode of 1.
buildspecs:
pass_test:
type: script
executor: generic.local.bash
description: This test will always pass
status:
state: PASS
run: |
echo "This test will pass"
sleep 2
fail_test:
type: script
executor: generic.local.bash
description: This test will run if test 'pass_test' is in state 'PASS'
status:
state: FAIL
needs:
- pass_test:
state: PASS
run: |
echo "This test will fail"
sleep 2
pass_and_fail_test:
type: script
executor: generic.local.bash
description: This test will run if pass_test is 'PASS' and fail_test is 'FAIL'
needs:
- pass_test:
state: PASS
- fail_test:
state: FAIL
run: |
sleep 2
exit 1
status:
returncode: 1
final_test:
type: script
executor: generic.local.bash
description: Test will run after 'pass_test', 'fail_test', and 'pass_and_fail_test'
needs:
- pass_test
- fail_test
- pass_and_fail_test:
returncode: 1
run: |
sleep 2
echo "Hello world"
Let’s build this test and take note all tests are run.
buildtest build -b tutorials/job_dependency/ex3.yml
$ buildtest build -b tutorials/job_dependency/ex3.yml
╭───────────────────────────── buildtest summary ──────────────────────────────╮
│ │
│ User: docs │
│ Hostname: build-24413852-project-280831-buildtest │
│ Platform: Linux │
│ Current Time: 2024/05/17 18:01:11 │
│ buildtest path: /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ buildtest version: 2.0 │
│ python path: /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ python version: 3.9.18 │
│ Configuration File: /tmp/tmpqzxycqdh/config.yml │
│ Test Directory: /tmp/tmpqzxycqdh/var/tests │
│ Report File: /tmp/tmpqzxycqdh/var/report.json │
│ Command: /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ │
╰──────────────────────────────────────────────────────────────────────────────╯
─────────────────────────── Discovering Buildspecs ────────────────────────────
Discovered buildspecs
╔══════════════════════════════════════════════════════════════════════════════╗
║ buildspec ║
╟──────────────────────────────────────────────────────────────────────────────╢
║ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/devel/t ║
║ utorials/job_dependency/ex3.yml ║
╟──────────────────────────────────────────────────────────────────────────────╢
║ Total: 1 ║
╚══════════════════════════════════════════════════════════════════════════════╝
Total Discovered Buildspecs: 1
Total Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
────────────────────────────── Parsing Buildspecs ──────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/devel/tutorials/job_dependency/ex3.yml: VALID
Total builder objects created: 4
Builders by type=script
┏━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┓
┃ ┃ ┃ ┃ ┃ ┃ ┃ descript ┃ buildsp ┃
┃ builder ┃ type ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ ion ┃ ecs ┃
┡━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━┩
│ pass_tes │ script │ generic. │ None │ None │ None │ This │ /home/d │
│ t/8f7041 │ │ local.ba │ │ │ │ test │ ocs/che │
│ 9e │ │ sh │ │ │ │ will │ ckouts/ │
│ │ │ │ │ │ │ always │ readthe │
│ │ │ │ │ │ │ pass │ docs.or │
│ │ │ │ │ │ │ │ g/user_ │
│ │ │ │ │ │ │ │ builds/ │
│ │ │ │ │ │ │ │ buildte │
│ │ │ │ │ │ │ │ st/chec │
│ │ │ │ │ │ │ │ kouts/d │
│ │ │ │ │ │ │ │ evel/tu │
│ │ │ │ │ │ │ │ torials │
│ │ │ │ │ │ │ │ /job_de │
│ │ │ │ │ │ │ │ pendenc │
│ │ │ │ │ │ │ │ y/ex3.y │
│ │ │ │ │ │ │ │ ml │
├──────────┼────────┼──────────┼──────────┼───────┼───────┼──────────┼─────────┤
│ fail_tes │ script │ generic. │ None │ None │ None │ This │ /home/d │
│ t/b7f7e8 │ │ local.ba │ │ │ │ test │ ocs/che │
│ 89 │ │ sh │ │ │ │ will run │ ckouts/ │
│ │ │ │ │ │ │ if test │ readthe │
│ │ │ │ │ │ │ 'pass_te │ docs.or │
│ │ │ │ │ │ │ st' is │ g/user_ │
│ │ │ │ │ │ │ in state │ builds/ │
│ │ │ │ │ │ │ 'PASS' │ buildte │
│ │ │ │ │ │ │ │ st/chec │
│ │ │ │ │ │ │ │ kouts/d │
│ │ │ │ │ │ │ │ evel/tu │
│ │ │ │ │ │ │ │ torials │
│ │ │ │ │ │ │ │ /job_de │
│ │ │ │ │ │ │ │ pendenc │
│ │ │ │ │ │ │ │ y/ex3.y │
│ │ │ │ │ │ │ │ ml │
├──────────┼────────┼──────────┼──────────┼───────┼───────┼──────────┼─────────┤
│ pass_and │ script │ generic. │ None │ None │ None │ This │ /home/d │
│ _fail_te │ │ local.ba │ │ │ │ test │ ocs/che │
│ st/8ea2e │ │ sh │ │ │ │ will run │ ckouts/ │
│ 2c7 │ │ │ │ │ │ if │ readthe │
│ │ │ │ │ │ │ pass_tes │ docs.or │
│ │ │ │ │ │ │ t is │ g/user_ │
│ │ │ │ │ │ │ 'PASS' │ builds/ │
│ │ │ │ │ │ │ and │ buildte │
│ │ │ │ │ │ │ fail_tes │ st/chec │
│ │ │ │ │ │ │ t is │ kouts/d │
│ │ │ │ │ │ │ 'FAIL' │ evel/tu │
│ │ │ │ │ │ │ │ torials │
│ │ │ │ │ │ │ │ /job_de │
│ │ │ │ │ │ │ │ pendenc │
│ │ │ │ │ │ │ │ y/ex3.y │
│ │ │ │ │ │ │ │ ml │
├──────────┼────────┼──────────┼──────────┼───────┼───────┼──────────┼─────────┤
│ final_te │ script │ generic. │ None │ None │ None │ Test │ /home/d │
│ st/0d836 │ │ local.ba │ │ │ │ will run │ ocs/che │
│ 9dc │ │ sh │ │ │ │ after │ ckouts/ │
│ │ │ │ │ │ │ 'pass_te │ readthe │
│ │ │ │ │ │ │ st', │ docs.or │
│ │ │ │ │ │ │ 'fail_te │ g/user_ │
│ │ │ │ │ │ │ st', and │ builds/ │
│ │ │ │ │ │ │ 'pass_an │ buildte │
│ │ │ │ │ │ │ d_fail_t │ st/chec │
│ │ │ │ │ │ │ est' │ kouts/d │
│ │ │ │ │ │ │ │ evel/tu │
│ │ │ │ │ │ │ │ torials │
│ │ │ │ │ │ │ │ /job_de │
│ │ │ │ │ │ │ │ pendenc │
│ │ │ │ │ │ │ │ y/ex3.y │
│ │ │ │ │ │ │ │ ml │
└──────────┴────────┴──────────┴──────────┴───────┴───────┴──────────┴─────────┘
──────────────────────────────── Building Test ─────────────────────────────────
pass_test/8f70419e: Creating Test Directory: /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex3/pass_test/8f70419e
fail_test/b7f7e889: Creating Test Directory: /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex3/fail_test/b7f7e889
pass_and_fail_test/8ea2e2c7: Creating Test Directory: /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex3/pass_and_fail_test/8ea2e2c7
final_test/0d8369dc: Creating Test Directory: /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex3/final_test/0d8369dc
──────────────────────────────── Running Tests ─────────────────────────────────
Spawning 1 processes for processing builders
───────────────────────────────── Iteration 1 ──────────────────────────────────
pass_and_fail_test/8ea2e2c7 Skipping job because it has job dependency on pass_test/8f70419e
pass_test/8f70419e does not have any dependencies adding test to queue
final_test/0d8369dc Skipping job because it has job dependency on pass_test/8f70419e which is in state PENDING
fail_test/b7f7e889 Skipping job because it has job dependency on pass_test/8f70419e
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━━━━━┩
│ pass_test/8f70419e │
└────────────────────┘
pass_test/8f70419e: Current Working Directory : /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex3/pass_test/8f70419e/stage
pass_test/8f70419e: Running Test via command: bash pass_test_build.sh
──────────────────── Output Message for pass_test/8f70419e ─────────────────────
This test will pass
pass_test/8f70419e: Test completed in 2.008191 seconds with returncode: 0
pass_test/8f70419e: Writing output file - /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex3/pass_test/8f70419e/pass_test.out
pass_test/8f70419e: Writing error file - /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex3/pass_test/8f70419e/pass_test.err
pass_test/8f70419e: Running Post Run Script: /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex3/pass_test/8f70419e/stage/pass_test_postrun.sh
pass_test/8f70419e: Post run script exit code: 1
────────────────── pass_test/8f70419e: Post Run Script Output ──────────────────
────────────────── pass_test/8f70419e: Post Run Script Error ───────────────────
/tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex3/pass_test/8f70419e/stage/pass_test_postrun.sh not found.
───────────────────────────────── Iteration 2 ──────────────────────────────────
pass_and_fail_test/8ea2e2c7 Skipping job because it has job dependency on fail_test/b7f7e889
final_test/0d8369dc Skipping job because it has job dependency on fail_test/b7f7e889 which is in state PENDING
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━━━━━┩
│ fail_test/b7f7e889 │
└────────────────────┘
fail_test/b7f7e889: Current Working Directory : /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex3/fail_test/b7f7e889/stage
fail_test/b7f7e889: Running Test via command: bash fail_test_build.sh
──────────────────── Output Message for fail_test/b7f7e889 ─────────────────────
This test will fail
fail_test/b7f7e889: Test completed in 2.007914 seconds with returncode: 0
fail_test/b7f7e889: Writing output file - /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex3/fail_test/b7f7e889/fail_test.out
fail_test/b7f7e889: Writing error file - /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex3/fail_test/b7f7e889/fail_test.err
fail_test/b7f7e889: Running Post Run Script: /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex3/fail_test/b7f7e889/stage/fail_test_postrun.sh
fail_test/b7f7e889: Post run script exit code: 1
────────────────── fail_test/b7f7e889: Post Run Script Output ──────────────────
────────────────── fail_test/b7f7e889: Post Run Script Error ───────────────────
/tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex3/fail_test/b7f7e889/stage/fail_test_postrun.sh not found.
───────────────────────────────── Iteration 3 ──────────────────────────────────
final_test/0d8369dc Skipping job because it has job dependency on pass_and_fail_test/8ea2e2c7
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ pass_and_fail_test/8ea2e2c7 │
└─────────────────────────────┘
pass_and_fail_test/8ea2e2c7: Current Working Directory : /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex3/pass_and_fail_test/8ea2e2c7/stage
pass_and_fail_test/8ea2e2c7: Running Test via command: bash pass_and_fail_test_build.sh
──────────────── Output Message for pass_and_fail_test/8ea2e2c7 ────────────────
pass_and_fail_test/8ea2e2c7: failed to submit job with returncode: 1
──────────────── Error Message for pass_and_fail_test/8ea2e2c7 ─────────────────
pass_and_fail_test/8ea2e2c7: Detected failure in running test, will attempt to retry test: 1 times
pass_and_fail_test/8ea2e2c7: Run - 1/1
pass_and_fail_test/8ea2e2c7: Running Test via command: bash pass_and_fail_test_build.sh
pass_and_fail_test/8ea2e2c7: failed to submit job with returncode: 1
pass_and_fail_test/8ea2e2c7: Test completed in 4.016805 seconds with returncode: 1
pass_and_fail_test/8ea2e2c7: Writing output file - /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex3/pass_and_fail_test/8ea2e2c7/pass_and_fail_test.out
pass_and_fail_test/8ea2e2c7: Writing error file - /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex3/pass_and_fail_test/8ea2e2c7/pass_and_fail_test.err
pass_and_fail_test/8ea2e2c7: Checking returncode - 1 is matched in list [1]
pass_and_fail_test/8ea2e2c7: Running Post Run Script: /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex3/pass_and_fail_test/8ea2e2c7/stage/pass_and_fail_test_postrun.sh
pass_and_fail_test/8ea2e2c7: Post run script exit code: 1
───────────── pass_and_fail_test/8ea2e2c7: Post Run Script Output ──────────────
────────────── pass_and_fail_test/8ea2e2c7: Post Run Script Error ──────────────
/tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex3/pass_and_fail_test/8ea2e2c7/stage/pass_and_fail_test_postrun.sh not found.
───────────────────────────────── Iteration 4 ──────────────────────────────────
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━━━━━━┩
│ final_test/0d8369dc │
└─────────────────────┘
final_test/0d8369dc: Current Working Directory : /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex3/final_test/0d8369dc/stage
final_test/0d8369dc: Running Test via command: bash final_test_build.sh
──────────────────── Output Message for final_test/0d8369dc ────────────────────
Hello world
final_test/0d8369dc: Test completed in 2.007748 seconds with returncode: 0
final_test/0d8369dc: Writing output file - /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex3/final_test/0d8369dc/final_test.out
final_test/0d8369dc: Writing error file - /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex3/final_test/0d8369dc/final_test.err
final_test/0d8369dc: Running Post Run Script: /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex3/final_test/0d8369dc/stage/final_test_postrun.sh
final_test/0d8369dc: Post run script exit code: 1
───────────────── final_test/0d8369dc: Post Run Script Output ──────────────────
────────────────── final_test/0d8369dc: Post Run Script Error ──────────────────
/tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex3/final_test/0d8369dc/stage/final_test_postrun.sh not found.
Test Summary
┏━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder ┃ executor ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ pass_test/8f70419e │ generic.local.bash │ PASS │ 0 │ 2.008 │
├─────────────────────────┼────────────────────┼────────┼────────────┼─────────┤
│ pass_and_fail_test/8ea2 │ generic.local.bash │ PASS │ 1 │ 4.017 │
│ e2c7 │ │ │ │ │
├─────────────────────────┼────────────────────┼────────┼────────────┼─────────┤
│ fail_test/b7f7e889 │ generic.local.bash │ FAIL │ 0 │ 2.008 │
├─────────────────────────┼────────────────────┼────────┼────────────┼─────────┤
│ final_test/0d8369dc │ generic.local.bash │ PASS │ 0 │ 2.008 │
└─────────────────────────┴────────────────────┴────────┴────────────┴─────────┘
Passed Tests: 3/4 Percentage: 75.000%
Failed Tests: 1/4 Percentage: 25.000%
Adding 4 test results to report file: /tmp/tmpqzxycqdh/var/report.json
Writing Logfile to /tmp/tmpqzxycqdh/var/logs/buildtest_cfrc_yhp.log
In this next example, we have three tests the first test will runtime_test
will sleep for 5 seconds but it will fail due to runtime
requirement of 2sec. The next two tests runtime_test_pass
and runtime_test_fail
both depend on runtime_test
however due to condition
only one of them can be run because runtime_test_pass
expects runtime_test to have state: PASS while runtime_test_fail
expects runtime_test
to have state: FAIL. This type of workflow can be used if you want to run a set of test based on one condition while running a different set of
test based on the negative condition.
buildspecs:
runtime_test:
type: script
executor: generic.local.bash
description: This test will sleep 5 second but will fail due to runtime 2sec
status:
runtime:
min: 2.0
run: sleep 5
runtime_test_pass:
type: script
executor: generic.local.bash
description: This test will run when runtime_test_pass is PASS
needs:
- runtime_test:
state: PASS
run: echo "Performing some action when test PASS"
runtime_test_fail:
type: script
executor: generic.local.bash
description: This test will run when runtime_test_pass is FAIL
needs:
- runtime_test:
state: FAIL
run: echo "Performing some action when test FAIL"
Let’s build this test and take note that we only run two tests and runtime_test_fail was skipped because test runtime_test has a
state: PASS
.
buildtest build -b tutorials/job_dependency/ex4.yml
$ buildtest build -b tutorials/job_dependency/ex4.yml
╭───────────────────────────── buildtest summary ──────────────────────────────╮
│ │
│ User: docs │
│ Hostname: build-24413852-project-280831-buildtest │
│ Platform: Linux │
│ Current Time: 2024/05/17 18:01:22 │
│ buildtest path: /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ buildtest version: 2.0 │
│ python path: /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ python version: 3.9.18 │
│ Configuration File: /tmp/tmpqzxycqdh/config.yml │
│ Test Directory: /tmp/tmpqzxycqdh/var/tests │
│ Report File: /tmp/tmpqzxycqdh/var/report.json │
│ Command: /home/docs/checkouts/readthedocs.org/user_builds/buildte │
│ │
╰──────────────────────────────────────────────────────────────────────────────╯
─────────────────────────── Discovering Buildspecs ────────────────────────────
Discovered buildspecs
╔══════════════════════════════════════════════════════════════════════════════╗
║ buildspec ║
╟──────────────────────────────────────────────────────────────────────────────╢
║ /home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/devel/t ║
║ utorials/job_dependency/ex4.yml ║
╟──────────────────────────────────────────────────────────────────────────────╢
║ Total: 1 ║
╚══════════════════════════════════════════════════════════════════════════════╝
Total Discovered Buildspecs: 1
Total Excluded Buildspecs: 0
Detected Buildspecs after exclusion: 1
────────────────────────────── Parsing Buildspecs ──────────────────────────────
Valid Buildspecs: 1
Invalid Buildspecs: 0
/home/docs/checkouts/readthedocs.org/user_builds/buildtest/checkouts/devel/tutorials/job_dependency/ex4.yml: VALID
Total builder objects created: 3
Builders by type=script
┏━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┓
┃ ┃ ┃ ┃ ┃ ┃ ┃ descript ┃ buildsp ┃
┃ builder ┃ type ┃ executor ┃ compiler ┃ nodes ┃ procs ┃ ion ┃ ecs ┃
┡━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━┩
│ runtime_ │ script │ generic. │ None │ None │ None │ This │ /home/d │
│ test/048 │ │ local.ba │ │ │ │ test │ ocs/che │
│ 65c2e │ │ sh │ │ │ │ will │ ckouts/ │
│ │ │ │ │ │ │ sleep 5 │ readthe │
│ │ │ │ │ │ │ second │ docs.or │
│ │ │ │ │ │ │ but will │ g/user_ │
│ │ │ │ │ │ │ fail due │ builds/ │
│ │ │ │ │ │ │ to │ buildte │
│ │ │ │ │ │ │ runtime │ st/chec │
│ │ │ │ │ │ │ 2sec │ kouts/d │
│ │ │ │ │ │ │ │ evel/tu │
│ │ │ │ │ │ │ │ torials │
│ │ │ │ │ │ │ │ /job_de │
│ │ │ │ │ │ │ │ pendenc │
│ │ │ │ │ │ │ │ y/ex4.y │
│ │ │ │ │ │ │ │ ml │
├──────────┼────────┼──────────┼──────────┼───────┼───────┼──────────┼─────────┤
│ runtime_ │ script │ generic. │ None │ None │ None │ This │ /home/d │
│ test_pas │ │ local.ba │ │ │ │ test │ ocs/che │
│ s/39419a │ │ sh │ │ │ │ will run │ ckouts/ │
│ f9 │ │ │ │ │ │ when │ readthe │
│ │ │ │ │ │ │ runtime_ │ docs.or │
│ │ │ │ │ │ │ test_pas │ g/user_ │
│ │ │ │ │ │ │ s is │ builds/ │
│ │ │ │ │ │ │ PASS │ buildte │
│ │ │ │ │ │ │ │ st/chec │
│ │ │ │ │ │ │ │ kouts/d │
│ │ │ │ │ │ │ │ evel/tu │
│ │ │ │ │ │ │ │ torials │
│ │ │ │ │ │ │ │ /job_de │
│ │ │ │ │ │ │ │ pendenc │
│ │ │ │ │ │ │ │ y/ex4.y │
│ │ │ │ │ │ │ │ ml │
├──────────┼────────┼──────────┼──────────┼───────┼───────┼──────────┼─────────┤
│ runtime_ │ script │ generic. │ None │ None │ None │ This │ /home/d │
│ test_fai │ │ local.ba │ │ │ │ test │ ocs/che │
│ l/ce5147 │ │ sh │ │ │ │ will run │ ckouts/ │
│ cc │ │ │ │ │ │ when │ readthe │
│ │ │ │ │ │ │ runtime_ │ docs.or │
│ │ │ │ │ │ │ test_pas │ g/user_ │
│ │ │ │ │ │ │ s is │ builds/ │
│ │ │ │ │ │ │ FAIL │ buildte │
│ │ │ │ │ │ │ │ st/chec │
│ │ │ │ │ │ │ │ kouts/d │
│ │ │ │ │ │ │ │ evel/tu │
│ │ │ │ │ │ │ │ torials │
│ │ │ │ │ │ │ │ /job_de │
│ │ │ │ │ │ │ │ pendenc │
│ │ │ │ │ │ │ │ y/ex4.y │
│ │ │ │ │ │ │ │ ml │
└──────────┴────────┴──────────┴──────────┴───────┴───────┴──────────┴─────────┘
──────────────────────────────── Building Test ─────────────────────────────────
runtime_test/04865c2e: Creating Test Directory: /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex4/runtime_test/04865c2e
runtime_test_pass/39419af9: Creating Test Directory: /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex4/runtime_test_pass/39419af9
runtime_test_fail/ce5147cc: Creating Test Directory: /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex4/runtime_test_fail/ce5147cc
──────────────────────────────── Running Tests ─────────────────────────────────
Spawning 1 processes for processing builders
───────────────────────────────── Iteration 1 ──────────────────────────────────
runtime_test_pass/39419af9 Skipping job because it has job dependency on runtime_test/04865c2e
runtime_test/04865c2e does not have any dependencies adding test to queue
runtime_test_fail/ce5147cc Skipping job because it has job dependency on runtime_test/04865c2e
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━━━━━━━━┩
│ runtime_test/04865c2e │
└───────────────────────┘
runtime_test/04865c2e: Current Working Directory : /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex4/runtime_test/04865c2e/stage
runtime_test/04865c2e: Running Test via command: bash runtime_test_build.sh
─────────────────── Output Message for runtime_test/04865c2e ───────────────────
runtime_test/04865c2e: Test completed in 5.008065 seconds with returncode: 0
runtime_test/04865c2e: Writing output file - /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex4/runtime_test/04865c2e/runtime_test.out
runtime_test/04865c2e: Writing error file - /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex4/runtime_test/04865c2e/runtime_test.err
runtime_test/04865c2e: Checking mintime < runtime: 2.0 < 5.008065
runtime_test/04865c2e: Running Post Run Script: /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex4/runtime_test/04865c2e/stage/runtime_test_postrun.sh
runtime_test/04865c2e: Post run script exit code: 1
──────────────── runtime_test/04865c2e: Post Run Script Output ─────────────────
───────────────── runtime_test/04865c2e: Post Run Script Error ─────────────────
/tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex4/runtime_test/04865c2e/stage/runtime_test_postrun.sh not found.
───────────────────────────────── Iteration 2 ──────────────────────────────────
runtime_test_fail/ce5147cc is cancelled because it depends on runtime_test/04865c2e to have state: FAIL but actual value is PASS
Builders Eligible to Run
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Builder ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ runtime_test_pass/39419af9 │
└────────────────────────────┘
runtime_test_pass/39419af9: Current Working Directory : /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex4/runtime_test_pass/39419af9/stage
runtime_test_pass/39419af9: Running Test via command: bash runtime_test_pass_build.sh
──────────────── Output Message for runtime_test_pass/39419af9 ─────────────────
Performing some action when test PASS
runtime_test_pass/39419af9: Test completed in 0.007058 seconds with returncode: 0
runtime_test_pass/39419af9: Writing output file - /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex4/runtime_test_pass/39419af9/runtime_test_pass.out
runtime_test_pass/39419af9: Writing error file - /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex4/runtime_test_pass/39419af9/runtime_test_pass.err
runtime_test_pass/39419af9: Running Post Run Script: /tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex4/runtime_test_pass/39419af9/stage/runtime_test_pass_postrun.sh
runtime_test_pass/39419af9: Post run script exit code: 1
────────────── runtime_test_pass/39419af9: Post Run Script Output ──────────────
────────────── runtime_test_pass/39419af9: Post Run Script Error ───────────────
/tmp/tmpqzxycqdh/var/tests/generic.local.bash/ex4/runtime_test_pass/39419af9/stage/runtime_test_pass_postrun.sh not found.
Test Summary
┏━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┓
┃ builder ┃ executor ┃ status ┃ returncode ┃ runtime ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━┩
│ runtime_test_pass/39419 │ generic.local.bash │ PASS │ 0 │ 0.007 │
│ af9 │ │ │ │ │
├─────────────────────────┼────────────────────┼────────┼────────────┼─────────┤
│ runtime_test/04865c2e │ generic.local.bash │ PASS │ 0 │ 5.008 │
└─────────────────────────┴────────────────────┴────────┴────────────┴─────────┘
Passed Tests: 2/2 Percentage: 100.000%
Failed Tests: 0/2 Percentage: 0.000%
Adding 2 test results to report file: /tmp/tmpqzxycqdh/var/report.json
Writing Logfile to /tmp/tmpqzxycqdh/var/logs/buildtest_j_7v4t1m.log