Existing test chains
Currently some test chains have been implemented testing the whole offline chain starting with the HLT1 step in Moore up to the production of the final user-ntuples generated with FunTuple in DaVinci. Each test is repeated both on a data and a simulated sample to assert the correct behaviour in both cases.
You can issue the following command to run a whole test chain (e.g. Tupling_default
) and generate the final root
files:
run make test ARGS="-R LHCbIntegrationTests.Tupling_default"
Or, if you are working on an LHCb stack:
make fast/LHCbIntegrationTests/test ARGS="-R Tupling_default"
Moore+DaVinci chain
Simple test checking that both Moore and DaVinci application can be run correctly. Similar chains can be created for other LHCb applications if they are included in any test of this package.
Validation of Moore-DaVinci chain with default configuration
Project to validate the Moore-DaVinci-Tupling
workflow. It consists of different tests,
each one dedicated to test a specific step of the chain.
All the test commands are reported in the Tupling_default/CMakeLists.txt
file.
The test chain starts with a MC .digi file available in the _TestFileDB_ and creates a root
file
containing a tuple produced with _FunTuple_.
The final tuple is created reading both a passthrough and a Sprucing dst
file.
Each step of the chain is run using the Utilitites/logscript.sh
script that saves the corresponding _stdout_ and _stderr_.
The _stderr_ is then used for step validation by means of the Utilities/validator.sh
script ensuring no error is
dumped in the _stderr_.
For the two final tuples an additional validation is performed using the Utilities/validator_tuple.py
script checking that
the _DecayTree_ is not empty and contains exactly all the expected branches.
The test chain consists of the following steps:
- Start the test chain running HLT1 Allen on a MC .digi file from the TestFileDB.
Function: [
Moore.production:hlt1
](https://gitlab.cern.ch/lhcb/Moore/-/blob/master/Hlt/Moore/python/Moore/production.py)Option file: [
Tupling_default/option_moore_hlt1_allen.yaml
](https://gitlab.cern.ch/lhcb/LHCbIntegrationTests/-/blob/master/Tupling_default/option_moore_hlt1_allen.yaml)
- Run the HLT2 on the HLT1 step output, testing the MC workflow.
Function: [
Moore.production:hlt2
](https://gitlab.cern.ch/lhcb/Moore/-/blob/master/Hlt/Moore/python/Moore/production.py)Option file: [
Tupling_default/option_moore_hlt2_all_lines.yaml
](https://gitlab.cern.ch/lhcb/LHCbIntegrationTests/-/blob/master/Tupling_default/option_moore_hlt2_all_lines.yaml)
- Run offline processing on the HLT2 step output with:
- a Passthrough line
Function: [
Hlt2Conf.Sprucing_tests:spruce_passthrough
](https://gitlab.cern.ch/lhcb/Moore/-/blob/master/Hlt/Hlt2Conf/python/Hlt2Conf/Sprucing_tests.py)Option file: [
Tupling_default/option_moore_passthrough.yaml
](https://gitlab.cern.ch/lhcb/LHCbIntegrationTests/-/blob/master/Tupling_default/option_moore_passthrough.yaml)
- a Sprucing line
Function: [
Hlt2Conf.Sprucing_tests:spruce_all_lines_realtime
](https://gitlab.cern.ch/lhcb/Moore/-/blob/master/Hlt/Hlt2Conf/python/Hlt2Conf/Sprucing_tests.py)Option file: [
Tupling_default/option_moore_spruce_all_lines.yaml
](https://gitlab.cern.ch/lhcb/LHCbIntegrationTests/-/blob/master/Tupling_default/option_moore_spruce.yaml)
- Run DaVinci and create a tuple from:
- the HLT2 step output
Function: [
DaVinciTests.options_lhcbintegration_mc_excl_line:main
](https://gitlab.cern.ch/lhcb/DaVinci/-/blob/master/DaVinciTests/python/DaVinciTests/options_lhcbintegration_mc_excl_line.py)Option file: [
Tupling_default/option_davinci_tupling_from_turbo.yaml
](https://gitlab.cern.ch/lhcb/LHCbIntegrationTests/-/blob/master/Tupling_default/option_davinci_tupling_from_turbo.yaml)
- the Spruce step output
Function: [
DaVinciTests.options_lhcbintegration_mc_excl_line:main
](https://gitlab.cern.ch/lhcb/DaVinci/-/blob/master/DaVinciTests/python/DaVinciTests/options_lhcbintegration_mc_excl_line.py)Option file: [
Tupling_default/option_davinci_tupling_from_spruce.yaml
](https://gitlab.cern.ch/lhcb/LHCbIntegrationTests/-/blob/master/Tupling_default/option_davinci_tupling_from_spruce.yaml)
Validation of the Moore-DaVinci chain with veloSP configuration
Project to validate the Moore-DaVinci-Tupling
workflow when using the veloSP sequence, developed for MC samples without Retina clusters. It consists of different tests,
each one dedicated to test a specific step of the chain.
All the test commands are reported in the Tupling_veloSP/CMakeLists.txt
file.
The test chain consists of a dedicated HLT1 step while all the other are the same as the ones run in
the Tupling_default test chain.
The test chain on simulation import the global [Utilities/jobMC_options.yaml
](https://gitlab.cern.ch/lhcb/LHCbIntegrationTests/-/blob/master/Utilities/jobMC_options.yaml) configuration and consists of the following steps:
- Start the test chain running HLT1 Allen on a MC .digi file from the TestFileDB.
Function: [
Moore.production:hlt1
](https://gitlab.cern.ch/lhcb/Moore/-/blob/master/Hlt/Moore/python/Moore/production.py)Option file: [
Tupling_veloSP/option_moore_hlt1_allen.yaml
](https://gitlab.cern.ch/lhcb/LHCbIntegrationTests/-/blob/master/Tupling_veloSP/option_moore_hlt1_allen.yaml)Extra option: hlt1_pp_veloSP
- Run the HLT2 on the HLT1 step output, testing the MC workflow.
Function: [
Moore.production:hlt2
](https://gitlab.cern.ch/lhcb/Moore/-/blob/master/Hlt/Moore/python/Moore/production.py)Option file: [
Tupling_default/option_moore_hlt2_all_lines.yaml
](https://gitlab.cern.ch/lhcb/LHCbIntegrationTests/-/blob/master/Tupling_default/option_moore_hlt2_all_lines.yaml)
- Run offline processing on the HLT2 step output with:
- a Passthrough line
Function: [
Hlt2Conf.Sprucing_tests:spruce_passthrough
](https://gitlab.cern.ch/lhcb/Moore/-/blob/master/Hlt/Hlt2Conf/python/Hlt2Conf/Sprucing_tests.py)Option file: [
Tupling_default/option_moore_passthrough.yaml
](https://gitlab.cern.ch/lhcb/LHCbIntegrationTests/-/blob/master/Tupling_default/option_moore_passthrough.yaml)
- a Sprucing line
Function: [
Hlt2Conf.Sprucing_tests:spruce_all_lines_realtime
](https://gitlab.cern.ch/lhcb/Moore/-/blob/master/Hlt/Hlt2Conf/python/Hlt2Conf/Sprucing_tests.py)Option file: [
Tupling_default/option_moore_spruce_all_lines.yaml
](https://gitlab.cern.ch/lhcb/LHCbIntegrationTests/-/blob/master/Tupling_default/option_moore_spruce.yaml)
- Run DaVinci and create a tuple from:
- the HLT2 step output
Function: [
DaVinciTests.options_lhcbintegration_mc_excl_line:main
](https://gitlab.cern.ch/lhcb/DaVinci/-/blob/master/DaVinciTests/python/DaVinciTests/options_lhcbintegration_mc_excl_line.py)Option file: [
Tupling_default/option_davinci_tupling_from_turbo.yaml
](https://gitlab.cern.ch/lhcb/LHCbIntegrationTests/-/blob/master/Tupling_default/option_davinci_tupling_from_turbo.yaml)
- the Spruce step output
Function: [
DaVinciTests.options_lhcbintegration_mc_excl_line:main
](https://gitlab.cern.ch/lhcb/DaVinci/-/blob/master/DaVinciTests/python/DaVinciTests/options_lhcbintegration_mc_excl_line.py)Option file: [
Tupling_default/option_davinci_tupling_from_spruce.yaml
](https://gitlab.cern.ch/lhcb/LHCbIntegrationTests/-/blob/master/Tupling_default/option_davinci_tupling_from_spruce.yaml)
Test for availability of Flavour Tagging variables
This integration test aims to determine if the flavour tagging variables are available in the ntuples.
The input is Upgrade Bd->DPi simulation in xdigi
format (used in [B2DX early measurements](https://gitlab.cern.ch/lhcb-b2oc/analyses/b2dx-early-measurements/-/blob/master/options/Hlt1/xdigi_input_Bd2DPi_down.py)).
Afterwards, a Moore job runs HLT1 and saves the result as a dst
file. Next, Moore is used to apply an exclusive flavour tagging HLT2 line, BdToDmPi_DmToPimPimKp
, saving all long and upstream particles as extra_outputs
and saving it to a dst
file.
Finally, Davinci is run to produce a root
file from this and it is checked if the resulting nTuple contains the flavour tagging variables and the expected values and number of events.
Each step of the chain is run using the Utilitites/logscript.sh
script that saves the corresponding _stdout_ and _stderr_.
The _stderr_ is then used for step validation by means of the Utilities/validator.sh
script ensuring no error is
dumped in the _stderr_.
For the final ntuple, an additional validation is performed using the Tupling_FT/validator_tuple_FT.py
script, checking that
the _DecayTree_ is not empty and contains all the expected flavour tagging branches.
The test chain consists of the following steps:
- Start test chain on Bd->DPi MC
xdigi
file alse used in [B2DX early measurements](https://gitlab.cern.ch/lhcb-b2oc/analyses/b2dx-early-measurements/-/blob/master/options/Hlt1/xdigi_input_Bd2DPi_down.py) Function: [
Moore.production:hlt1
](https://gitlab.cern.ch/lhcb/Moore/-/blob/master/Hlt/Moore/python/Moore/production.py)Option file::
Tupling_FT/option_moore_hlt1_allen.yaml
- Start test chain on Bd->DPi MC
- Run HLT2 on the HLT1 output
Function: [
Moore.tests.lhcbintegrationtests_options_tupling_ft:hlt2_ft
](https://gitlab.cern.ch/lhcb/Moore/-/blob/master/Hlt/Moore/python/Moore/tests/lhcbintegrationtests_options_tupling_ft.py)Option file:
Tupling_FT/option_moore_hlt2.yaml
- Run DaVinci on the HLT2 output and create an ntuple:
Function: [
DaVinciTests.options_lhcbintegration_mc_ft_bd2dpi:main
](https://gitlab.cern.ch/lhcb/DaVinci/-/blob/master/DaVinciTests/python/DaVinciTests/options_lhcbintegration_mc_ft_bd2dpi.py)Option file
Tupling_FT/option_davinci_tupling_from_hlt2.yaml
- Test the resulting root file
Python file
Tupling_FT/validator_tuple_FT.py
json file
Tupling_FT/validator_tuple_FT.json
Test for neutrals
This integration test aims to determine if the neutral info in proto particles are stored correctly in the ntuples, using the Hlt2RD_BToEEGamma
line.
The test chain consists of the following steps:
- Run HLT2 on the HLT1 input
Function: [
Moore.tests.lhcbintegrationtests_options_neutralst:neutrals
](https://gitlab.cern.ch/lhcb/Moore/-/blob/master/Hlt/Moore/python/Moore/tests/lhcbintegrationtests_options_neutrals.py)Option file:
Tupling_neutrals/option_moore_hlt2.yaml
- Run Passthrough on HLT2 output
Function: [
Hlt2Conf.Sprucing_tests:spruce_passthrough
](https://gitlab.cern.ch/lhcb/Moore/-/blob/master/Hlt/Hlt2Conf/python/Hlt2Conf/Sprucing_tests.py)Option file:
Tupling_neutrals/option_moore_passthrough.yaml
- Run DaVinci on the Passthrough output and create an ntuple:
Function: [
DaVinciTests.options_lhcbintegration_neutrals:main
](https://gitlab.cern.ch/lhcb/DaVinci/-/blob/master/DaVinciTests/python/DaVinciTests/options_lhcbintegration_neutrals.py)Option file
Tupling_neutrals/option_davinci_from_pass.yaml
- Test the resulting root file
Python file
Tupling_neutrals/variables_tuple.py
Validation of Moore-DaVinci chain with default configuration for PV reconstruction
Project to validate the Moore-DaVinci-Tupling
workflow for PV reconstruction. It consists of different tests,
each one dedicated to test a specific step of the chain.
All the test commands are reported in the Tupling_DTF_pvs/CMakeLists.txt
file.
The test chain starts with a MC .digi file and creates a root
file
containing a tuple produced with _FunTuple_.
The final tuple is created reading a passthrough dst
file.
Each step of the chain is run using the Utilitites/logscript.sh
script that saves the corresponding _stdout_ and _stderr_.
The _stderr_ is then used for step validation by means of the Utilities/validator.sh
script ensuring no error is
dumped in the _stderr_.
For the final tuple an additional validation is performed using the validator_tuple.py
script checking that the expected branches are consistent.
The test chain consists of the following steps:
- Run the HLT2 on digi input, testing the MC workflow.
Function: [
Moore.production:hlt2
](https://gitlab.cern.ch/lhcb/Moore/-/blob/master/Hlt/Moore/python/Moore/production.py)Option file: [
Tupling_DTF_pvs/option_moore_hlt2_all_lines.yaml
](https://gitlab.cern.ch/lhcb/LHCbIntegrationTests/-/blob/master/Tupling_DTF_pvs/option_moore_hlt2_all_lines.yaml)
- Run offline processing on the HLT2 step output with:
- a Passthrough line
Function: [
Hlt2Conf.Sprucing_tests:spruce_passthrough
](https://gitlab.cern.ch/lhcb/Moore/-/blob/master/Hlt/Hlt2Conf/python/Hlt2Conf/Sprucing_tests.py)Option file: [
Tupling_DTF_pvs/option_moore_passthrough.yaml
](https://gitlab.cern.ch/lhcb/LHCbIntegrationTests/-/blob/master/Tupling_DTF_pvs/option_moore_passthrough.yaml)
- Run DaVinci and create a tuple from:
- the HLT2 step output
Function: [
DaVinciExamples.tupling.option_davinci_Tupling_DTF_pvs:main
](https://gitlab.cern.ch/lhcb/DaVinci/-/blob/master/DaVinciExamples/python/DaVinciExamples/tupling/option_davinci_tupling_DTF_pvs.py?ref_type=heads)Option file: [
Tupling_DTF_pvs/option_davinci_tupling_from_turbo.yaml
](https://gitlab.cern.ch/lhcb/LHCbIntegrationTests/-/blob/master/Tupling_DTF_pvs/option_davinci_tupling_from_turbo.yaml)