{ "cells": [ { "cell_type": "markdown", "id": "89051158", "metadata": {}, "source": [ "# XAMSL data access\n", "\n", "_S. Di Pede, last update: 30.3.2022_\n", "\n", "By the time of writing this documentation, you can have access to the data if you activate the ``amstrax_2021`` environment on Stoomboot.\n", "\n", "This documentation page uses the same structure idea of the [STRAXEN documentation](https://straxen.readthedocs.io/en/latest/index.html).\n", "\n", "As a first step, check that you have access to the MongoDB on the xams-daq PC. Have a look at the [AMSTRAX documentation](https://amstrax.readthedocs.io/en/latest/setup.html) and ask your colleagues for the credentials. " ] }, { "cell_type": "code", "execution_count": 1, "id": "a314ab7c", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2022-03-30 14:40:05,669 - utilix - WARNING - Could not load a configuration file. You can create one at /user/serenap/.xenon_config, or set a custom path using\n", "\n", "export XENON_CONFIG=path/to/your/config\n", "\n" ] } ], "source": [ "import strax\n", "import straxen\n", "import amstrax" ] }, { "cell_type": "markdown", "id": "2f151841", "metadata": {}, "source": [ "If the imports do not succeed please check that the location of the **__init__.py** file is here: ``/data/xenon/xamsl/software/amstrax/amstrax/``" ] }, { "cell_type": "code", "execution_count": 2, "id": "351c508f", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Amstrax location: /data/xenon/xamsl/software/amstrax/amstrax/__init__.py\n", "Strax location: /data/xenon/xamsl/software/strax/strax/__init__.py\n", "Straxen location: /data/xenon/xamsl/software/straxen/straxen/__init__.py\n" ] } ], "source": [ "import os.path\n", "print(f'Amstrax location: {os.path.abspath(amstrax.__file__)}')\n", "print(f'Strax location: {os.path.abspath(strax.__file__)}')\n", "print(f'Straxen location: {os.path.abspath(straxen.__file__)}')" ] }, { "cell_type": "markdown", "id": "153d1c44", "metadata": {}, "source": [ "In ``amstrax`` we have two contexts for the two different detectors operated in the setup:\n", "\n", " * [XAMSL detector](https://wiki.nikhef.nl/xenon/doku.php?id=xamsl) \n", " * [XAMS detector](https://wiki.nikhef.nl/xenon/doku.php?id=xams)\n", " \n", "and you can access the data just with the following commands:" ] }, { "cell_type": "code", "execution_count": 3, "id": "903d1506", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[amstrax.rundb.RunDB, readonly: True, strax.storage.files.DataDirectory, readonly: True, path: /data/xenon/xamsl/raw/, take_only: ('raw_records_v1724', 'raw_records_v1730', 'raw_records_aqmon'), strax.storage.files.DataDirectory, readonly: True, path: /data/xenon/xamsl/processed/, strax.storage.files.DataDirectory, path: ./amstrax_data]\n", "[amstrax.rundb.RunDB, readonly: True, strax.storage.files.DataDirectory, readonly: True, path: /data/xenon/xams/raw/, take_only: ('raw_records_v1724', 'raw_records_v1730', 'raw_records_aqmon'), strax.storage.files.DataDirectory, readonly: True, path: /data/xenon/xams/processed/, strax.storage.files.DataDirectory, path: ./amstrax_data]\n" ] } ], "source": [ "st = amstrax.contexts.xams_little()\n", "st_xams = amstrax.contexts.xams()" ] }, { "cell_type": "markdown", "id": "0e173589", "metadata": {}, "source": [ "The ``contexts`` method prints out the data storage directories:\n", "\n", " * the run DataBase: ``amstrax.rundb.RunDB``, storing the metadata\n", " * two strax storage DataDirectory: ``/data/xenon/xamsl/raw/`` and ``/data/xenon/xamsl/processed/``, storing, respectively, the raw data (raw_records) and all the other high-level processed data\n", " * a general path where the data are stored in case they have not been processed yet in the above strax storage folders: ``./amstrax_data``. " ] }, { "cell_type": "markdown", "id": "0f09aba8", "metadata": {}, "source": [ "### Run selection\n", "\n", "In this documentation, we focus only on accessing the XAMSL data. To select the measurement of your interest please have a look at the [details of XAMSL measurements](https://wiki.nikhef.nl/xenon/doku.php?id=xamsl:run1_lxe_measurements).\n", "\n", "To select _all_ the available runs in the runDB (not only XAMSL measurements) you can use the strax method:" ] }, { "cell_type": "code", "execution_count": 4, "id": "18603944", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Fetching run info from MongoDB: 100%|██████████████████████████████████████████████████████████| 2574/2574 [00:00<00:00, 7309.73it/s]\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "0150827047e94a5394ab34d76b92fc0e", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Checking data availability: 0%| | 0/2 [00:00\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
namenumbermodestartendtagslivetimeprocessing_statusraw_records_v1730_availableraw_records_v1724_available
00000000V1724_Run02020-11-23 08:36:02.7402020-11-23 08:36:50.8460 days 00:00:48.106000NaNFalseFalse
10000011V1724_Run02020-11-24 07:51:24.7982020-11-24 07:51:36.8270 days 00:00:12.029000NaNFalseFalse
20000022V1724_Run02020-11-24 12:55:19.5772020-11-24 12:55:28.5940 days 00:00:09.017000NaNFalseFalse
30000033V1724_Run02020-11-24 13:08:10.0732020-11-24 13:08:19.0930 days 00:00:09.020000NaNFalseFalse
40000044V1730_Run32020-11-24 13:54:39.0232020-11-24 13:55:00.0720 days 00:00:21.049000NaNFalseFalse
.................................
25690025692569test_run_two_digitizers2021-12-16 13:01:03.5532021-12-16 13:01:11.6760 days 00:00:08.123000NaNFalseFalse
25700025702570test_run_two_digitizers2021-12-16 13:02:03.8612021-12-16 13:02:11.9740 days 00:00:08.113000NaNFalseFalse
25710025712571test_run_two_digitizers2021-12-16 13:03:04.1142021-12-16 13:03:09.1340 days 00:00:05.020000NaNFalseFalse
25720025722572test_run_two_digitizers2021-12-16 13:04:04.4042021-12-16 13:04:09.5320 days 00:00:05.128000pendingFalseFalse
25730025732573test_run_two_digitizers2021-12-16 13:05:04.6602021-12-16 13:05:09.6850 days 00:00:05.025000pendingFalseFalse
\n", "

2574 rows × 10 columns

\n", "" ], "text/plain": [ " name number mode start \\\n", "0 000000 0 V1724_Run0 2020-11-23 08:36:02.740 \n", "1 000001 1 V1724_Run0 2020-11-24 07:51:24.798 \n", "2 000002 2 V1724_Run0 2020-11-24 12:55:19.577 \n", "3 000003 3 V1724_Run0 2020-11-24 13:08:10.073 \n", "4 000004 4 V1730_Run3 2020-11-24 13:54:39.023 \n", "... ... ... ... ... \n", "2569 002569 2569 test_run_two_digitizers 2021-12-16 13:01:03.553 \n", "2570 002570 2570 test_run_two_digitizers 2021-12-16 13:02:03.861 \n", "2571 002571 2571 test_run_two_digitizers 2021-12-16 13:03:04.114 \n", "2572 002572 2572 test_run_two_digitizers 2021-12-16 13:04:04.404 \n", "2573 002573 2573 test_run_two_digitizers 2021-12-16 13:05:04.660 \n", "\n", " end tags livetime processing_status \\\n", "0 2020-11-23 08:36:50.846 0 days 00:00:48.106000 NaN \n", "1 2020-11-24 07:51:36.827 0 days 00:00:12.029000 NaN \n", "2 2020-11-24 12:55:28.594 0 days 00:00:09.017000 NaN \n", "3 2020-11-24 13:08:19.093 0 days 00:00:09.020000 NaN \n", "4 2020-11-24 13:55:00.072 0 days 00:00:21.049000 NaN \n", "... ... ... ... ... \n", "2569 2021-12-16 13:01:11.676 0 days 00:00:08.123000 NaN \n", "2570 2021-12-16 13:02:11.974 0 days 00:00:08.113000 NaN \n", "2571 2021-12-16 13:03:09.134 0 days 00:00:05.020000 NaN \n", "2572 2021-12-16 13:04:09.532 0 days 00:00:05.128000 pending \n", "2573 2021-12-16 13:05:09.685 0 days 00:00:05.025000 pending \n", "\n", " raw_records_v1730_available raw_records_v1724_available \n", "0 False False \n", "1 False False \n", "2 False False \n", "3 False False \n", "4 False False \n", "... ... ... \n", "2569 False False \n", "2570 False False \n", "2571 False False \n", "2572 False False \n", "2573 False False \n", "\n", "[2574 rows x 10 columns]" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "runs = st.select_runs()\n", "runs" ] }, { "cell_type": "markdown", "id": "b5a23e9c", "metadata": {}, "source": [ "### Datastructure\n", "\n", "The [datastructure of XAMSL](https://amstrax.readthedocs.io/en/latest/reference/datastructure_xams_little.html) is different from the [datastructure of XAMS](https://amstrax.readthedocs.io/en/latest/reference/datastructure_xams.html). The two datastructures are kept separate to allow easier reprocessing and changes, therefore you need to keep in mind the right context and data-kind when you want to look into the data. \n", "\n", "The available data kinds for XAMSL can be looked through the plugins class registry:" ] }, { "cell_type": "code", "execution_count": 5, "id": "28f1eb71", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Which data kinds are available for XAMSL data?\n", "\n", "radon_records\n", "radon_pulse_counts\n", "radon_hits\n", "radon_peaks\n", "radon_lone_hits\n", "radon_peak_basics\n", "raw_records_v1724\n", "raw_records_v1730\n", "raw_records_aqmon\n" ] } ], "source": [ "plugins = st._plugin_class_registry\n", "\n", "print(f\"Which data kinds are available for XAMSL data?\\n\")\n", "\n", "for datakind in plugins.keys():\n", " print(datakind)" ] }, { "cell_type": "markdown", "id": "696fa07f", "metadata": {}, "source": [ "The XAMS data kinds are instead:" ] }, { "cell_type": "code", "execution_count": 6, "id": "67e14eb2", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Which data kinds are available for XAMS data?\n", "\n", "records\n", "pulse_counts\n", "hits\n", "peaks\n", "lone_hits\n", "peak_basics\n", "raw_records_v1724\n", "raw_records_v1730\n", "raw_records_aqmon\n" ] } ], "source": [ "plugins_xams = st_xams._plugin_class_registry\n", "\n", "print(f\"Which data kinds are available for XAMS data?\\n\")\n", "\n", "for datakind_xams in plugins_xams.keys():\n", " print(datakind_xams)" ] }, { "cell_type": "markdown", "id": "5e8750d8", "metadata": {}, "source": [ "You might have noticed that XAMSL plugins have the ``radon_`` keyword before the data-kind name except in the ``raw_records`` level." ] }, { "cell_type": "markdown", "id": "362d1ce1", "metadata": {}, "source": [ "Info about the datakind can be retrieved using:" ] }, { "cell_type": "code", "execution_count": 7, "id": "f954dc25", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Field nameData typeComment
0timeint64Start time since unix epoch [ns]
1lengthint32Length of the interval in samples
2dtint16Width of one sample [ns]
3channelint16Channel/PMT number
4pulse_lengthint32Length of pulse to which the record belongs (w...
5record_iint16Fragment number in the pulse
6areaint32Integral in ADC counts x samples
7reduction_leveluint8Level of data reduction applied (strax.Reducti...
8baselinefloat32Baseline in ADC counts. data = int(baseline) -...
9baseline_rmsfloat32Baseline RMS in ADC counts. data = baseline - ...
10amplitude_bit_shiftint16Multiply data by 2**(this number). Baseline is...
11data('<i2', (110,))Waveform data in raw counts above integer part...
\n", "
" ], "text/plain": [ " Field name Data type \\\n", "0 time int64 \n", "1 length int32 \n", "2 dt int16 \n", "3 channel int16 \n", "4 pulse_length int32 \n", "5 record_i int16 \n", "6 area int32 \n", "7 reduction_level uint8 \n", "8 baseline float32 \n", "9 baseline_rms float32 \n", "10 amplitude_bit_shift int16 \n", "11 data ('\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
runpmt_bottom_vpmt_top_vnotes
12191588-560-590reci on/getter on-digi th 30ADC
12201589-560-590reci on/getter on-digi th 30ADC
12211590-560-590reci on/getter off-digi th 30ADC
12221591-560-590reci on/getter off-digi th 30ADC
12231592-560-590reci on/getter off-digi th 30ADC
...............
15791948-560-590reci on/getter off-digi th 30ADC
15801949-560-590reci on/getter off-digi th 30ADC
15811950-560-590reci on/getter off-digi th 30ADC
15821951-560-590reci on/getter off-digi th 30ADC
15831952-560-590reci on/getter on-digi th 30ADC
\n", "

365 rows × 4 columns

\n", "" ], "text/plain": [ " run pmt_bottom_v pmt_top_v notes\n", "1219 1588 -560 -590 reci on/getter on-digi th 30ADC\n", "1220 1589 -560 -590 reci on/getter on-digi th 30ADC\n", "1221 1590 -560 -590 reci on/getter off-digi th 30ADC\n", "1222 1591 -560 -590 reci on/getter off-digi th 30ADC\n", "1223 1592 -560 -590 reci on/getter off-digi th 30ADC\n", "... ... ... ... ...\n", "1579 1948 -560 -590 reci on/getter off-digi th 30ADC\n", "1580 1949 -560 -590 reci on/getter off-digi th 30ADC\n", "1581 1950 -560 -590 reci on/getter off-digi th 30ADC\n", "1582 1951 -560 -590 reci on/getter off-digi th 30ADC\n", "1583 1952 -560 -590 reci on/getter on-digi th 30ADC\n", "\n", "[365 rows x 4 columns]" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "moi = measurements.loc[(measurements['run'] >= 1588) & (measurements['run'] <= 1952)]\n", "moi" ] }, { "cell_type": "code", "execution_count": 17, "id": "6e22b92d", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "1219 reci on/getter on-digi th 30ADC\n", "1220 reci on/getter on-digi th 30ADC\n", "1221 reci on/getter off-digi th 30ADC\n", "1222 reci on/getter off-digi th 30ADC\n", "1223 reci on/getter off-digi th 30ADC\n", " ... \n", "1579 reci on/getter off-digi th 30ADC\n", "1580 reci on/getter off-digi th 30ADC\n", "1581 reci on/getter off-digi th 30ADC\n", "1582 reci on/getter off-digi th 30ADC\n", "1583 reci on/getter on-digi th 30ADC\n", "Name: notes, Length: 365, dtype: object" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "moi['notes']" ] }, { "cell_type": "markdown", "id": "8acc8051", "metadata": {}, "source": [ "### Data not available! -> You can process data by yourself! \n", " \n", "If, for some reason, you receive a message that data are not available, you can try to make the data by yourself.\n", "\n", "Specify the run number you want to make data of." ] }, { "cell_type": "code", "execution_count": null, "id": "1cb58c82", "metadata": {}, "outputs": [], "source": [ "run_list = ['','',...]" ] }, { "cell_type": "markdown", "id": "e1f7834d", "metadata": {}, "source": [ "And the target data-kind you need" ] }, { "cell_type": "code", "execution_count": null, "id": "c1a43466", "metadata": {}, "outputs": [], "source": [ "target = '...'" ] }, { "cell_type": "markdown", "id": "922946f8", "metadata": {}, "source": [ "Then just look over and use the strax function ``st.make``. " ] }, { "cell_type": "code", "execution_count": null, "id": "f073433f", "metadata": {}, "outputs": [], "source": [ "from strax.utils import tqdm\n", "\n", "for r in tqdm(run_list):\n", " print(r)\n", " st.make(r,target,progress_bar=True)" ] }, { "cell_type": "markdown", "id": "794962b5", "metadata": {}, "source": [ "As we said before, these new processed data will be saved in the storage directory ``./amstrax_data``. You don't need to remake the data every time as strax will know now where to find them." ] }, { "cell_type": "markdown", "id": "b864da45", "metadata": {}, "source": [ "### Veel plezier met XAMSL data!" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.0" } }, "nbformat": 4, "nbformat_minor": 5 }