XAMSL data access

S. Di Pede, last update: 30.3.2022

By the time of writing this documentation, you can have access to the data if you activate the amstrax_2021 environment on Stoomboot.

This documentation page uses the same structure idea of the STRAXEN documentation.

As a first step, check that you have access to the MongoDB on the xams-daq PC. Have a look at the AMSTRAX documentation and ask your colleagues for the credentials.

[1]:

import strax
import straxen
import amstrax

2022-03-30 14:40:05,669 - utilix - WARNING - Could not load a configuration file. You can create one at /user/serenap/.xenon_config, or set a custom path using

export XENON_CONFIG=path/to/your/config

If the imports do not succeed please check that the location of the init.py file is here: /data/xenon/xamsl/software/amstrax/amstrax/

[2]:

import os.path
print(f'Amstrax location: {os.path.abspath(amstrax.__file__)}')
print(f'Strax location: {os.path.abspath(strax.__file__)}')
print(f'Straxen location: {os.path.abspath(straxen.__file__)}')

Amstrax location: /data/xenon/xamsl/software/amstrax/amstrax/__init__.py
Strax location: /data/xenon/xamsl/software/strax/strax/__init__.py
Straxen location: /data/xenon/xamsl/software/straxen/straxen/__init__.py

In amstrax we have two contexts for the two different detectors operated in the setup:

and you can access the data just with the following commands:

[3]:

st = amstrax.contexts.xams_little()
st_xams = amstrax.contexts.xams()

[amstrax.rundb.RunDB, readonly: True, strax.storage.files.DataDirectory, readonly: True, path: /data/xenon/xamsl/raw/, take_only: ('raw_records_v1724', 'raw_records_v1730', 'raw_records_aqmon'), strax.storage.files.DataDirectory, readonly: True, path: /data/xenon/xamsl/processed/, strax.storage.files.DataDirectory, path: ./amstrax_data]
[amstrax.rundb.RunDB, readonly: True, strax.storage.files.DataDirectory, readonly: True, path: /data/xenon/xams/raw/, take_only: ('raw_records_v1724', 'raw_records_v1730', 'raw_records_aqmon'), strax.storage.files.DataDirectory, readonly: True, path: /data/xenon/xams/processed/, strax.storage.files.DataDirectory, path: ./amstrax_data]

The contexts method prints out the data storage directories:

the run DataBase: amstrax.rundb.RunDB, storing the metadata
two strax storage DataDirectory: /data/xenon/xamsl/raw/ and /data/xenon/xamsl/processed/, storing, respectively, the raw data (raw_records) and all the other high-level processed data
a general path where the data are stored in case they have not been processed yet in the above strax storage folders: ./amstrax_data.

Run selection

In this documentation, we focus only on accessing the XAMSL data. To select the measurement of your interest please have a look at the details of XAMSL measurements.

To select all the available runs in the runDB (not only XAMSL measurements) you can use the strax method:

[4]:

runs = st.select_runs()
runs

Fetching run info from MongoDB: 100%|██████████████████████████████████████████████████████████| 2574/2574 [00:00<00:00, 7309.73it/s]

[4]:

	name	number	mode	start	end	tags	livetime	processing_status	raw_records_v1730_available	raw_records_v1724_available
0	000000	0	V1724_Run0	2020-11-23 08:36:02.740	2020-11-23 08:36:50.846		0 days 00:00:48.106000	NaN	False	False
1	000001	1	V1724_Run0	2020-11-24 07:51:24.798	2020-11-24 07:51:36.827		0 days 00:00:12.029000	NaN	False	False
2	000002	2	V1724_Run0	2020-11-24 12:55:19.577	2020-11-24 12:55:28.594		0 days 00:00:09.017000	NaN	False	False
3	000003	3	V1724_Run0	2020-11-24 13:08:10.073	2020-11-24 13:08:19.093		0 days 00:00:09.020000	NaN	False	False
4	000004	4	V1730_Run3	2020-11-24 13:54:39.023	2020-11-24 13:55:00.072		0 days 00:00:21.049000	NaN	False	False
...	...	...	...	...	...	...	...	...	...	...
2569	002569	2569	test_run_two_digitizers	2021-12-16 13:01:03.553	2021-12-16 13:01:11.676		0 days 00:00:08.123000	NaN	False	False
2570	002570	2570	test_run_two_digitizers	2021-12-16 13:02:03.861	2021-12-16 13:02:11.974		0 days 00:00:08.113000	NaN	False	False
2571	002571	2571	test_run_two_digitizers	2021-12-16 13:03:04.114	2021-12-16 13:03:09.134		0 days 00:00:05.020000	NaN	False	False
2572	002572	2572	test_run_two_digitizers	2021-12-16 13:04:04.404	2021-12-16 13:04:09.532		0 days 00:00:05.128000	pending	False	False
2573	002573	2573	test_run_two_digitizers	2021-12-16 13:05:04.660	2021-12-16 13:05:09.685		0 days 00:00:05.025000	pending	False	False

2574 rows × 10 columns

Datastructure

The datastructure of XAMSL is different from the datastructure of XAMS. The two datastructures are kept separate to allow easier reprocessing and changes, therefore you need to keep in mind the right context and data-kind when you want to look into the data.

The available data kinds for XAMSL can be looked through the plugins class registry:

[5]:

plugins = st._plugin_class_registry

print(f"Which data kinds are available for XAMSL data?\n")

for datakind in plugins.keys():
    print(datakind)

Which data kinds are available for XAMSL data?

radon_records
radon_pulse_counts
radon_hits
radon_peaks
radon_lone_hits
radon_peak_basics
raw_records_v1724
raw_records_v1730
raw_records_aqmon

The XAMS data kinds are instead:

[6]:

plugins_xams = st_xams._plugin_class_registry

print(f"Which data kinds are available for XAMS data?\n")

for datakind_xams in plugins_xams.keys():
    print(datakind_xams)

Which data kinds are available for XAMS data?

records
pulse_counts
hits
peaks
lone_hits
peak_basics
raw_records_v1724
raw_records_v1730
raw_records_aqmon

You might have noticed that XAMSL plugins have the radon_ keyword before the data-kind name except in the raw_records level.

Info about the datakind can be retrieved using:

[7]:

st.data_info('radon_records')

[7]:

	Field name	Data type	Comment
0	time	int64	Start time since unix epoch [ns]
1	length	int32	Length of the interval in samples
2	dt	int16	Width of one sample [ns]
3	channel	int16	Channel/PMT number
4	pulse_length	int32	Length of pulse to which the record belongs (w...
5	record_i	int16	Fragment number in the pulse
6	area	int32	Integral in ADC counts x samples
7	reduction_level	uint8	Level of data reduction applied (strax.Reducti...
8	baseline	float32	Baseline in ADC counts. data = int(baseline) -...
9	baseline_rms	float32	Baseline RMS in ADC counts. data = baseline - ...
10	amplitude_bit_shift	int16	Multiply data by 2**(this number). Baseline is...
11	data	('<i2', (110,))	Waveform data in raw counts above integer part...

Data loading

You can load the data using the st.get_array function of the strax/straxen framework. Below, we load data for the measurement 000736. You might notice that we only load raw_records_v1730 because this was a measurement taken with the V1730 digitizer.

[8]:

rr = st.get_array('000736', 'raw_records_v1730',progress_bar=True)
rec = st.get_array('000736', 'radon_records',progress_bar=True)
pulse_count = st.get_array('000736', 'radon_pulse_counts',progress_bar=True)
peaks = st.get_array('000736', 'radon_peaks',progress_bar=True)
lone_hits = st.get_array('000736', 'radon_lone_hits',progress_bar=True)
peak_basics = st.get_array('000736', 'radon_peak_basics',progress_bar=True)

Details about raw_records in XAMSL

As you can see in this wiki-page, you will find you raw data only in one of the raw_records type and only in specific channels. Let’s take as an example the measurement 000282 and 000736, which have been taken respectively with the V1724 and V1730 digitizers.

[9]:

rr_000736_v1724=st.get_array('000736', 'raw_records_v1724',progress_bar=True)
rr_000282_v1724=st.get_array('000282', 'raw_records_v1724',progress_bar=True)
rr_000282_v1730=st.get_array('000282', 'raw_records_v1730',progress_bar=True)
rec_000282=st.get_array('000282', 'radon_records',progress_bar=True)

We do expect to see data in: * channel 0, 1 in measurement 000736 of raw_records_v1730 and their respective records * channel 2, 3 in measurement 000282 of raw_records_v1724 and their respective records

All the other channels must be empty. Let’s check.

[10]:

print(f'Measurement 000736, raw_records_v1730')
for ch in range(8):
    rr_ch=rr[rr['channel']==ch]
    print(f'Channel {ch}, data:')
    print(rr_ch['data'])

print(f'\nMeasurement 000736, raw_records_v1724')
for ch in range(8):
    rr_ch=rr_000736_v1724[rr_000736_v1724['channel']==ch]
    print(f'Channel {ch}, data:')
    print(rr_ch['data'])

Measurement 000736, raw_records_v1730
Channel 0, data:
[[8220 8222 8225 ... 7530 7654 7743]
 [7777 7808 7851 ... 8220 8222 8220]
 [8223 8220 8222 ... 8221 8222 8224]
 ...
 [8222 8221 8225 ... 8224 8219 8222]
 [8221 8227 8225 ... 8223 8225 8223]
 [8225 8223 8225 ...    0    0    0]]
Channel 1, data:
[[8199 8200 8200 ... 8012 8003 8025]
 [8077 8102 8098 ... 8199 8199 8203]
 [8203 8204 8205 ... 8199 8199 8202]
 ...
 [8199 8202 8200 ... 8201 8199 8198]
 [8202 8199 8201 ... 8201 8200 8201]
 [8199 8202 8199 ...    0    0    0]]
Channel 2, data:
[]
Channel 3, data:
[]
Channel 4, data:
[]
Channel 5, data:
[]
Channel 6, data:
[]
Channel 7, data:
[]

Measurement 000736, raw_records_v1724
Channel 0, data:
[]
Channel 1, data:
[]
Channel 2, data:
[]
Channel 3, data:
[]
Channel 4, data:
[]
Channel 5, data:
[]
Channel 6, data:
[]
Channel 7, data:
[]

[11]:

print(f'Measurement 000282, raw_records_v1730')
for ch in range(8):
    rr_ch=rr_000282_v1730[rr_000282_v1730['channel']==ch]
    print(f'Channel {ch}, data:')
    print(rr_ch['data'])

print(f'\nMeasurement 000282, raw_records_v1724')
for ch in range(8):
    rr_ch=rr_000282_v1724[rr_000282_v1724['channel']==ch]
    print(f'Channel {ch}, data:')
    print(rr_ch['data'])

Measurement 000282, raw_records_v1730
Channel 0, data:
[]
Channel 1, data:
[]
Channel 2, data:
[]
Channel 3, data:
[]
Channel 4, data:
[]
Channel 5, data:
[]
Channel 6, data:
[]
Channel 7, data:
[]

Measurement 000282, raw_records_v1724
Channel 0, data:
[]
Channel 1, data:
[]
Channel 2, data:
[[16082 16082 16082 ... 16077 16079 16081]
 [16076 16077 16078 ...     0     0     0]
 [16083 16083 16082 ...     0     0     0]
 ...
 [16080 16083 16081 ...     0     0     0]
 [16083 16081 16082 ...     0     0     0]
 [16077 16077 16078 ...     0     0     0]]
Channel 3, data:
[[16099 16099 16098 ...     0     0     0]
 [16085 16084 16086 ...     0     0     0]
 [16100 16098 16098 ... 15994 16020 15965]
 ...
 [16100 16095     0 ...     0     0     0]
 [16014 16033 15998 ...     0     0     0]
 [16094 16093 16093 ... 16078     0     0]]
Channel 4, data:
[]
Channel 5, data:
[]
Channel 6, data:
[]
Channel 7, data:
[]

The same rule stands for the radon_records of the specific measurement.

[12]:

print(f'Measurement 000736')
for ch in range(8):
    rr_ch=rec[rec['channel']==ch]
    print(f'Channel {ch}, data:')
    print(rr_ch['data'])

print(f'\nMeasurement 000282')
for ch in range(8):
    rr_ch=rec_000282[rec_000282['channel']==ch]
    print(f'Channel {ch}, data:')
    print(rr_ch['data'])

Measurement 000736
Channel 0, data:
[[  0   0   0 ... 692 568 479]
 [445 414 371 ...   0   0   0]
 [  0   0   0 ...   0   0   0]
 ...
 [  0   0   0 ...   0   0   0]
 [  0   0   0 ...   0   0   0]
 [  0   0   0 ...   0   0   0]]
Channel 1, data:
[[  0   0   0 ... 188 197 175]
 [123  98 102 ...   0   0   0]
 [  0   0   0 ...   0   0   0]
 ...
 [  0   0   0 ...   0   0   0]
 [  0   0   0 ...   0   0   0]
 [  0   0   0 ...   0   0   0]]
Channel 2, data:
[]
Channel 3, data:
[]
Channel 4, data:
[]
Channel 5, data:
[]
Channel 6, data:
[]
Channel 7, data:
[]

Measurement 000282
Channel 0, data:
[]
Channel 1, data:
[]
Channel 2, data:
[[0 0 0 ... 5 3 1]
 [0 0 0 ... 0 0 0]
 [0 0 0 ... 0 0 0]
 ...
 [0 0 0 ... 0 0 0]
 [0 0 0 ... 0 0 0]
 [0 0 0 ... 0 0 0]]
Channel 3, data:
[[  0   0   0 ...   0   0   0]
 [  0   0   0 ...   0   0   0]
 [  0   0   0 ... 102  76 131]
 ...
 [  0   0   0 ...   0   0   0]
 [ 40  21  56 ...   0   0   0]
 [  0   0   0 ...  18   0   0]]
Channel 4, data:
[]
Channel 5, data:
[]
Channel 6, data:
[]
Channel 7, data:
[]

Details of the XAMSL measurements

You can also load the details of the XAMSL measurements inside your working Jupyter Notebook during your analysis. The amstrax_files contain informations of the lab-logbook such as start/stop of the recirculation.

[13]:

import amstrax_files

This is the list of the files contained in the amstrax_files at the time of writing this documentation.

[14]:

print(amstrax_files.list_files())

['999999.tar', 'example.json', 'rundoc_999999.json', 'xamsl_measurements.csv']

Load the xamsl_measurements details file

[15]:

measurements = amstrax_files.get_file('xamsl_measurements.csv')

Select the range of measurements you are interested in e.g. the recirculation and purification effect investigation with the V1730 digitizer during stable HV conditions on the PMTs.

[16]:

moi = measurements.loc[(measurements['run'] >= 1588) & (measurements['run'] <= 1952)]
moi

[16]:

	run	pmt_bottom_v	pmt_top_v	notes
1219	1588	-560	-590	reci on/getter on-digi th 30ADC
1220	1589	-560	-590	reci on/getter on-digi th 30ADC
1221	1590	-560	-590	reci on/getter off-digi th 30ADC
1222	1591	-560	-590	reci on/getter off-digi th 30ADC
1223	1592	-560	-590	reci on/getter off-digi th 30ADC
...	...	...	...	...
1579	1948	-560	-590	reci on/getter off-digi th 30ADC
1580	1949	-560	-590	reci on/getter off-digi th 30ADC
1581	1950	-560	-590	reci on/getter off-digi th 30ADC
1582	1951	-560	-590	reci on/getter off-digi th 30ADC
1583	1952	-560	-590	reci on/getter on-digi th 30ADC

365 rows × 4 columns

[17]:

moi['notes']

[17]:

1219      reci on/getter on-digi th 30ADC
1220      reci on/getter on-digi th 30ADC
1221     reci on/getter off-digi th 30ADC
1222     reci on/getter off-digi th 30ADC
1223     reci on/getter off-digi th 30ADC
                      ...
1579     reci on/getter off-digi th 30ADC
1580     reci on/getter off-digi th 30ADC
1581     reci on/getter off-digi th 30ADC
1582     reci on/getter off-digi th 30ADC
1583      reci on/getter on-digi th 30ADC
Name: notes, Length: 365, dtype: object

Data not available! -> You can process data by yourself!

If, for some reason, you receive a message that data are not available, you can try to make the data by yourself.

Specify the run number you want to make data of.

[ ]:

run_list = ['','',...]

And the target data-kind you need

[ ]:

target = '...'

Then just look over and use the strax function st.make.

[ ]:

from strax.utils import tqdm

for r in tqdm(run_list):
    print(r)
    st.make(r,target,progress_bar=True)

As we said before, these new processed data will be saved in the storage directory ./amstrax_data. You don’t need to remake the data every time as strax will know now where to find them.