We use a rather broad palette of different programs and libraries to make everything work. In order to not have worry to much about the individual parts, we have written __DAQC__, which handles all the different parts and provides a user-friendly interface for control and monitoring. The following list is an extremely brief description of the different components. Most of the time, however, all you need to worry about is __DAQC__, which will be described in next section.
-`drasi`: The data acquisition (data pump) that handles the data transport aquired from our modules. It allows the use of a so called `f_user.c`, which is a piece of C code that handles the actual readout of the modules. The `f_user.c` is custom made. We run 3 drasi instances: one on each VME computer and an event builder on __Bumblebee__.
-`drasi`: The data acquisition (data pump) that handles the data transport aquired from our modules. It allows the use of a so called `f_user.c`, which is a piece of C code that handles the actual readout of the modules. The `f_user.c` is custom made. We run 2 drasi instances: one on each VME computer and an event builder on __Bumblebee__.
-`nurdlib`: The VME library that knows the data layout of a wide variety of modules and how to do an actual data readout. We use this to perform the tasks of our `f_user.c`.
-`trlo II`: Firmware for the VOLUM-models. We use this to program our trigger logic.
-`ucesb`: Data unpacking software. We use it mainly for two purposes. It is used to attach to the data stream from the event builder and save the data to disk. It is also used to attach to the data stream from the event builder and fan it out to multiple online analysis programs. A major advantage of `ucesb` is that it checks that the data is not corrupt online.
...
...
@@ -74,7 +74,7 @@ The web frontend will show you something like the following:
1. __VME computer*__ (__MOT2__): This box shows you whether the VME computer is online, i.e. if they can be reached from __Bumblebee__.
2. __Readout*__: This box shows the status of the readout, i.e. the event builder on __Bumblebee__ and the two `drasi` instances running on the VME-computer. The `1/1` means that the vme node is running. The event builder broadcasts the data such that we can attach onto it and save it to a file.
2. __Readout*__: This box shows the status of the readout, i.e. the event builder on __Bumblebee__ and the `drasi` instance running on the VME-computer. The `1/1` means that the vme node is running. The event builder broadcasts the data such that we can attach onto it and save it to a file.
3. __Relay__: The relay is a `ucesb` instance that attaches to the event builder's data stream and fans it out to several data streams. This also validates the data, i.e. if it crashes, the data may be corrupt. The relay provides data to, among others, the online analysis. So if the relay dies, so does the online analysis. The relay is *NOT* responsible for the data taking or recording, and is thus non-essential.
4. __Go4__: The online analysis. We run the online analysis as a server. That means that we do not NEED a GUI. Of course, to see anything, we need a GUI. However, by running the analysis as a server, we can attach multiple GUI instances. Moreover, if the GUI crashes, the online analysis process do not necessarily crash. In this way, we can also access Go4's web version in a browser. Open up a browser and type `bumblebee:5000`, and you will see the online analysis (if it is running, of course).
Notice that __DAQC__ only know about the server, and not about any of the GUI instances. So if they die, simply reopen them.
...
...
@@ -98,7 +98,7 @@ The web frontend will show you something like the following:
```
## Go4
To see the online analysis, you can open the dedicated GUI program GO4 on __Bumblebee__, by just typing `go4` in a terminal. If you want to watch the online analysis from a remote computer on the CERN network, you can open a browser and type `bumblebee:5000`. This will open a web version of the online analysis.
To see the online analysis, you can open the dedicated GUI program GO4 on __Bumblebee__, by just typing `go4` in a terminal. If you want to watch the online analysis from a remote computer on the local network, you can open a browser and type `bumblebee:5000`. This will open a web version of the online analysis.
The GUI looks like this
...
...
@@ -126,7 +126,7 @@ The prewritten code makes it easy to include calibrations. Simply create the cal
Grafana is hosted as a webservice and can be accessed at
__MISSING LINK__
https://grafana.kern.phys.au.dk/?orgId=1
There are a few dashboards of interest. For the general setup performance, one can look at the `Triggers` and `Dead time` which shows the rate and dead time for each trigger. We will make purpose built dashboard(s) for each experiment, once we know which triggers goes where etc.
...
...
@@ -138,8 +138,8 @@ monitor
#### Startup guide
This section will give you the step-by-step guide to how to start up the acqusition from scratch. It is usually only required if the VME computer lost power or was rebooted. Otherwise a simple restart of __DAQC__ should be enough.
1. __Turn on the VME-crate and the NIM-crates__: Make sure that all crates have power and are running. Wait a minute or two, to make sure both VME computers are online by ssh'ing into mot2.
2. __Start up DAQC__: Next we start up the acquisition:
1. __Turn on the VME-crate and the NIM-crates__: Make sure that all crates have power and are running. Wait a minute or two, to make sure the VME computer is online by ssh'ing into mot2.
2. __Start up DAQC__: Next we start up the acquisition, on __Bumblebee__:
```bash
screen -R daqc
daqc
...
...
@@ -158,6 +158,7 @@ This section will give you the step-by-step guide to how to start up the acqusit
2018-06-14 11:02:43,845 - [mot2] Started Drasi
2018-06-14 11:02:43,846 - drasi_mot2 revived
```
Detach from the screen again by executing: `Ctrl+A D` (`Ctrl+A ?` for a full list of `screen` commands).
6. __Check that the data acqusition is running__: We can now go to the __DAQC__ frontend to monitor everything. Go to a browser and enter `bumblebee:8080`. You should now see __DAQC__. Everything should be green. That means we are happy.
...
...
@@ -176,13 +177,13 @@ This section will give you the step-by-step guide to how to start up the acqusit
#### FAQ
- __Start DAQC__:
Simply open a terminal and issue:
Simply open a terminal on __Bumblebee__ and issue:
```bash
screen -R daqc
daqc
```
You are now inside a screen session called `daqc`, and __DAQC__ is starting up. If __DAQC__ was already running, we simply entered the screen session called `daqc`.
To detach from the screen session: `Ctrl+A D` (`Ctrl+A ?` for a full list of `screen` commands).
__What happend? (for the interested reader):__
We run `daqc` in a screen session. This makes it easy to manage a single `daqc` instance from any terminal. More over, we run `daqc` in a socalled **virtual environment** or `virtualenv` due to the various python dependencies. Therefore, when executing `daqc` we actually execute a script that makes sure that we are both inside a screen session and inside a `virtualenv`. The command `daqc` is thus just an alias that executes a script that does the following:
...
...
@@ -255,15 +256,15 @@ This section will give you the step-by-step guide to how to start up the acqusit
- __Save the Mesytec settings__:
Go to the MesytecControl. If you are inside a module, exit this with `x`. Now enter `s` and follow the commands.
- __Save the Mesytec settings__:
- __Load the Mesytec settings__:
Go to the MesytecControl. If you are inside a module, exit this with `x`. Now enter `l` and follow the commands.
- __Add a Mesytec settings file to DAQC__:
In the __DAQC__ setttings file `settings.yml`, you will find a section called `mesytec`, with a tag called `file`. Change this to point to the location of the settings file you want to use for reference. Restart __DAQC__.
@@ -274,7 +275,7 @@ This section will give you the step-by-step guide to how to start up the acqusit
```
#### Troubleshooting
Since the DAQ consists of so many moving parts, a lot can go wrong. However, a lot of these moving parts are non-essential, i.e. data can still be taken. The readout itself is very, very stable, i.e the two __RIO__'s and the event builder on __Bumblebee__. We have yet to see a proper crash of these.
Since the DAQ consists of so many moving parts, a lot can go wrong. However, a lot of these moving parts are non-essential, i.e. data can still be taken. The readout itself is very, very stable, i.e the __Mot__ and the event builder on __Bumblebee__. We have yet to see a proper crash of these.
What is much more likely to cause problems, and probably will during every experiment is __Go4__. Therefore, this should be the first suspicion.
