ASHS
====

There is a specially patched version of the original `ASHS`_ (Automatic
Segmentation of Hippocampal Subfields) tool available on the Tardis.

In most cases you are probably going to run this tool with a pre-installed
atlas and a set of a T1 and T2 Nifti images to start the automatic segmentation
pipeline. To do this you have to tell ASHS where its root folder is located by
setting a variable called ``ASHS_ROOT``. A number of environment modules will
help with that.

.. code-block:: bash

    $ module load ashs
    $ echo $ASHS_ROOT
    /opt/software/ashs/1.0.0-mpib0


**Note** that there are two major, incompatible versions available right now
(February 2018). The legacy version (0.1.x), and a new version (1.0.0), which the
authors also refer to as *fastashs*. Versions suffixed with *mpibN* contain a
set of patches developed at MPIB to work with Torque.


Atlases
-------

The atlases available need to match the ashs version to function properly.

Checkout the folder :file:`/opt/software/ashs/data` to see available atlases or
download your on. Right now there are:

.. code-block:: bash

   /opt/software/ashs/data/
   ├── 1.0
   │   ├── ashs_atlas_mpib_20180208
   │   └── ashs_atlas_upennpmc_20170810
   └── legacy
       ├── ashs_atlas_mpib_2016
       ├── ashs_atlas_paul_2012
       └── ashs_atlas_upennpmc_20140416


The data in *legacy* is supposed to work with the 0.1.x-branch of ASHS and the
atlases in *1.0* have been created with the newer branch.

Parallelization
---------------

The legacy branch of ASHS necessarily relied on qsub internally. Segmentation,
even for a single subject, was almost prohibitively slow to be run on a single
core. The patched version should distribute nicely over the cluster when using
the parameter `-Q` (see below for examples).

**However**, the newer ASHS branch is able to use multiple cores for each
subject. That means you now have the choice to either distribute a segmentation
across the whole cluster or submit a single, multi-threaded version. This
should be the preferred approach as it is much more robust and closer to the
original scripts.

The table below shows a comparison for a test segmentation run using the **1.0** branch:

=========  =========
cores      real time
=========  =========
800 (pbs)     20m
20  (par)     15m
8             23m
4             35m
2             55m
1            110m
=========  =========


Due to defensive inter-stage delays, the first approach (using cluster-wide
distribution with `-Q`) was even slower than the 20-core parallel version on a
single node using `-P`. Even single-threaded performance is at acceptable
speeds now. With recent workstations a batch segmentation for a small number of
subjects could be done overnight - without a cluster.


Segmentation
-------------

Given your files are already in place, a typical invocation for a single subject
would then be **either**:

Old approach
^^^^^^^^^^^^

.. code-block:: bash

   module load ashs
   $ASHS_ROOT/bin/ashs_main.sh \
       -a /opt/software/ashs/data/1.0/ashs_atlas_mpib_20180208\
       -g T1.nii -f T2.nii -w wdir -T -Q



This will run the ASHS pipeline in the foreground submitting jobs and waiting
for them to finish. Results will be put in the specified folder *wdir*. You can
use a different atlas folder by specifying a custom location after the ``-a``
switch. To conserve disk space please always use the tidy ``-T`` option. It
is also possible to run some stages selectively. Use the option ``-h`` to list
all parameters and a help text.

To run the program in the background and catch all the output so you needn't
keep the shell open use the ampersand (*&*) and shell redirection mechanism like this:

.. code-block:: bash

    $ASHS_ROOT/bin/ashs_main.sh [..opts..]    >ashs.log 2>&1 &

For a number of image pairs you could use a simple for loop.


.. code-block:: bash

    for id in 01 03 10 32 ; do
       $ASHS_ROOT/bin/ashs_main.sh -a /opt/software/ashs/data/1.0/ashs_atlas_mpib_20180208/ \
	                               -g T1_${id}.nii -f T2_${id}.nii -w wdir_${id} -T -Q \
								    >ashs_${id}.log 2>&1 &
    done

Be careful with looping over a large number (>30) of image pairs in this way as
some intermediate steps will generate a lot of jobs on its own. Also check
``qstat`` and the log file you specified to track ASHS' progress.


New approach
^^^^^^^^^^^^

Since ASHS version 1.0 this is the preferred way to submit segmentations for a number of subjects.

.. code-block:: bash

	sbatch ashs_job.jobs -c 8 --mem 10gb


with :file:`ashs_job.sh` containing:

.. code-block:: bash

    #!/bin/bash
    module load ashs
    export SUBJECT_ID=23
    $ASHS_ROOT/bin/ashs_main.sh \
        -a /opt/software/ashs/data/1.0/ashs_atlas_mpib_20180208\
        -g ${SUBJECT_ID}/T1.nii -f ${SUBJECT_ID}/T2.nii -w {SUBJECT_ID}_wdir -T -P

Note the difference in the last parameter here (-Q vs -P).
Looping over a number of subject IDs would look something like this.

.. code-block:: bash

    for id in 01 03 10 32 ; do
        export SUBJECT_ID=$id  # all exports are passed to SLURM
        sbatch ashs_job.sh -c 8 --mem 10gb
    done

Known Issues
------------

.. ATTENTION::
   This only refers to the old submission approach. When using fastashs and `-P`
   you can ignore this issue.


Due to the very large amount of intermediate jobs, script submission may become
unreliable when submitting more than 20 or 30 images at once.

Please consider either manually submitting smaller chunks of jobs or use a
script similar to the one below that will keep track of the current chunk
position.  Subsequent runs of this script will submit chunksize number of new
`ashs_main.sh` calls:


.. code-block:: bash

    #!/bin/bash

    # This script will submit ASHS jobs in small chunks. To submit a next chunk
    # just run this script again. The state is inferred from the existence of the
    # working directory only.

    # somewhat safe value for concurrent ashs run at the moment
    imagesperchunk=30

    # this will only run the first N images
    submitted=0
    for subject in $( ls -1 *nii) ; do
        if [ $submitted -ge $imagesperchunk ] ; then
            echo "Submitted $imagesperchunk images, done for now."
            exit 0
        fi

        name=${subject%%\.nii}
        wdir=workdir_${name}_dir
        if [ -d $wdir ] ; then
            echo "Skipping $subject, working directory exists."
            continue
        fi
        submitted=$(( submitted + 1 ))
        echo "Starting script for subject: $subject"
        ASHS_ROOT=/opt/ashs /opt/ashs/bin/ashs_main.sh \
          -a /opt/ashs/data/atlas_paul/ \
          -g T1/t1_${subject} \
          -f ${subject} \
          -w $wdir \
          -T \
          -Q >$subject.log 2>&1 &

        sleep 2s
    done

    echo "No images left, exiting."
    exit 0


Atlas Creation
--------------

We had some success creating an own atlas using manual segmentations following the official `Atlas Creation Guide <https://sites.google.com/site/hipposubfields/building-an-atlas>`_. Basically you need to prepare the following data:

1. 20 T1 and T2 images
2. Segmentations in T2 in Nifti format for both hemispheres
3. A label description file (ITKsnap) mapping the values in the segmentations to a name, color and some options like this::

      0      0    0    0       0  0  0    "Clear Label"
      11   102  205  170       1  1  1    "CA1L"
      12     0    0  128       1  1  1    "CA1R"
      ...

Let's assume your niftis are in a folder called niftis/ and the segmentations in a folder called Segmentations relative to the current directory then:

4. A data manifest file listing all the necessary files for each subject in this format::

     1102 niftis/1102/T1*gz niftis/1102/high*gz Segmentations/1102_*left.nii.gz Segmentations/1102_*right.nii.gz
     1105 niftis/1105/T1*gz niftis/1105/high*gz Segmentations/1105_*left.nii.gz Segmentations/1105_*right.nii.gz
     ...

Note that unless Torque support will be added to ashs natively, you need at least version *0.1.0-mpib1* from the legacy branch or *1.0.0-mpib0* from the fastashs branch.

With all the files in place you can run ``ashs_main.sh`` like this:


.. code-block:: bash

   module load ashs
   $ASHS_ROOT/bin/ashs_train.sh -D manifest.txt -L labels.txt -w atlas_wdir

.. _ASHS: https://sites.google.com/site/hipposubfields/
