faq.rst 2.87 KB
Newer Older
1
F.A.Q.
Michael Krause's avatar
Michael Krause committed
2
======
Michael Krause's avatar
Michael Krause committed
3
4
5


qsub:  file must be an ascii script
Michael Krause's avatar
Michael Krause committed
6
-----------------------------------
Michael Krause's avatar
Michael Krause committed
7
8
9
10

This is mostly a copy and pasting error, especially when working with Microsoft
Word and other "editors".
Qsub jobs need to be in ascii format and cannot handle UTF-8 special characters.
Michael Krause's avatar
Michael Krause committed
11
To test if your jobfile/script contains non-ascii characters you can use the ``file`` command.
Michael Krause's avatar
Michael Krause committed
12
13
14
15
16
17
Converting can be done with your editor of choice or with the ``iconv`` program.

Example:

.. code-block:: bash

Michael Krause's avatar
Michael Krause committed
18
   # UTF-8, will probably produce a faulty job file
Michael Krause's avatar
Michael Krause committed
19
20
21
22
23
24
25
   root@master:$ file MRtrix3_TARDIS_0.3.sh
   MRtrix3_TARDIS_0.3.sh: Bourne-Again shell script, UTF-8 Unicode text executable

   # ASCII, will probably work
   root@master:$ file MRtrix3_preproc.sh
   MRtrix3_preproc.sh: Bourne-A gain shell script, ASCII text executable

Michael Krause's avatar
Michael Krause committed
26
27
   # convert from utf-8 to ascii will never work, but tell you the position of
   # the non-ascii character
Michael Krause's avatar
Michael Krause committed
28
29
30
31
32
33
34
35
   root@master:$ iconv -f utf-8 -t ascii MRtrix3_TARDIS_0.3.sh
   [...]
   #PBS -j oe
   #PBS iconv: illegal input sequence at position 3811


As you can see in the example above the character after ``#PBS`` following the
line ``#PBS -j oe`` could not be parsed to UTF-8. In this case the hyphen
Michael Krause's avatar
Michael Krause committed
36
37
38
character ``-`` was a UTF-8 special character instead. Just delete it and
replace the character with an actual hyphen and your script should
automatically become an ascii file.
Michael Krause's avatar
ci test    
Michael Krause committed
39

Michael Krause's avatar
Michael Krause committed
40

Michael Krause's avatar
Michael Krause committed
41
42
heudiconv: UnicodeDecodeError
-----------------------------
Michael Krause's avatar
Michael Krause committed
43

44
When running Heudiconv, macOS users sometimes notice an error similar to this:
Michael Krause's avatar
Michael Krause committed
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65

.. code-block:: python

      File "/src/heudiconv/heudiconv/bids.py", line 77, in populate_bids_templates
        populate_aggregated_jsons(path)
      File "/src/heudiconv/heudiconv/bids.py", line 109, in populate_aggregated_jsons
        json_ = load_json(fpath)
      File "/src/heudiconv/heudiconv/utils.py", line 182, in load_json
        data = json.load(fp)
      File "/opt/miniconda-latest/lib/python3.6/json/__init__.py", line 296, in load
        return loads(fp.read(),
      File "/opt/miniconda-latest/lib/python3.6/codecs.py", line 321, in decode
        (result, consumed) = self._buffer_decode(data, self.errors, final)
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb0 in position 37: invalid start byte

Usually this indicates that Heudiconv tried to load json files that are in fact
`._` binary files created by `Apple's Finder
<https://apple.stackexchange.com/a/14981>`_. It seems that there is no way to
avoid their creation and you will have to clean them manually. Other
workarounds are setting your BIDS folder read-only before loading them in Finder or
avoiding Finder altogether.
66

Michael Krause's avatar
Michael Krause committed
67
68
69
70
To clean them try running **either**: ``dot_clean -m bids_folder`` on your Mac
or ``find bids_folder -name '._*' -delete`` on a Linux/macOS. Be careful with
the second approach and maybe list files first without the `-delete` switch to
make sure you do not delete anything important.