Newer
Older
This is mostly a copy and pasting error, especially when working with Microsoft
Word and other "editors".
Qsub jobs need to be in ascii format and cannot handle UTF-8 special characters.
To test if your jobfile/script contains non-ascii characters you can use the ``file`` command.
Converting can be done with your editor of choice or with the ``iconv`` program.
Example:
.. code-block:: bash
root@master:$ file MRtrix3_TARDIS_0.3.sh
MRtrix3_TARDIS_0.3.sh: Bourne-Again shell script, UTF-8 Unicode text executable
# ASCII, will probably work
root@master:$ file MRtrix3_preproc.sh
MRtrix3_preproc.sh: Bourne-A gain shell script, ASCII text executable
# convert from utf-8 to ascii will never work, but tell you the position of
# the non-ascii character
root@master:$ iconv -f utf-8 -t ascii MRtrix3_TARDIS_0.3.sh
[...]
#PBS -j oe
#PBS iconv: illegal input sequence at position 3811
As you can see in the example above the character after ``#PBS`` following the
line ``#PBS -j oe`` could not be parsed to UTF-8. In this case the hyphen
character ``-`` was a UTF-8 special character instead. Just delete it and
replace the character with an actual hyphen and your script should
automatically become an ascii file.
heudiconv: UnicodeDecodeError
-----------------------------
When running Heudiconv, macOS users sometimes notice an error similar to this:
.. code-block:: python
File "/src/heudiconv/heudiconv/bids.py", line 77, in populate_bids_templates
populate_aggregated_jsons(path)
File "/src/heudiconv/heudiconv/bids.py", line 109, in populate_aggregated_jsons
json_ = load_json(fpath)
File "/src/heudiconv/heudiconv/utils.py", line 182, in load_json
data = json.load(fp)
File "/opt/miniconda-latest/lib/python3.6/json/__init__.py", line 296, in load
return loads(fp.read(),
File "/opt/miniconda-latest/lib/python3.6/codecs.py", line 321, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb0 in position 37: invalid start byte
Usually this indicates that Heudiconv tried to load json files that are in fact
`._` binary files created by `Apple's Finder
<https://apple.stackexchange.com/a/14981>`_. It seems that there is no way to
avoid their creation and you will have to clean them manually. Other
workarounds are setting your BIDS folder read-only before loading them in Finder or
avoiding Finder altogether.
To clean them try running **either**: ``dot_clean -m bids_folder`` on your Mac or
``find bids_folder -name '._*' -delete`` on a Linux/macOS.