The Styx Book

Welcome! This book is aimed at

• new users wanting to use Styx for the first time
• advanced users wanting to create custom ways to deploy Styx

Getting started

To get started install the NiWrap Python package. Let's also install the Docker integration so you don't have to worry about installing any software dependencies.

pip install niwrap styxdocker

Running commands is then as easy as calling the method from the appropriate module. For example, to call FSL bet:

from niwrap import fsl

bet_output = fsl.bet(
    infile="my_file.nii.gz",
    binary_mask=True,
)

This runs the command

bet my_file.nii.gz -m

and stores all available output files for easy access in bet_output.

from styxdefs import set_global_runner
from styxdocker import DockerRunner

set_global_runner(DockerRunner())

These can then be used as an input to another Styx wrapper or with any other Python package like nilearn:

from nilearn.plotting import plot_anat

plot_anat(bet_output.outfile)

The next chapter will explain how to use Runners to control how the commands get executed and intermediate files get stored.

Runners

Runners define how commands will be executed and how output files will be stored.

By default they will be executed with the system shell and outputs will be stored in a folder named styx_temp/ in the current working directory.

While this provides a good start, users may want more control where the data gets store or might not have all the software dependencies installed. The first step before packaging and deploying a pipeline should be to modify this behavior.

Official Runners

There are a of number official runners:

• styxdefs.LocalRunner - This is the default Runner. It executes commands locally using the system shell.
• styxdefs.DummyRunner - This Runner dry-runs commands, useful when writing new wrappers to ensure commands are as expected.
• styxdocker.DockerRunner - This Runner executes commands in a Docker container.
• styxsingularity.SingularityRunner - This Runner executes commands in an Apptainer/Singularity container.
• styxgraph.GraphRunner - This is a special Runner, capturing information about how commands are connected, returning a diagram.

Setting up a Runner

If you for example want to change where the LocalRunner stores data, we create a new instance of it and set it to be used globally:

from styxdefs import set_global_runner, LocalRunner

my_runner = LocalRunner()
my_runner.data_dir = "/some/folder"
set_global_runner(my_runner)

# Now you can use any Styx functions as usual

The same method can be used to set up other Runners:

from styxdefs import set_global_runner
from styxdocker import DockerRunner

my_runner = DockerRunner()
set_global_runner(my_runner)

# Now you can use any Styx functions as usual

Alternatively, if a specific function should be executed with a different Runner without modifying the global Runner, we can pass it as an argument to the wrapped command:

my_other_runner = DockerRunner()

fsl.bet(
    infile="my_file.nii.gz",
    runner=my_other_runner,
)

# Now you can use any Styx functions as usual

Middleware Runners

Middleware Runners are special runners that can be used on top of other runners. Currently, the GraphRunner, which creates a diagram by capturing how commands are connected, is the only official runner:

from styxdefs import set_global_runner, get_global_runner
from styxgraph import GraphRunner

my_runner = DockerRunner()
set_global_runner(GraphRunner(my_runner))  # Use GraphRunner middleware

# Use any Styx functions as usual
# ...

print(get_global_runner().mermaid())  # Print mermaid diagram

I/O: Runner file handling

TODO:

• Explain output tuple, root attribute.

Default runner behaviour

While custom Styx runners allow users to implement file I/O handling any way they want, the 'default' runners (LocalRunner as well as DockerRunner and SingularityRunner) work very similarly by default.

They create a directory (default path styx_temp/ in the current working directory) in which a folder gets created for every Styx function call. These will look like this:

79474bd248c4b2f1_5_bet/
^^^^^^^^^^^^^^^^ ^ ^^^
|                | |
|                | `-- interface name (FSL BET)
|                `---- number of execution (5th)
`--------------------- unique random hash

Every time you create a new runner a new random hash will be generated. This ensures paths created are unique and avoid conflict with previous executions of your pipeline. The hash is followed by a counting number which chronologically will count up by one for every Styx function call. This makes the folder sortable. Lastly they contain the Styx function name to be human readable.

Tips & best practices

Managing runner output

You may only want to keep certain outputs generated from your workflow. One strategy is to set the runner's output directory to temporary storage, copying only what should be saved to a more permanent location.

import shutil
from styxdefs import set_global_runner, LocalRunner

my_runner = LocalRunner()
my_runner.data_dir = "/some/temp/folder"
set_global_runner(my_runner)

# Perform some task
# ...

shutil.copy2(task_output.out_files, "/some/permanent/output/folder")

# Remove temporary directory for cleanup
shutil.rmtree(runner.data_dir) 

Styx on HPC clusters

TODO:

• Explain singularityrunner, hint at putting styx_tmp on local scratch storage.

Advanced concepts

Writing custom runners

Contributing

Contributing to NiWrap

Automated descriptor creation in NiWrap

Some packages use shared internal data structures which store command line parsing information. By serializing these data structures, we can extract it and subsequently generate Boutiques descriptors from it which will always be correct and can be automatically updated whenever the packages change.

The rough steps to extract metadata are as follows:

1. Add JSON serialization code to the package.
2. Instead of emitting a help message, emit a JSON object.
3. Write a script to run all the commands and extract the JSON objects.
4. Write a script to generate Boutiques descriptors from the JSON objects.

Packages where this is viable:

• Connectome Workbench
• MRTrix3

Contributing to the Styx compiler

Examples

Short examples

• Anatomical preproc
• ?

Styx in the wild

• Non-human primate diffusion processing

Example: MRI anatomical preproc

The following is a toy implementation of a minimalistic MRI T1 anatomical preprocessing.

from niwrap import fsl
from styxdefs import set_global_runner
from styxdocker import DockerRunner
import os

def anatomical_preprocessing(input_file):
    # Step 1: Reorient to standard space
    reorient_output = fsl.fslreorient2std(
        input_image=input_file,
    )

    # Step 2: Robustly crop the image
    robustfov_output = fsl.robustfov(
        input_file=reorient_output.output_image,
    )

    # Step 3: Brain extraction
    bet_output = fsl.bet(
        infile=robustfov_output.output_roi_volume,
        fractional_intensity=0.5,  # Fractional intensity threshold
        robust_iters=True,
        binary_mask=True,
        approx_skull=True,
    )

    # Step 4: Tissue segmentation
    seg_output = fsl.fast(
        in_files=[bet_output.outfile],
        img_type=3  # 3 tissue classes
    )

    print("Anatomical preprocessing completed.")
    return bet_output, seg_output

if __name__ == "__main__":
    input_file = "path/to/your/input/T1w.nii.gz"
    output_dir = "my_output"

    # Set up the Docker runner
    set_global_runner(DockerRunner(data_dir=output_dir))

    # Create output directory if it doesn't exist
    os.makedirs(output_dir, exist_ok=True)

    # Run the anatomical preprocessing
    brain, segmentation = anatomical_preprocessing(input_file)

Full source.

Example: Generate mermaid graph

This example takes the previous example (anatomical preprocessing) and shows how the GraphRunner can be used to reconstruct an execution graph to then generate a mermaid graph.

from niwrap import fsl
from styxdefs import set_global_runner
from styxdocker import DockerRunner
from styxgraph import GraphRunner
import os

def anatomical_preprocessing(input_file):
    # Step 1: Reorient to standard space
    reorient_output = fsl.fslreorient2std(
        input_image=input_file,
    )

    # Step 2: Robustly crop the image
    robustfov_output = fsl.robustfov(
        input_file=reorient_output.output_image,
    )

    # Step 3: Brain extraction
    bet_output = fsl.bet(
        infile=robustfov_output.output_roi_volume,
        fractional_intensity=0.5,  # Fractional intensity threshold
        robust_iters=True,
        binary_mask=True,
        approx_skull=True,
    )

    # Step 4: Tissue segmentation
    seg_output = fsl.fast(
        in_files=[bet_output.outfile],
        img_type=3  # 3 tissue classes
    )

    print("Anatomical preprocessing completed.")
    return bet_output, seg_output

if __name__ == "__main__":
    input_file = r"C:\Users\floru\Downloads\T1.nii.gz"#"path/to/your/input/T1w.nii.gz"
    output_dir = "my_output"

    # Set up the Docker runner
    runner = DockerRunner(data_dir=output_dir)
    graph_runner = GraphRunner(base=runner)
    set_global_runner(graph_runner)

    # Create output directory if it doesn't exist
    os.makedirs(output_dir, exist_ok=True)

    # Run the anatomical preprocessing
    brain, segmentation = anatomical_preprocessing(input_file)

    print(graph_runner.node_graph_mermaid())

Full source.

graph TD
  fslreorient2std
  robustfov
  bet
  fast
  fslreorient2std --> robustfov
  robustfov --> bet
  bet --> fast

Example: Dynamic runners

A common pattern in processing pipelines with Styx is dynamically choosing what runner Styx should use. This allows the same pipeline to run e.g. both on your local machine for testing as well as on your HPC cluster.

from styxdefs import set_global_runner
from styxdocker import DockerRunner
from styxsingularity import SingularityRunner

runner_type = "docker"  # You could read this from a CLI argument,
                        # config file, or check what system you are
                        # running on.

if runner_type == "docker":
    print("Using docker runner.")
    runner = DockerRunner()

elif runner_type == "singularity":
    print("Using singularity runner.")
    runner = SingularityRunner()

else:
    print("Using local runner.")
    runner = LocalRunner()

set_global_runner(runner)

Full source.