Skip to content

Commit

Permalink
Merge pull request #412 from nextflow-io/dev
Browse files Browse the repository at this point in the history
Hello Nextflow refresh including new modules on containers, config, nf-core, and seqera
  • Loading branch information
mribeirodantas authored Oct 28, 2024
2 parents 0423150 + f01455d commit 9fe4c2f
Show file tree
Hide file tree
Showing 151 changed files with 11,949 additions and 2,500 deletions.
29 changes: 16 additions & 13 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -1,27 +1,30 @@
{
"name": "nfcore",
"image": "nfcore/gitpod:latest",
"image": "ghcr.io/nextflow-io/training:latest",
"remoteUser": "gitpod",

"features": {
"ghcr.io/devcontainers/features/docker-outside-of-docker:1": {}
},
"remoteEnv": {
"NXF_HOME": "/workspaces/.nextflow",
"HOST_PROJECT_PATH": "${localWorkspaceFolder}"
},
// Configure tool-specific properties.
"customizations": {
// Configure properties specific to VS Code.
"vscode": {
// Set *default* container specific settings.json values on container create.
"settings": {
"python.defaultInterpreterPath": "/opt/conda/bin/python",
"python.linting.enabled": true,
"python.linting.pylintEnabled": true,
"python.formatting.autopep8Path": "/opt/conda/bin/autopep8",
"python.formatting.yapfPath": "/opt/conda/bin/yapf",
"python.linting.flake8Path": "/opt/conda/bin/flake8",
"python.linting.pycodestylePath": "/opt/conda/bin/pycodestyle",
"python.linting.pydocstylePath": "/opt/conda/bin/pydocstyle",
"python.linting.pylintPath": "/opt/conda/bin/pylint"
"python.defaultInterpreterPath": "/opt/conda/bin/python"
},

// Add the IDs of extensions you want installed when the container is created.
"extensions": ["ms-python.python", "ms-python.vscode-pylance", "nf-core.nf-core-extensionpack"]
"extensions": [
"ms-python.python",
"ms-python.vscode-pylance",
"nf-core.nf-core-extensionpack",
"nextflow.nextflow",
"codezombiech.gitignore"
]
}
},
"portsAttributes": {
Expand Down
14 changes: 11 additions & 3 deletions .github/gitpod.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,17 @@ RUN apt-get update --quiet && \
curl \
tree \
graphviz \
software-properties-common

software-properties-common && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*

# Taken from: https://github.com/nf-core/tools/blob/master/nf_core/gitpod/gitpod.Dockerfile
# Install Apptainer (Singularity)
RUN add-apt-repository -y ppa:apptainer/ppa && \
apt-get update --quiet && \
apt install -y apptainer
apt install -y apptainer && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*

# Install Conda
RUN wget --quiet https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh && \
Expand All @@ -35,6 +38,11 @@ ENV PATH="/opt/conda/bin:$PATH"
RUN mkdir -p /workspace/data \
&& chown -R gitpod:gitpod /opt/conda /workspace/data

# Install Tower Agent
RUN curl -fSL https://github.com/seqeralabs/tower-agent/releases/latest/download/tw-agent-linux-x86_64 > tw-agent && \
chmod +x tw-agent && \
mv tw-agent /usr/local/bin/tw-agent

# Change user to gitpod
USER gitpod

Expand Down
10 changes: 10 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,13 @@ repos:
hooks:
- id: editorconfig-checker
alias: ec

- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0
hooks:
- id: trailing-whitespace
exclude_types:
- svg
- id: end-of-file-fixer
exclude_types:
- svg
148 changes: 70 additions & 78 deletions docs/hello_nextflow/01_orientation.md
Original file line number Diff line number Diff line change
@@ -1,103 +1,95 @@
# Orientation

The Gitpod environment contains some test data that will be used in this workshop. All software required are already installed and configured in it too.
The Gitpod environment contains all the software, code and data necessary to work through this training course, so you don't need to install anything yourself.
However, you do need a (free) account to log in, and you should take a few minutes to familiarize yourself with the interface.

!!! note
If you have not yet done so, please follow [this link](../../envsetup/) before going any further.

Follow [this link](../../envsetup/) if you have not yet setup your Gitpod environment.
## Materials provided

## Getting started
Throughout this training course, we'll be working in the `hello-nextflow/` directory, which loads by default when you open the Gitpod workspace.
This directory contains all the code files, test data and accessory files you will need.

You will complete this module in the `hello-nextflow/` folder.
Feel free to explore the contents of this directory; the easiest way to do so is to use the file explorer on the left hand side of the Gitpod workspace.
Alternatively, you can use the `tree` command.
Throughout the course, we use the output of `tree` to represent directory structure and contents in a readable form, sometimes with minor modifications for clarity.

Here we generate a table of contents to the second level down:

```bash
cd /workspace/gitpod/hello-nextflow
tree . -L 2
```

In this folder you will all test data, code and accessory needed to work through this training module.

!!! question "Exercise"
If you run this inside `hello-nextflow`, you should see the following output:

View all the folder and files in the `hello-nextflow` directory.

```console
tree .
```

You should see the following output:

```console title="Output"
/workspace/gitpod/hello-nextflow
```console title="Directory contents"
.
├── containers
│ ├── build
│ ├── data
│ ├── results
│ └── scripts
├── data
│ ├── bam
│ │ ├── reads_father.bam
│ │ ├── reads_mother.bam
│ │ └── reads_son.bam
│ ├── greetings.txt
│ ├── intervals.list
│ ├── ref.tar.gz
│ ├── greetings.csv
│ ├── ref
│ ├── sample_bams.txt
│ └── samplesheet.csv
├── hello-gatk.nf
├── hello-modules.nf
├── hello-nf-test.nf
├── hello-config
│ ├── demo-params.json
│ ├── main.nf
│ └── nextflow.config
├── hello-containers.nf
├── hello-genomics.nf
├── hello-modules
│ ├── demo-params.json
│ ├── main.nf
│ └── nextflow.config
├── hello-nf-test
│ ├── demo-params.json
│ ├── main.nf
│ └── nextflow.config
├── hello-operators.nf
├── hello-world.nf
├── nextflow.config
└── scripts
├── hello-gatk-1.nf
├── hello-gatk-2.nf
├── hello-gatk-3.nf
├── hello-gatk-4.nf
├── hello-gatk-5.nf
├── hello-gatk-6.nf
├── hello-modules-1.nf
├── hello-modules-2.nf
├── hello-modules-3.nf
├── hello-world-10.nf
├── hello-world-1.nf
├── hello-world-2.nf
├── hello-world-3.nf
├── hello-world-4.nf
├── hello-world-5.nf
├── hello-world-6.nf
├── hello-world-7.nf
├── hello-world-8.nf
├── hello-world-9.nf
├── modules
│ └── local
│ ├── gatk
│ │ ├── haplotypecaller
│ │ │ └── main.nf
│ │ └── jointgenotyping
│ │ ├── main.nf
│ │ └── tests
│ │ └── inputs
│ │ ├── family_trio_map.tsv
│ │ ├── reads_father.bam.g.vcf
│ │ ├── reads_father.bam.g.vcf.idx
│ │ ├── reads_mother.bam.g.vcf
│ │ ├── reads_mother.bam.g.vcf.idx
│ │ ├── reads_son.bam.g.vcf
│ │ └── reads_son.bam.g.vcf.idx
│ └── samtools
│ └── index
│ └── main.nf
└── nextflow.config

12 directories, 43 files

└── solutions
├── hello-config
├── hello-genomics
├── hello-modules
├── hello-nf-test
├── hello-operators
└── hello-world

18 directories, 17 files
```

Each file will be used in this training module.
!!!note

**The `data` directory** contains the input data we'll use in Part 2: Hello GATK, which uses an example from genomics to demonstrate how to build a simple analysis pipeline. The data is described in detail in that section of the training.
Don't worry if this seems like a lot; we'll go through the relevant pieces at each step of the course.
This is just meant to give you an overview.

**The `scripts` directory** contains the completed workflow scripts that result from each step of the tutorial and are intended to be used as a reference to check your work. The name and number in the filename correspond to the step of the relevant tutorial. For example, the file `hello-world-4.nf` is the expected result of completing steps 1 through 4 of Part 1: Hello World.
**Here's a summary of what you should know to get started:**

**The file `greetings.txt`** is a plain text file used to provide inputs in Part 1: Hello World.
- **The `.nf` files** are workflow scripts that are named based on what part of the course they're used in.

**The file `hello-gatk.nf`** is a stub that serves as a starting point to Part 2: Hello GATK. In its initial state, it is NOT a functional workflow script.
- **The `hello-*` directories** are directories used in the later Parts of the course where we are working with more than just one workflow file.

**The file `hello-world.nf`** is a simple but fully functional workflow script that serves as a starting point to Part 1: Hello World.
- **The file `nextflow.config`** is a configuration file that sets minimal environment properties.
You can ignore it for now.

- **The `data` directory** contains the input data we'll use in most of the course. The dataset is described in detail in Part 3, when we introduce it for the first time.

- **The `solutions` directory** contains the completed workflow scripts that result from each step of the course.
They are intended to be used as a reference to check your work and troubleshoot any issues.
The name and number in the filename correspond to the step of the relevant part of the course.
For example, the file `hello-world-4.nf` is the expected result of completing steps 1 through 4 of Part 1: Hello World.

!!!tip

If for whatever reason you move out of this directory, you can always run this command to return to it:

```bash
cd /workspace/gitpod/hello-nextflow
```

**The file `nextflow.config`** is a configuration file that sets minimal environment properties.
Now, to begin the course, click on the arrow in the bottom right corner of this page.
Loading

0 comments on commit 9fe4c2f

Please sign in to comment.