Merge branch 'dev' into migrate-to-main

.github/RELEASE_CHECKLIST.md

@@ -9,15 +9,15 @@
 7. Create a PR from `dev` to `main`
 8. Make sure all CI tests are passing again (additional tests are run on PRs to `main`)
 9. Request review (2 approvals required)
-10. Run `rich-codex` to regenerate docs screengrabs (actions `workflow_dispatch` button)
-11. Merge the PR into `main`
-12. Wait for CI tests on the commit to passed
-13. (Optional but a good idea) Run a manual sync on `nf-core/testpipeline` and check that CI is passing on the resulting PR.
-14. Create a new release copying the `CHANGELOG` for that release into the description section.
+10. Merge the PR into `main`
+11. Wait for CI tests on the commit to passed
+12. (Optional but a good idea) Run a manual sync on `nf-core/testpipeline` and check that CI is passing on the resulting PR.
+13. Create a new release copying the `CHANGELOG` for that release into the description section.
 
 ## After release
 
 1. Check the automated template synchronisation has been triggered properly. This should automatically open PRs directly to individual pipeline repos with the appropriate changes to update the pipeline template.
 2. Check that the automatic `PyPi` deployment has worked: [pypi.org/project/nf-core](https://pypi.org/project/nf-core/)
 3. Check `BioConda` has an automated PR to bump the version, and merge. eg. [bioconda/bioconda-recipes #20065](https://github.com/bioconda/bioconda-recipes/pull/20065)
 4. Create a tools PR to `dev` to bump back to the next development version in `CHANGELOG.md` and `setup.py` and change the gitpod container to `nfcore/gitpod:dev`.
+5. Run `rich-codex` on the [tools/website repo](https://github.com/nf-core/website/actions/workflows/rich-codex.yml) to regenerate docs screengrabs (actions `workflow_dispatch` button)

.github/workflows/create-test-wf.yml

@@ -71,10 +71,6 @@ jobs:
           mkdir create-test-wf && cd create-test-wf
           export NXF_WORK=$(pwd)
           nf-core --log-file log.txt pipelines create -n testpipeline -d "This pipeline is for testing" -a "Testing McTestface"
-          # echo current directory
-          pwd
-          # echo content of current directory
-          ls -la
           nextflow run nf-core-testpipeline -profile self_hosted_runner,test --outdir ./results
 
       - name: Upload log file artifact

.github/workflows/sync.yml

@@ -19,6 +19,10 @@ on:
         description: "Force a PR to be created"
         type: boolean
         default: false
+      pipeline:
+        description: "Pipeline to sync"
+        type: string
+        default: "all"
 
 # Cancel if a newer run is started
 concurrency:
@@ -35,6 +39,14 @@ jobs:
         run: |
           if [ "${{ github.event.inputs.testpipeline }}" == "true" ]; then
             echo '{"pipeline":["testpipeline"]}' > pipeline_names.json
+          elif [ "${{ github.event.inputs.pipeline }}" != "all" ]; then
+            curl -O https://nf-co.re/pipeline_names.json
+            # check if the pipeline exists
+            if ! grep -q "\"${{ github.event.inputs.pipeline }}\"" pipeline_names.json; then
+              echo "Pipeline ${{ github.event.inputs.pipeline }} does not exist"
+              exit 1
+            fi
+            echo '{"pipeline":["${{ github.event.inputs.pipeline }}"]}' > pipeline_names.json
           else
             curl -O https://nf-co.re/pipeline_names.json
           fi
@@ -82,7 +94,7 @@ jobs:
         run: |
           git config --global user.email "[email protected]"
           git config --global user.name "nf-core-bot"
-          nf-core --log-file sync_log_${{ matrix.pipeline }}.txt sync -d nf-core/${{ matrix.pipeline }} \
+          nf-core --log-file sync_log_${{ matrix.pipeline }}.txt pipelines sync -d nf-core/${{ matrix.pipeline }} \
             --from-branch dev \
             --pull-request \
             --username nf-core-bot \

.gitpod.yml

@@ -1,4 +1,4 @@
-image: nfcore/gitpod:dev
+image: nfcore/gitpod:latest
 tasks:
   - name: install current state of nf-core/tools and setup pre-commit
     command: |

CHANGELOG.md

@@ -1,6 +1,6 @@
 # nf-core/tools: Changelog
 
-## v3.0.0dev
+## [v3.0.0 - Titanium Tapir](https://github.com/nf-core/tools/releases/tag/3.0.0) - [2024-10-08]
 
 **Highlights**
 
@@ -9,7 +9,6 @@
 - A new Text User Interface app when running `nf-core pipelines create` to help us guide you through the process better (no worries, you can still use the cli if you give all values as parameters)
 - We replaced nf-validation with nf-schema in the pipeline template
 - CI tests now lint with the nf-core tools version matching the template version of the pipeline, to minimise errors in opened PRs with every new tools release.
-- New command `nf-core pipelines ro-crate` to create a [Research Object (RO) crate](https://www.researchobject.org/ro-crate/) for a pipeline
 - `nf-core licences` command is deprecated.
 - Changed default branch to `main`.
 - The structure of nf-core/tools pytests has been updated.
@@ -31,12 +30,12 @@
 - add option to exclude changelog from custom pipeline template ([#3104](https://github.com/nf-core/tools/pull/3104))
 - add option to exclude license from pipeline template ([#3125](https://github.com/nf-core/tools/pull/3125))
 - add option to exclude email from pipeline template ([#3126](https://github.com/nf-core/tools/pull/3126))
-- Use nf-schema instead of nf-validation ([#3116](https://github.com/nf-core/tools/pull/3116))
 - add option to exclude nf-schema from the template ([#3116](https://github.com/nf-core/tools/pull/3116))
 - add option to exclude fastqc from pipeline template ([#3129](https://github.com/nf-core/tools/pull/3129))
 - add option to exclude documentation from pipeline template ([#3130](https://github.com/nf-core/tools/pull/3130))
 - add option to exclude test configs from pipeline template ([#3133](https://github.com/nf-core/tools/pull/3133))
 - add option to exclude tower.yml from pipeline template ([#3134](https://github.com/nf-core/tools/pull/3134))
+- Use nf-schema instead of nf-validation ([#3116](https://github.com/nf-core/tools/pull/3116))
 - test pipeline with conda and singularity on PRs to master ([#3149](https://github.com/nf-core/tools/pull/3149))
 - run nf-core lint `--release` on PRs to master ([#3148](https://github.com/nf-core/tools/pull/3148))
 - Add tests to ensure all files are part of a template customisation group and all groups are tested ([#3099](https://github.com/nf-core/tools/pull/3099))
@@ -62,16 +61,16 @@
 
 ### Pipeline create command
 
-- Create: allow more special characters on the pipeline name for non-nf-core pipelines ([#3008](https://github.com/nf-core/tools/pull/3008))
-- Create: Mock git cretentials to generate stable textual snapshots ([#3007](https://github.com/nf-core/tools/pull/3007))
-- Create app: display input textbox with equally spaced grid ([#3038](https://github.com/nf-core/tools/pull/3038))
-- Pipelines: allow numbers in custom pipeline name ([#3094](https://github.com/nf-core/tools/pull/3094))
+- Allow more special characters on the pipeline name for non-nf-core pipelines ([#3008](https://github.com/nf-core/tools/pull/3008))
+- Mock git cretentials to generate stable textual snapshots ([#3007](https://github.com/nf-core/tools/pull/3007))
+- Display input textbox with equally spaced grid ([#3038](https://github.com/nf-core/tools/pull/3038))
+- Allow numbers in custom pipeline name ([#3094](https://github.com/nf-core/tools/pull/3094))
 
 ### Components
 
 - The `modules_nfcore` tag in the `main.nf.test` file of modules/subworkflows now displays the organization name in custom modules repositories ([#3005](https://github.com/nf-core/tools/pull/3005))
 - Add `--migrate_pytest` option to `nf-core <modules|subworkflows> test` command ([#3085](https://github.com/nf-core/tools/pull/3085))
-- Components: allow spaces at the beginning of include statements ([#3115](https://github.com/nf-core/tools/pull/3115))
+- Allow spaces at the beginning of include statements ([#3115](https://github.com/nf-core/tools/pull/3115))
 - Add option `--fix` to update the `meta.yml` file of subworkflows ([#3077](https://github.com/nf-core/tools/pull/3077))
 
 ### Download
@@ -89,8 +88,8 @@
 
 - Change default branch to `main` for the nf-core/tools repository
 - Update output of generation script for API docs to new structure ([#2988](https://github.com/nf-core/tools/pull/2988))
+- Remove `rich-codex.yml` action, images are now generated on the website repo ([#2989](https://github.com/nf-core/tools/pull/2989))
 - Add no clobber and put bash options on their own line ([#2991](https://github.com/nf-core/tools/pull/2991))
-- update minimal textual version and snapshots ([#2998](https://github.com/nf-core/tools/pull/2998))
 - move pipeline subcommands for v3.0 ([#2983](https://github.com/nf-core/tools/pull/2983))
 - return directory if base_dir is the root directory ([#3003](https://github.com/nf-core/tools/pull/3003))
 - Remove nf-core licences command ([#3012](https://github.com/nf-core/tools/pull/3012))
@@ -114,6 +113,7 @@
 - Update python:3.12-slim Docker digest to 59c7332 ([#3124](https://github.com/nf-core/tools/pull/3124))
 - Update pre-commit hook pre-commit/mirrors-mypy to v1.11.1 ([#3091](https://github.com/nf-core/tools/pull/3091))
 - Update to pytest v8 and move it to dev dependencies ([#3058](https://github.com/nf-core/tools/pull/3058))
+- Update minimal textual version and snapshots ([#2998](https://github.com/nf-core/tools/pull/2998))
 
 ## [v2.14.1 - Tantalum Toad - Patch](https://github.com/nf-core/tools/releases/tag/2.14.1) - [2024-05-09]
 

README.md

@@ -21,7 +21,7 @@ For documentation of the internal Python functions, please refer to the [Tools P
 
 ## Installation
 
-For full installation instructions, please see the [nf-core documentation](https://nf-co.re/docs/usage/tools).
+For full installation instructions, please see the [nf-core documentation](https://nf-co.re/docs/nf-core-tools/installation).
 Below is a quick-start for those who know what they're doing:
 
 ### Bioconda

nf_core/components/list.py

@@ -25,7 +25,7 @@ def __init__(
         self.remote = remote
         super().__init__(component_type, pipeline_dir, remote_url, branch, no_pull)
 
-    def _configure_repo_and_paths(self, nf_dir_req=True) -> None:
+    def _configure_repo_and_paths(self, nf_dir_req: bool = True) -> None:
         """
         Override the default with nf_dir_req set to False to allow
         info to be run from anywhere and still return remote info

nf_core/components/nfcore_component.py

@@ -5,7 +5,7 @@
 import logging
 import re
 from pathlib import Path
-from typing import Any, List, Optional, Tuple, Union
+from typing import Any, Dict, List, Optional, Tuple, Union
 
 log = logging.getLogger(__name__)
 
@@ -50,7 +50,7 @@ def __init__(
         self.passed: List[Tuple[str, str, Path]] = []
         self.warned: List[Tuple[str, str, Path]] = []
         self.failed: List[Tuple[str, str, Path]] = []
-        self.inputs: List[list[dict[str, dict[str, str]]]] = []
+        self.inputs: List[List[Dict[str, Dict[str, str]]]] = []
         self.outputs: List[str] = []
         self.has_meta: bool = False
         self.git_sha: Optional[str] = None

nf_core/modules/modules_differ.py

@@ -391,8 +391,8 @@ def get_new_and_old_lines(patch):
     def try_apply_single_patch(file_lines, patch, reverse=False):
         """
         Tries to apply a patch to a modified file. Since the line numbers in
-        the patch does not agree if the file is modified, the old and new
-        lines inpatch are reconstructed and then we look for the old lines
+        the patch do not agree if the file is modified, the old and new
+        lines in the patch are reconstructed and then we look for the old lines
         in the modified file. If all hunk in the patch are found in the new file
         it is updated with the new lines from the patch file.
 

nf_core/pipeline-template/.github/CONTRIBUTING.md

@@ -37,7 +37,7 @@ You have the option to test your changes locally by running the pipeline. For re
 nf-test test --profile debug,test,docker --verbose
 ```
 
-{% endif %}
+{% endif -%}
 When you create a pull request with changes, [GitHub Actions](https://github.com/features/actions) will run automatic tests.
 Typically, pull-requests are only fully reviewed when these tests are passing, though of course we can help out before then.
 
@@ -95,7 +95,7 @@ If you wish to contribute a new step, please use the following coding standards:
    {%- if multiqc %}
 9. Update MultiQC config `assets/multiqc_config.yml` so relevant suffixes, file name clean up and module plots are in the appropriate order. If applicable, add a [MultiQC](https://https://multiqc.info/) module.
 10. Add a description of the output files and if relevant any appropriate images from the MultiQC report to `docs/output.md`.
-    {% endif %}
+    {%- endif %}
 
 ### Default values
 

nf_core/pipeline-template/assets/multiqc_config.yml

@@ -3,11 +3,11 @@ report_comment: >
   This report has been generated by the <a href="https://github.com/{{ name }}/tree/dev" target="_blank">{{ name }}</a>
   analysis pipeline.{% if is_nfcore %} For information about how to interpret these results, please see the
   <a href="https://nf-co.re/{{ short_name }}/dev/docs/output" target="_blank">documentation</a>.{% endif %}
-  {%- else %}
+  {%- else -%}
   This report has been generated by the <a href="https://github.com/{{ name }}/releases/tag/{{ version }}" target="_blank">{{ name }}</a>
   analysis pipeline.{% if is_nfcore %} For information about how to interpret these results, please see the
   <a href="https://nf-co.re/{{ short_name }}/{{ version }}/docs/output" target="_blank">documentation</a>.{% endif %}
-  {% endif %}
+  {%- endif %}
 report_section_order:
   "{{ name_noslash }}-methods-description":
     order: -1000

nf_core/pipeline-template/docs/output.md

@@ -58,7 +58,8 @@ Results generated by MultiQC collate pipeline QC from supported tools e.g. FastQ
 
 - `pipeline_info/`
   - Reports generated by Nextflow: `execution_report.html`, `execution_timeline.html`, `execution_trace.txt` and `pipeline_dag.dot`/`pipeline_dag.svg`.
-    {% if email %}- Reports generated by the pipeline: `pipeline_report.html`, `pipeline_report.txt` and `software_versions.yml`. The `pipeline_report*` files will only be present if the `--email` / `--email_on_fail` parameter's are used when running the pipeline. {% endif %}
+    {%- if email %}
+  - Reports generated by the pipeline: `pipeline_report.html`, `pipeline_report.txt` and `software_versions.yml`. The `pipeline_report*` files will only be present if the `--email` / `--email_on_fail` parameter's are used when running the pipeline. {% endif %}
   - Reformatted samplesheet files used as input to the pipeline: `samplesheet.valid.csv`.
   - Parameters used by the pipeline run: `params.json`.
 

requirements-dev.txt

@@ -6,7 +6,7 @@ responses
 ruff
 Sphinx
 sphinx-rtd-theme
-textual-dev==1.5.1
+textual-dev==1.6.1
 types-PyYAML
 types-requests
 types-jsonschema

setup.py

@@ -2,7 +2,7 @@
 
 from setuptools import find_packages, setup
 
-version = "3.0.0dev"
+version = "3.0.0"
 
 with open("README.md") as f:
     readme = f.read()