Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

78 Commits

Folders and files

Repository files navigation

Heap

Heap: (noun) a disorderly collection of objects placed haphazardly on top of each other, or (verb) put objects or loose substances on top of each other.

In this case: loosely organized collection of probabilistic annotations for transcripts and proteins, to help figure out what these transcripts and proteins do.

Setup

HMM

Install hmmer3 package: e.g. on MacOS run brew install hmmer

Download the following profiles

  • KEGG KO profile HMMs: https://www.genome.jp/ftp/db/kofam/profiles.tar.gz

    • Then concatenate all the profiles together: cat profiles/*.hmm > ko.hmm
      • Remove all entries that are RNA or small RNAs: make a list of KOs to include, then use hmmfetch -f to create a new .hmm file
      • Run hmmfetch --index ko.hmm
      • Run hmmpress ko.hmm
      • If you run the above two out of order, may need to mv ko.hmm.h3m.ssi ko.hmm.ssi
    • Also create ko_thresholds.tsv from KEGG FTP site https://www.genome.jp/ftp/db/kofam/ko_list.gz
      • Remove RNA entries
      • For rows withput a threshold, set threshold value to "-1", from "-"
      • Change "knum" column header to "model"
  • Pfam HMM profiles: https://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/Pfam-A.hmm.gz

    • Run hmmpress Pfam-A.hmm

Put these files in the same directory then set the following environment variable

HMM_DB_DIR=</host/hmm_dir>

FoldSeek

docker pull ghcr.io/steineggerlab/foldseek:latest

Then download databases to a host directory mounted onto the container

docker run -v /host/db_dir:/app --rm \
  ghcr.io/steineggerlab/foldseek databases \
  Alphafold/Swiss-Prot /app/afdb-swissprot /tmp

docker run -v /host/db_dir:/app --rm \
  ghcr.io/steineggerlab/foldseek databases \
  PDB /app/pdb /tmp

docker run -v /host/db_dir:/app --rm \
  ghcr.io/steineggerlab/foldseek databases \
  ProstT5 /app/prost-t5-weights /tmp

Then, to use foldseek scripts, set the following environment variables

FOLDSEEK_DB_DIR=</host/db_dir>

Hosting Docker Images on Google Cloud

This repo

docker build --platform linux/amd64 -t us-east1-docker.pkg.dev/needle-489321/tangle-docker/heap:latest .
docker push us-east1-docker.pkg.dev/needle-489321/tangle-docker/heap:latest

Foldseek image

(Note you may need to cleanup existing non-amd64 versions of the foldseem image first, using docker rmi)

docker pull --platform linux/amd64 ghcr.io/steineggerlab/foldseek:latest
docker tag ghcr.io/steineggerlab/foldseek:latest us-east1-docker.pkg.dev/needle-489321/tangle-docker/foldseek:amd64
docker push us-east1-docker.pkg.dev/needle-489321/tangle-docker/foldseek:amd64

And, to use these images on Google Cloud, make sure everything under FOLDSEEK_DB_DIR or HMM_DB_DIR are synced to Google Cloud storage, into a bucket.

HMM Searching

Pfam

Following script can be used to search for Pfam domains

PYTHONPATH=. python3 scripts/hmmscan-pfam.py \
  --query-database-name exp-doi:10.1126_sciadv.aba2498 \
  query.faa test_result.tsv

The query database name should uniquely identify the source of the query accessions in the query fasta file.

This script uses hmmscan with GA cutoff scores as the reporting threshold, so reported domains are likely correct.

To prepare and submit this job to run on Google Cloud, use the following script to create a run directory under runs (or whatever value for --run-dir), and then follow instructions in the README file in that run directory.

PYTHPATH=. python3 gcloud/hmmscan-pfam/setup.py \
  --query-database-name exp-doi:10.1126_sciadv.aba2498.SRR9331959_algae_denovo \
  --run-dir=runs \
  proteins.faa.gz

KO

Searching

You can search against KO HMM profile similarly,

PYTHONPATH=. python3 scripts/hmmscan-ko.py \
  --query-database-name exp-doi:10.1126_sciadv.aba2498 \
  query.faa test_result_unassigned.tsv

To prepare and submit this job to run on Google Cloud, use the following script to create a run directory under runs (or whatever value for --run-dir), and then follow instructions in the README file in that run directory.

PYTHPATH=. python3 gcloud/hmmscan-ko/setup.py \
  --query-database-name exp-doi:10.1126_sciadv.aba2498.SRR9331959_algae_denovo \
  --run-dir=runs \
  proteins.faa.gz

Assignment

But because KO HMM profiles do not use GA scores, but rather a separate threshold file, a second script is needed to make assignments.

PYTHONPATH=. python3 scripts/ko-assign.py \
  test_result_unassigned.tsv test_result_assigned.tsv

Use the --scoring-ratio-min option to change the min ratio of bitscore over threshold used to filter the results. Default is 0.99. If you change to 0.7, for example, the resulting file may include putative matches worth investigating.

Other HMM DBs

Use scripts/hmmscan.py and scripts/ko-assign (with --threshold-file option) to perform scans with other databases.

python3 scripts/hmmscan.py \
  --query-database-name _ \
  --target-database-name sieve_nfkb_ikb \
  nfkb_ikb.hmm needle_proteins.faa needle_nfkb_ikb.tsv

python3 scripts/ko-assign.py \
  --threshold-file nfkb_ikb_thresholds_low.csv \
  needle_nfkb_ikb_assigned.tsv \
  needle_nfkb_ikb.tsv

The threshold file needs to have the following headers

model,threshold,score_type,profile_type,f_measure,nseq,nseq_used,alen,mlen,eff_nseq,re_pos,definition

FoldSeek Searching

Use the following script to search the SwissProt database

PYTHONPATH=. python3 scripts/foldseek-swissprot.py \
  --query-database-name exp-doi:10.1126_sciadv.aba2498 \
  query.faa test_result.tsv

The query database name should uniquely identify the source of the query accessions in the query fasta file.

From a .cif file, you can use the Docker image directly

docker run --rm -v /host/db-dir:/db -v /host/cif-file-dir:/app \
  ghcr.io/steineggerlab/foldseek easy-search \
  /app/fold_2026_03_12_14_08_model_0.cif /db/pdb /app/res.tsv /tmp \
  --format-output "query,target,prob,evalue,bits,fident,qstart,qend,tstart,tend"

To prepare and submit this job to run on Google Cloud, use the following script to create a run directory under runs (or whatever value for --run-dir), and then follow instructions in the README file in that run directory.

PYTHPATH=. python3 gcloud/foldseek-swissprot/setup.py \
  --query-database-name exp-doi:10.1126_sciadv.aba2498.SRR9331959_algae_denovo \
  --run-dir=runs \
  proteins.faa.gz

The output file from above needs to be converted to the standard TSV format, use the scripts/foldseek-swissprot.py script with --input-from-foldseek option, e.g.

PYTHONPATH=. python3 scripts/foldseek-swissprot.py \
  --query-database-name exp-doi:10.1126_sciadv.aba2498 \
  --input-from-foldseek \
  sequence_fs_raw.tsv sequence_fs.tsv

Using foldseek to create a 3Di database, with Prost-T5

docker run --rm --platform linux/amd64 \
  -v <hmm_dir>:/hmm \
  -v <foldseek_dir>:/foldseek \
  -v <3di_output_dir>:/output \
  ghcr.io/steineggerlab/foldseek createdb \
  /hmm/ko.consensus.faa /output/ko.3di \
  --prostt5-model /foldseek/prost-t5-weights --threads 6

# two steps to dump db to fasta file of 3Di sequences

docker run --rm --platform linux/amd64 \
  -v <3di_output_dir>:/output \
  ghcr.io/steineggerlab/foldseek \
  lndb /output/ko.3di_h /output/ko.3di_ss_h

docker run --rm --platform linux/amd64 \
  -v <3di_output_dir>:/output \
  ghcr.io/steineggerlab/foldseek \
  convert2fasta /output/ko.3di_ss /output/ko.3di.faa

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages