Coverage

What has been measured, across every experiment in the tables, and what has not. Every × is a run still to do.

Data from 15 Oct 2026, commit 4a4b1171 (source)

measured, with the number of true instances scored × not run – does not apply: hmmscan searches no relative, and the shuffled-alphabet run is a control for kmerseek only

aligns amino-acid letters: phmmer, MMseqs2builds a profile of a protein family: jackhmmer, MMseqs2, three rounds, HH-suite (hhsearch)compares 3-D shape letters (3Di): Foldseek, ProstT5 + Foldseekcounts shared k-mers in a reduced alphabet: kmerseekscans curated Pfam family models: hmmscan with Pfam

Table 2a. Tool by subset of the proteins. The number under each filled square is the count of true instances scored, summed over experiments, query sets, relatives and truth sets. Hover a column name for its id in the tables.

Tool	whole set, per relative	% identity to nearest relative	AlphaFold pLDDT	Swiss-Prot feature type	region length
phmmer	1,329,895	952,047	276,686	181,647	1,133,694
MMseqs2	1,158,512	952,047	276,686	181,647	1,133,694
jackhmmer	1,329,803	952,047	276,686	181,647	1,133,694
MMseqs2, three rounds	1,158,512	952,047	276,686	181,647	1,133,694
HH-suite (hhsearch)	1,158,512	952,047	276,686	181,647	1,133,694
Foldseek	1,110,504	939,872	231,206	145,722	1,085,594
ProstT5 + Foldseek	1,158,512	952,047	276,686	181,647	1,133,694
kmerseek	1,584,821	1,105,115	276,686	181,647	1,133,694
hmmscan with Pfam	8,793	–	8,234	×	8,793
Folddisco (no card)	590,559	516,687	121,049	73,872	590,559
MMseqs2, search mode not recorded (no card)	171,383	×	×	×	×
Reseek (no card)	615,377	516,687	121,049	73,872	590,559

Table 2b. Tool by experiment. The number under each filled square is the count of true instances scored, summed over the query sets, relatives, truth sets and subsets of that experiment. Hover a column name for its id in the tables.

Tool	Alphabets ranked on three test proteins (notebook 241)	Invertebrate dark set: accuracy of the best kmerseek setting	Invertebrate dark set: what kmerseek reaches	Invertebrate dark set: Fmax against Pfam (notebook 291)	Deep-twilight controls, 16 proteins (notebook 252)	Short linear motifs (ELM) covered (notebook 253)	Domain finding, 9 relatives	Domain finding, 15 relatives (this leaderboard)	Domain finding, test set of 2 relatives	SCOPe40 structural classes, kmerseek 0.4 (notebook 277)	SCOPe40, kmerseek 0.4, k below the usual minimum	SCOPe40 control: alphabet letters shuffled
phmmer	×	159,304	×	11,987	92	24,818	1,547,820	1,785,042	344,906	×	×	–
MMseqs2	×	×	×	×	×	24,818	1,547,820	1,785,042	344,906	×	×	–
jackhmmer	×	159,304	×	11,987	×	24,818	1,547,820	1,785,042	344,906	×	×	–
MMseqs2, three rounds	×	×	×	×	×	24,818	1,547,820	1,785,042	344,906	×	×	–
HH-suite (hhsearch)	×	×	×	×	×	24,818	1,547,820	1,785,042	344,906	×	×	–
Foldseek	×	×	×	×	92	24,818	1,547,820	1,595,262	344,906	×	×	–
ProstT5 + Foldseek	×	×	×	×	×	24,818	1,547,820	1,785,042	344,906	×	×	–
kmerseek	2	159,304	93,098	11,987	92	24,818	1,547,820	1,785,042	344,906	44,970	147,070	122,854
hmmscan with Pfam	×	×	×	×	×	×	6,998	7,203	11,619	×	×	–
Folddisco (no card)	×	×	×	×	×	×	1,547,820	×	344,906	×	×	–
MMseqs2, search mode not recorded (no card)	×	159,304	×	11,987	92	×	×	×	×	×	×	–
Reseek (no card)	×	×	×	×	×	24,818	1,547,820	×	344,906	×	×	–

111 squares are filled, 81 are not run and 12 do not apply. Where each experiment's output came from, and which outputs were left out and why: inventory.md (copied with the tables).

What to run next

One line per experiment: the tools it has not run yet.

Alphabets ranked on three test proteins (notebook 241): phmmer, MMseqs2, jackhmmer, MMseqs2, three rounds, HH-suite (hhsearch), Foldseek, ProstT5 + Foldseek, hmmscan with Pfam, Folddisco, MMseqs2, search mode not recorded, Reseek
Invertebrate dark set: what kmerseek reaches: phmmer, MMseqs2, jackhmmer, MMseqs2, three rounds, HH-suite (hhsearch), Foldseek, ProstT5 + Foldseek, hmmscan with Pfam, Folddisco, MMseqs2, search mode not recorded, Reseek
SCOPe40 structural classes, kmerseek 0.4 (notebook 277): phmmer, MMseqs2, jackhmmer, MMseqs2, three rounds, HH-suite (hhsearch), Foldseek, ProstT5 + Foldseek, hmmscan with Pfam, Folddisco, MMseqs2, search mode not recorded, Reseek
SCOPe40, kmerseek 0.4, k below the usual minimum: phmmer, MMseqs2, jackhmmer, MMseqs2, three rounds, HH-suite (hhsearch), Foldseek, ProstT5 + Foldseek, hmmscan with Pfam, Folddisco, MMseqs2, search mode not recorded, Reseek
Invertebrate dark set: accuracy of the best kmerseek setting: MMseqs2, MMseqs2, three rounds, HH-suite (hhsearch), Foldseek, ProstT5 + Foldseek, hmmscan with Pfam, Folddisco, Reseek
Invertebrate dark set: Fmax against Pfam (notebook 291): MMseqs2, MMseqs2, three rounds, HH-suite (hhsearch), Foldseek, ProstT5 + Foldseek, hmmscan with Pfam, Folddisco, Reseek
Deep-twilight controls, 16 proteins (notebook 252): MMseqs2, jackhmmer, MMseqs2, three rounds, HH-suite (hhsearch), ProstT5 + Foldseek, hmmscan with Pfam, Folddisco, Reseek
Short linear motifs (ELM) covered (notebook 253): hmmscan with Pfam, Folddisco, MMseqs2, search mode not recorded
Domain finding, 15 relatives (this leaderboard): Folddisco, MMseqs2, search mode not recorded, Reseek
Domain finding, 9 relatives: MMseqs2, search mode not recorded
Domain finding, test set of 2 relatives: MMseqs2, search mode not recorded