too many populations in the population table

[grahamgower]

I expected 1 population in the output tree sequence, but instead the population table has 7 populations!

initialize() {
	initializeMutationRate(1e-7);
	initializeMutationType("m1", 0.5, "f", 0.0);
	initializeGenomicElementType("g1", m1, 1.0);
	initializeGenomicElement(g1, 0, 99999);
	initializeRecombinationRate(1e-8);
	initializeTreeSeq();
}

1 {
	sim.addSubpop("p6", 500);
	p6.name = "awesomepop";
}

2000 late() { sim.treeSeqOutput("/tmp/too-many-populations.ts"); }

$ tskit info too-many-populations.ts
╔═════════════════════════╗
║TreeSequence             ║
╠═══════════════╤═════════╣
║Trees          │       11║
╟───────────────┼─────────╢
║Sequence Length│   100000║
╟───────────────┼─────────╢
║Time Units     │    ticks║
╟───────────────┼─────────╢
║Sample Nodes   │     1000║
╟───────────────┼─────────╢
║Total Size     │212.1 KiB║
╚═══════════════╧═════════╝
╔═══════════╤════╤════════╤════════════╗
║Table      │Rows│Size    │Has Metadata║
╠═══════════╪════╪════════╪════════════╣
║Edges      │1880│58.8 KiB│          No║
╟───────────┼────┼────────┼────────────╢
║Individuals│ 500│50.6 KiB│         Yes║
╟───────────┼────┼────────┼────────────╢
║Migrations │   0│ 8 Bytes│          No║
╟───────────┼────┼────────┼────────────╢
║Mutations  │ 141│ 9.2 KiB│         Yes║
╟───────────┼────┼────────┼────────────╢
║Nodes      │1853│69.4 KiB│         Yes║
╟───────────┼────┼────────┼────────────╢
║Populations│   7│ 2.4 KiB│         Yes║
╟───────────┼────┼────────┼────────────╢
║Provenances│   1│ 2.1 KiB│          No║
╟───────────┼────┼────────┼────────────╢
║Sites      │ 141│ 3.3 KiB│          No║
╚═══════════╧════╧════════╧════════════╝

[bhaller]

Yes, this is intended behavior. Subpopulations are placed into the population table at the index that their id in SLiM indicates; p6 is placed into slot 6, which means that slots 0-5 need to have empty dummy subpopulations in them. If you don't want that to happen, use p0, not p6. :-> I forget exactly why that design was settled upon, but there were reasons; @petrelharp probably remembers. :-> Anyhow, unless this design actually needs to be changed for some reason – which I don't see from the issue as described here – this is "works as intended", so I'm closing.

[petrelharp]

We've discussed decoupling "population name" from "position in the table", but as you imagine it's a can of worms.

[grahamgower]

I reckon there still might be a problem with the metadata though.

$ python
Python 3.9.9 (main, Nov 20 2021, 21:30:06)
[GCC 11.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tskit
>>> ts = tskit.load("too-many-populations.ts")
>>> len(ts.populations())
7
>>> bytes(ts.tables.populations.metadata)
b'{"migration_records":[],"name":"awesomepop","sex_ratio":0.0,"slim_id":6}'

Shouldn't there be 7 metadata records for 7 populations?

[bhaller]

The empty subpopulations that are added to pad out the table do not receive metadata, since they don't really even exist. This is why the metadata schema for the population table says "type":["object","null"] – the slim_id property is required if the metadata is an object, but the metadata is also allowed to be null. So if I understand correctly, this is all intended behavior, if admittedly a rather odd design.