Add force_offset64 option to TableCollection.asdict

[jeromekelleher]

To keep compatibility with existing code, the TableCollection.asdict method will need to return 32 bit offset values, like the lightweight TableCollection. So, as another step on the 64 bit road, we should add the option to force returning 64 bit offset values. This is actually trickier then for the LightweightTableCollection, because we're getting the table columns via attribute accesses rather than method calls, and so will have to change some of the low-level infrastructure.

Another tricky question arises then: when someone does a direct access to (say) NodeTable.metadata_offset should they get a 64 bit value, or a 32/64 bit value depending on the size? I think probably 64 bit, but it may turn out that this breaks too much code. We'll see, I guess.

[benjeffery]

because we're getting the table columns via attribute accesses rather than method calls, and so will have to change some of the low-level infrastructure.

I'm not clear here on why we need to change the low level - don't we just convert to the desired width at the python level?

I agree on making the direct access 64bit only as I can't see how breakage can come from that, especially as we are going to a larger size.

[jeromekelleher]

I'm not clear here on why we need to change the low level - don't we just convert to the desired width at the python level?

Right, you could do a post-hoc conversion up to 64 bit, I didn't think of that. I've figured out a better way in #1602 though, that reuses the C code for making the dict encoding so we only have one implementation of asdict.