ImageDisplay incorrectly repeats first time point of 5D (xyzct) ImgPlus

[bhoeckendorf]

I'm using the following code to display a 5D (xyzct) ImgPlus in ImageJ. When scrolling through time, the display repeats the first time point instead of switching to the newly selected time points. I have verified that the data is loaded properly, i.e. the axes of the ImgPlus instance have the expected order and length and traversing over the time axis yields time-varying values.

Dataset data = datasetService.create( imgPlus );
displayService.createDisplay( data );

[rimadoma]

Out of interest, does the same thing happen if you call uiService.show(data)?

[bhoeckendorf]

Yes, the time slider still has no apparent effect.
Thanks for the shortcut though, I'll use UIService from now on instead of DatasetService and DisplayService.

[rimadoma]

I reckon this happens because the Dataset doesn't have any metadata about the axes, e.g. which ones are x,y,c,z and t. Try calling

Dataset data = datasetService.create( imgPlus );
data.setAxes(new CalibratedAxis[] { new DefaultLinearAxis(Axes.X),
	new DefaultLinearAxis(Axes.Y), new DefaultLinearAxis(Axes.CHANNEL),
	new DefaultLinearAxis(Axes.Z), new DefaultLinearAxis(Axes.TIME) });
uiService.show(data);

[bhoeckendorf]

I ran the following code to first print the AxisTypes of the ImgPlus and then manually assign the same AxesTypes to the Dataset as per your suggestion. The axes of the ImgPlus are configured properly and the problem persists.

Code:

for (int i = 0; i < img.numDimensions(); ++i)
{
    Axis ax = img.axis( i );
    if (ax instanceof CalibratedAxis)
    {
        AxisType axType = (( CalibratedAxis ) ax).type();
        System.out.println(axType);
    } else {
        System.out.println("!instanceof CalibratedAxis");
    }
}

final Dataset data = datasetService.create( img );
data.setAxes(new CalibratedAxis[] { new DefaultLinearAxis( Axes.X),
        new DefaultLinearAxis(Axes.Y), new DefaultLinearAxis(Axes.Z),
        new DefaultLinearAxis(Axes.CHANNEL), new DefaultLinearAxis(Axes.TIME) });

uiService.show( img );

Output:

X
Y
Z
Channel
Time

[rimadoma]

You have uiService.show( img ); which should be uiService.show( data );. The (legacy) UI is quite particular about the order of axes in a hyperstack: it needs to be xyczt, not xyzct :)

[bhoeckendorf]

Apologies, just noticed this myself and already fixed and re-run the test. Same result.

[bhoeckendorf]

I see, so then the problem is that ImageJ can not deal with the memory layout of my data. My images can be quite big and are stored in xyzct order on disk in a way that provides very efficient access. So reordering the dimensions seems like it would be a bit of a headache. Any suggestions?

[rimadoma]

What I know from ImageJ2 architecture is that concerns about loading images/data from disk are abstracted away, and that the order of axes in an ImgPlus is a high level concern which doesn't affect how the image is loaded to memory.

I have some simple code that creates 5D hyperstacks for testing purposes, I hope it helps you to find an answer to your problem: https://github.com/bonej-org/BoneJ2/tree/master/Modern/testImgs/src/main/java/org/bonej/testImages

[ctrueden]

So reordering the dimensions seems like it would be a bit of a headache.

Have you tried Views.permute to change the order to XYCZT, then wrap as ImgPlus/Dataset?

It would be great for the ImageJ Legacy layer to do this automatically if given an ImgPlus or Dataset with different axis order, but I'm not sure whether it does currently.

So your code above would become:

ImgView<T> permutedImg = new ImgView<>(Views.permute(img, 2, 3));
final Dataset data = datasetService.create( permutedImg );
data.setAxes(new CalibratedAxis[] { new DefaultLinearAxis( Axes.X),
        new DefaultLinearAxis(Axes.Y), new DefaultLinearAxis(Axes.CHANNEL),
        new DefaultLinearAxis(Axes.Z), new DefaultLinearAxis(Axes.TIME) });

uiService.show( img );

I did not test it, though.

[rimadoma]

@ctrueden In my experience no, order has to be XYCZT or sliders / image display doesn't work properly

[ctrueden]

In my experience no, order has to be XYCZT or sliders / image display doesn't work properly

I filed imagej/imagej-legacy#147 to track this idea. Note that imagej/imagej-legacy#146 may also be relevant here... if there is an actual bug beyond just the need for a hardcoded order.

[bhoeckendorf]

Have you tried Views.permute to change the order to XYCZT, then wrap as ImgPlus/Dataset?

Thanks @ctrueden, this actually seems to work. Views.permute takes a RandomAccessibleInterval as argument and returns an IntervalView, neither of which know about the axis and sampling meta data of the ImgPlus. The initial result thus appears to be a 3D stack. After re-setting the meta data on the Dataset produced from the permuted volume, I now get the 5D display that I expected and I can browse all axes.

What I know from ImageJ2 architecture is that concerns about loading images/data from disk are abstracted away, and that the order of axes in an ImgPlus is a high level concern which doesn't affect how the image is loaded to memory.

Thanks for sharing your code, @rimadoma. Unfortunately I use a custom reader that loads 3D-5D blocks in xyzct order, which is also the order in which the images are stored. Hence, I do not see an easy way to handle this on the data loading side without doing some reordering of these blocks, which I'd really like to avoid.

Thankfully, using a permuted View seems to be a good-enough solution for now.

[ctrueden]

@bhoeckendorf Glad you got it working.

Some day, SCIFIO will be blockized! And on that day, we can have a party. And port your reader over.

[bhoeckendorf]

@ctrueden Since you filed imagej/imagej-legacy#147: Its not too hard to do obviously, but I thought I'd share the code that fixes it for me anyway. I would open a pull request, but I'm not sure this is 'correct enough' for the general case. Also I don't know which class to apply this to. The hierarchy always seems a bit convoluted to me.

Dataset data;
// ImageJ expects xyczt order, loaded order is xyzct.
// Permute if needed, be mindful of squeezed singleton dimensions.
int zDim = 2, cDim = -1;
for ( int i = 0; i < img.numDimensions(); ++i ) {
    final Axis ax = img.axis( i );
    if ( ax instanceof CalibratedAxis ) {
        final AxisType axType = (( CalibratedAxis ) ax).type();
        if ( axType == Axes.CHANNEL ) {
            cDim = i;
            break;
        }
    }
}
if ( cDim != -1 && cDim != 2 ) {
    data = datasetService.create( Views.permute( img, zDim, cDim ) );
    final CalibratedAxis[] axes = new CalibratedAxis[ img.numDimensions() ];
    for ( int i = 0; i < axes.length; ++i ) {
        if ( i == zDim )
            axes[ i ] = ( CalibratedAxis ) img.axis( cDim );
        else if ( i == cDim )
            axes[ i ] = ( CalibratedAxis ) img.axis( zDim );
        else
            axes[ i ] = ( CalibratedAxis ) img.axis( i );
    }
    data.setAxes( axes );
} else {
    data = datasetService.create( img );
}

uiService.show( data );