Skip to content

Commit

Permalink
CgmesImporter, allow importing with wrong dataextension when the main…
Browse files Browse the repository at this point in the history
…file doesn't exist

This happens when filenames have the 2 following properties:
- the filename contains a dot:
- the filename does not contain the dataextension
For example this filename triggers the bug:  "microgrid.v2.complete.zip"
but these work:
   "microgrid_v2_complete.zip" (because empty dataextension)
or "microgrid.v2.complete.xml.zip" (because ".xml" dataextension)

The problem is that the main extension is derived as ".v2.complete" but the
cgmes importer only accepts an empty dataextension or ".xml" (it checks the
dataextension to avoid importing as cgmes from a datasource that was meant by
the user to be used as another format). But for now no other importer works
without a main file, so when the mainfile doesn't exist so we shouldn't block
the import just like when the dataextension is not specified

NOTE: this happens because we want to allow archive filenames to be very
flexible (with or without the dataextension, and for cgmes even with a totally
different basename as what is inside).

NOTE: if we ever have another importer that works like the cgmes importer,
this means we will not be able to handle the following case:
archive/network_EQ.xml
archive/network_TP.xml # cgmes files
archive/network_foo.other
archive/network_bar.other # some new importer format
and be able to import from a single directory containing both networks by specifying
the dataextension. But this should be a rare case, and moving the data to separate
directories or using separate basenames works around the problem.
  • Loading branch information
jonenst committed Sep 19, 2024
1 parent f8e1580 commit 7fe3adc
Show file tree
Hide file tree
Showing 2 changed files with 14 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ public ReadOnlyDataSource dataSource() {
}

private boolean checkIfMainFileNotWithCgmesData(boolean isCim14) throws IOException {
if (dataSource.getDataExtension() == null || dataSource.getDataExtension().isEmpty()) {
if (dataSource.getDataExtension() == null || dataSource.getDataExtension().isEmpty() || !dataSource.exists(null, dataSource.getDataExtension())) {
return false;
} else if (EXTENSION.equals(dataSource.getDataExtension()) && dataSource.exists(null, EXTENSION)) {
try (InputStream is = dataSource.newInputStream(null, EXTENSION)) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;

/**
* @author Jon Harper {@literal <jon.harper at rte-france.com>}
Expand Down Expand Up @@ -71,13 +72,24 @@ void testFileDoesNotExist() throws IOException {
byte[] data = "Test String".getBytes();
out.write(data, 0, data.length);
out.closeEntry();
e = new ZipEntry("foo.xml");
out.putNextEntry(e);
data = getClass().getResourceAsStream("/empty_cim16_EQ.xml").readAllBytes();
out.write(data, 0, data.length);
out.closeEntry();
} catch (IOException ex) {
throw new RuntimeException(ex);
}
}
ReadOnlyDataSource dataSource = new ZipArchiveDataSource(testDir, "foo.iidm.zip", "test", "xml", null);
CgmesOnDataSource cgmesOnDataSource = new CgmesOnDataSource(dataSource);
assertFalse(cgmesOnDataSource.exists());
assertTrue(cgmesOnDataSource.exists());
ReadOnlyDataSource dataSource2 = new ZipArchiveDataSource(testDir, "foo.iidm.zip", "test", "iidm", null);
CgmesOnDataSource cgmesOnDataSource2 = new CgmesOnDataSource(dataSource2);
assertTrue(cgmesOnDataSource2.exists());
ReadOnlyDataSource dataSource3 = new ZipArchiveDataSource(testDir, "foo.iidm.zip", "foo", "bar", null);
CgmesOnDataSource cgmesOnDataSource3 = new CgmesOnDataSource(dataSource3);
assertFalse(cgmesOnDataSource3.exists());
}
}
}

0 comments on commit 7fe3adc

Please sign in to comment.