Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing ruleset reference in the SDMX repository #373

Merged
merged 3 commits into from
Nov 29, 2024
Merged

Missing ruleset reference in the SDMX repository #373

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
85f34bc
Highlight the problem with the ruleset
hadrienk Nov 29, 2024
6983915
Disable SDMX testRefFromRepo test for now
NicoLaval Nov 29, 2024
0fc9bb7
Add ruleset reference to transformation in VTL BPE sample
NicoLaval Nov 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions vtl-sdmx/src/test/java/fr/insee/vtl/SDMXVTLWorkflowTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
import io.sdmx.utils.core.io.ReadableDataLocationTmp;
import org.apache.spark.sql.SparkSession;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.Test;

import javax.script.ScriptContext;
Expand All @@ -30,6 +31,27 @@ public void setUp() {
engine = mgr.getEngineByExtension("vtl");
}

@Disabled
@Test
void testRefFromRepo() {
// Works partially, the transformation does not pull in the ruleset. Maybe the transformation is wrong and does not
// reference ruleset?
String url = "https://registry.sdmx.io/sdmx/v2/structure/transformationscheme/FR1/BPE_CENSUS/+/?format=sdmx-3.0&references=all";
ReadableDataLocation rdl = new ReadableDataLocationTmp(url);
SDMXVTLWorkflow sdmxVtlWorkflow = new SDMXVTLWorkflow(engine, rdl, Java8Helpers.mapOf());
System.out.println(sdmxVtlWorkflow.getEmptyDatasets());

System.out.println(sdmxVtlWorkflow.getTransformationsVTL());

// Invalid step definition for:CHECK_MUNICIPALITY
// - Caused by: fr.insee.vtl.engine.exceptions.UndefinedVariableException: undefined variable UNIQUE_MUNICIPALITY
engine.getBindings(ScriptContext.ENGINE_SCOPE).putAll(sdmxVtlWorkflow.getEmptyDatasets());
Map<String, PersistentDataset> result = sdmxVtlWorkflow.run();
assertThat(result).containsKeys(
"BPE_CENSUS_NUTS3_2021", "BPE_MUNICIPALITY", "BPE_NUTS3"
);
}

@Test
void testGetEmptyDataset() {

Expand Down
1 change: 1 addition & 0 deletions vtl-sdmx/src/test/resources/DSD_BPE_CENSUS.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -371,6 +371,7 @@
<str:TransformationSchemes>
<str:TransformationScheme agencyID="FR1" id="BPE_CENSUS" version="1.0" vtlVersion="2.0">
<com:Name xml:lang="en">Transformation Scheme for BPE - CENSUS</com:Name>
<str:RulesetScheme>urn:sdmx:org.sdmx.infomodel.transformation.RulesetScheme=FR1:RS1(1.0)</str:RulesetScheme>
<str:Transformation id="STEP_1" isPersistent="false">
<com:Name xml:lang="en">Step 1</com:Name>
<com:Description xml:lang="en">Validation of municipality code in input file</com:Description>
Expand Down