Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect synthetic source for object arrays withtin nested objects #115261

Closed
lkts opened this issue Oct 21, 2024 · 3 comments · Fixed by #116065
Closed

Incorrect synthetic source for object arrays withtin nested objects #115261

lkts opened this issue Oct 21, 2024 · 3 comments · Fixed by #116065
Assignees
Labels
>bug :StorageEngine/Mapping The storage related side of mappings Team:StorageEngine

Comments

@lkts
Copy link
Contributor

lkts commented Oct 21, 2024

For object arrays that are within nested object and contain leaf values that are "mixed" (some synthetic source values come from doc_values and some from ignored source) synthetic source is not correct.

Following test demonstrates that:

    public void testObjectArrayWithinNestedObjects() throws IOException {
        DocumentMapper documentMapper = createMapperService(syntheticSourceMapping(b -> {
            b.startObject("path").startObject("properties");
            {
                b.startObject("to").field("type", "nested").startObject("properties");
                {
                    b.startObject("obj").startObject("properties");
                    {
                        b.startObject("id").field("type", "integer").field("synthetic_source_keep", "arrays").endObject();
                    }
                    b.endObject().endObject();
                }
                b.endObject().endObject();
            }
            b.endObject().endObject();
        })).documentMapper();

        var syntheticSource = syntheticSource(documentMapper, b -> {
            b.startObject("path");
            {
                b.startObject("to");
                {
                    b.startArray("obj");
                    {
                        b.startObject().array("id", 1, 20, 3).endObject();
                        b.startObject().field("id", 10).endObject();
                    }
                    b.endArray();
                }
                b.endObject();
            }
            b.endObject();
        });
        assertEquals("""
            {"path":{"to":{"obj":{"id":[1,20,3,10]}}}}""", syntheticSource);
    }

This test produces {"path":{"to":{"obj":{"id":[1,20,3]}}}}.

@lkts lkts added :StorageEngine/Mapping The storage related side of mappings >bug labels Oct 21, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-storage-engine (Team:StorageEngine)

@lkts
Copy link
Contributor Author

lkts commented Oct 21, 2024

Related #114891.

@lkts
Copy link
Contributor Author

lkts commented Oct 28, 2024

This bug is fixed by applying a workaround (see PR above). We would like to improve the fix so that it does not require storing additional data in ignored source.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :StorageEngine/Mapping The storage related side of mappings Team:StorageEngine
Projects
None yet
3 participants