Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recursive schema #457

Closed
nmrtv opened this issue Apr 13, 2021 · 8 comments
Closed

Recursive schema #457

nmrtv opened this issue Apr 13, 2021 · 8 comments
Labels
bug Something isn't working

Comments

@nmrtv
Copy link
Contributor

nmrtv commented Apr 13, 2021

When schema uses itself in one of the fields, it raises an exception that the name is not defined. For python versions < 3.10, the line from __future__ import annotations must be included at the top of the autogenerated file.

@tefra
Copy link
Owner

tefra commented Apr 13, 2021

Hi @neriusmika can you provide a sample schema?

xsdata already takes care of circular references with forward references. Most of my use cases are from w3c test suite and are similar to this

<?xml version="1.0" encoding="UTF-8" ?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified">
  <xsd:element name="b">
    <xsd:complexType>
      <xsd:sequence>
        <xsd:element ref="b" minOccurs="0" />
      </xsd:sequence>
      <xsd:attribute name="id" type="xsd:string" />
    </xsd:complexType>
  </xsd:element>
</xsd:schema>
@dataclass
class B:
    class Meta:
        name = "b"

    b: Optional["B"] = field(
        default=None,
        metadata={
            "type": "Element",
        }
    )
    id: Optional[str] = field(
        default=None,
        metadata={
            "type": "Attribute",
        }
    )
import sys

from issue.models import B
from xsdata.formats.dataclass.serializers import XmlSerializer

b = B(b=B(b=B(), id="2"), id="1")

serializer = XmlSerializer()
serializer.write(sys.stdout, b)
<?xml version="1.0" encoding="UTF-8"?>
<b id="1"><b id="2"><b/></b></b>

FYI the first python 3.10 alpha versions have a few issues in the typing module that affect forward references.

@nmrtv
Copy link
Contributor Author

nmrtv commented Apr 13, 2021

I have tried with urn:oasis:names:specification:ubl:schema:xsd:Order-2 from http://docs.oasis-open.org/ubl/UBL-2.1.html. The error was in a file for "common aggregate components" on ContractingPartyType1.

FYI the first python 3.10 alpha versions have a few issues in the typing module that affect forward references.

Then adding quotes is ok. The problem is that it doesn't handle recursive schemas in all cases then.

Btw, Ubl is a good source for some heavy schema testing. That single file for common aggregate components is 54000+ lines of python code :)

@nmrtv
Copy link
Contributor Author

nmrtv commented Apr 13, 2021

The failing autogenerated class:

@dataclass
class ContractingPartyType1:
    """<ns1:Component xmlns:ns1="urn:un:unece:uncefact:documentation:2">

    <ns1:ComponentType>ABIE</ns1:ComponentType>
    <ns1:DictionaryEntryName>Contracting Party.
    Details</ns1:DictionaryEntryName> <ns1:Definition>A class to
    describe an individual, a group, or a body having a procurement role
    in a tendering process.</ns1:Definition>
    <ns1:ObjectClass>Contracting Party</ns1:ObjectClass>
    </ns1:Component>

    Attributes
        buyer_profile_uri: <ns1:Component
            xmlns:ns1="urn:un:unece:uncefact:documentation:2">
            <ns1:ComponentType>BBIE</ns1:ComponentType>
            <ns1:DictionaryEntryName>Contracting Party. Buyer Profile_
            URI. Identifier</ns1:DictionaryEntryName>
            <ns1:Definition>The buyer profile is typically located on a
            web site where the contracting party publishes its
            procurement opportunities</ns1:Definition>
            <ns1:Cardinality>0..1</ns1:Cardinality>
            <ns1:ObjectClass>Contracting Party</ns1:ObjectClass>
            <ns1:PropertyTermQualifier>Buyer
            Profile</ns1:PropertyTermQualifier>
            <ns1:PropertyTerm>URI</ns1:PropertyTerm>
            <ns1:RepresentationTerm>Identifier</ns1:RepresentationTerm>
            <ns1:DataType>Identifier. Type</ns1:DataType>
            <ns1:AlternativeBusinessTerms>Buyer
            Profile</ns1:AlternativeBusinessTerms> </ns1:Component>
        contracting_party_type: <ns1:Component
            xmlns:ns1="urn:un:unece:uncefact:documentation:2">
            <ns1:ComponentType>ASBIE</ns1:ComponentType>
            <ns1:DictionaryEntryName>Contracting Party. Contracting
            Party Type</ns1:DictionaryEntryName>   <ns1:Definition>The
            type of contracting party that is independent of its
            role.</ns1:Definition>
            <ns1:Cardinality>0..n</ns1:Cardinality>
            <ns1:ObjectClass>Contracting Party</ns1:ObjectClass>
            <ns1:PropertyTerm>Contracting Party Type</ns1:PropertyTerm>
            <ns1:AssociatedObjectClass>Contracting Party
            Type</ns1:AssociatedObjectClass>
            <ns1:RepresentationTerm>Contracting Party
            Type</ns1:RepresentationTerm> </ns1:Component>
        contracting_activity: <ns1:Component
            xmlns:ns1="urn:un:unece:uncefact:documentation:2">
            <ns1:ComponentType>ASBIE</ns1:ComponentType>
            <ns1:DictionaryEntryName>Contracting Party. Contracting
            Activity</ns1:DictionaryEntryName>   <ns1:Definition>The
            nature of the type of business of the
            organization</ns1:Definition>
            <ns1:Cardinality>0..n</ns1:Cardinality>
            <ns1:ObjectClass>Contracting Party</ns1:ObjectClass>
            <ns1:PropertyTerm>Contracting Activity</ns1:PropertyTerm>
            <ns1:AssociatedObjectClass>Contracting
            Activity</ns1:AssociatedObjectClass>
            <ns1:RepresentationTerm>Contracting
            Activity</ns1:RepresentationTerm> </ns1:Component>
        party: <ns1:Component
            xmlns:ns1="urn:un:unece:uncefact:documentation:2">
            <ns1:ComponentType>ASBIE</ns1:ComponentType>
            <ns1:DictionaryEntryName>Contracting Party.
            Party</ns1:DictionaryEntryName>   <ns1:Definition>The
            contracting party itself.</ns1:Definition>
            <ns1:Cardinality>1</ns1:Cardinality>
            <ns1:ObjectClass>Contracting Party</ns1:ObjectClass>
            <ns1:PropertyTerm>Party</ns1:PropertyTerm>
            <ns1:AssociatedObjectClass>Party</ns1:AssociatedObjectClass>
            <ns1:RepresentationTerm>Party</ns1:RepresentationTerm>
            </ns1:Component>
    """
    class Meta:
        name = "ContractingPartyType"

    buyer_profile_uri: Optional[BuyerProfileUri] = field(
        default=None,
        metadata={
            "name": "BuyerProfileURI",
            "type": "Element",
            "namespace": "urn:oasis:names:specification:ubl:schema:xsd:CommonBasicComponents-2",
        }
    )
    contracting_party_type: List[ContractingPartyType1] = field(
        default_factory=list,
        metadata={
            "name": "ContractingPartyType",
            "type": "Element",
            "namespace": "urn:oasis:names:specification:ubl:schema:xsd:CommonAggregateComponents-2",
        }
    )
    contracting_activity: List[ContractingActivity] = field(
        default_factory=list,
        metadata={
            "name": "ContractingActivity",
            "type": "Element",
            "namespace": "urn:oasis:names:specification:ubl:schema:xsd:CommonAggregateComponents-2",
        }
    )
    party: Optional[Party] = field(
        default=None,
        metadata={
            "name": "Party",
            "type": "Element",
            "namespace": "urn:oasis:names:specification:ubl:schema:xsd:CommonAggregateComponents-2",
            "required": True,
        }
    )

@nmrtv
Copy link
Contributor Author

nmrtv commented Apr 13, 2021

@tefra
Copy link
Owner

tefra commented Apr 13, 2021

That's an interesting one, it will take a moment to dissect but from a quick debug I see that the analyzer failed to detect the circular reference between all the layers of reducing the classes.

Thanks for reporting @neriusmika

@tefra tefra added the bug Something isn't working label Apr 13, 2021
tefra added a commit that referenced this issue Apr 13, 2021
tefra added a commit that referenced this issue Apr 13, 2021
The sanitizer has to rename duplicate class names
and their references. This could lead in renaming
incorrecly references just because they share the
same qualified name.

Notes:
Closes #457
@tefra
Copy link
Owner

tefra commented Apr 13, 2021

Actually it's not a circular reference, due to the duplicate names some references are renamed by mistake.

I am still testing but here is the fix #458

I am definitely adding this collection to the tefra/xsdata-samples#39

Note to self: this collection really begs for a new config option to skip generating docstrings

@nmrtv
Copy link
Contributor Author

nmrtv commented Apr 13, 2021

Note to self: this collection really begs for a new config option to skip generating docstrings

Yes, actually I was thinking about that too :)

Thank you for a quick fix.

tefra added a commit that referenced this issue Apr 14, 2021
The sanitizer has to rename duplicate class names
and their references. This could lead in renaming
incorrecly references just because they share the
same qualified name.

Notes:
Closes #457
@tefra tefra closed this as completed in 22bbde7 Apr 14, 2021
@tefra
Copy link
Owner

tefra commented Apr 14, 2021

Thank you for reporting @neriusmika

really cool xsd collection!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants