Bad UTF-8 handling for NTriple parser

[Arthur-VaisseLesteven]

The current NTriple parser does not handle properly utf-8 string.

the following triple trigger an error :
<http://linkedgeodata.org/triplify/user22701> <http://www.w3.org/2000/01/rdf-schema#label> "CiaránMooney" .

when executing this code :

line = codecs.open("file", "r", "utf-8").readline()
parser = NTriples.NTriplesParser(sink = mySink)
parser.parsestring(line)#where line is the previous triple

The parser fail to parse the "á" character.
Modifying the parse function of the NTriplesParser class allow to solve this bug.

Current :

  def parse(self, f):
        """Parse f as an N-Triples file."""
        if not hasattr(f, 'read'):
            raise ParseError("Item to parse must be a file-like object.")

        f = ascii(f)
        ...

Fix:

  def parse(self, f):
        """Parse f as an N-Triples file."""
        if not hasattr(f, 'read'):
            raise ParseError("Item to parse must be a file-like object.")

        f = codecs.getreader("utf-8")(f)
        ...

Hope this help.

AVL

[joernhees]

i think this problem exists because the old N-Triples spec did not allow utf-8 encoding (but used 7bit ascii + \u... escapes): http://www.w3.org/2001/sw/RDFCore/ntriples/

RDF 1.1 changed this: http://www.w3.org/TR/n-triples/#n-triples-mediatype

question here is if someone still wants the old N-Triples behavior (e.g., for legacy reasons)... should we make this configurable? or maybe just on serialization?

[wojtasskorcz]

Are there any plans to resolve this issue in the near future? I have the same problem as @Arthur-VaisseLesteven

[gromgull]

I am afraid I have no time for rdflib atm, but pull-requests are welcome!

[indeyets]

@joernhees new spec seems to be backwards compatible in terms of parsing, so I don't think there's any reason to avoid implement it. serialization is a whole different story, though. it should have a flag for emitting old-style files