Should it support parsing containers of std::byte?

[jasujm]

Thanks for what is probably the most awesome C++ JSON library!

Not reporting this as an issue, because this question is a matter of taste and the maintainers of the library may well disagree with my view.

Using the version 3.9.1 of the library, I get compilation errors when trying to compile the following program:

$ g++ --version
g++ (Ubuntu 10.2.0-13ubuntu1) 10.2.0
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ cat test.cc
#include <iostream>
#include <span>

#include <nlohmann/json.hpp>

const char DATA[] = R"("Hello, world!")";

int main()
{
    const auto s = std::as_bytes(std::span(DATA));
    std::cout << nlohmann::json::parse(s) << std::endl;
    return 0;
}
$ g++ -std=c++20 test.cc
In file included from /usr/include/nlohmann/detail/input/binary_reader.hpp:18,
                 from /usr/include/nlohmann/json.hpp:55,
                 from test.cc:4:
/usr/include/nlohmann/detail/input/lexer.hpp: In instantiation of ‘nlohmann::detail::lexer<BasicJsonType, InputAdapterType>::token_type nlohmann::detail::lexer<BasicJsonType, InputAdapterType>::scan() [with BasicJsonType = nlohmann::basic_json<>; InputAdapterType = nlohmann::detail::iterator_input_adapter<__gnu_cxx::__normal_iterator<const std::byte*, std::span<const std::byte, 16> > >; nlohmann::detail::lexer<BasicJsonType, InputAdapterType>::token_type = nlohmann::detail::lexer_base<nlohmann::basic_json<> >::token_type]’:
/usr/include/nlohmann/detail/input/parser.hpp:458:41:   required from ‘nlohmann::detail::parser<BasicJsonType, InputAdapterType>::token_type nlohmann::detail::parser<BasicJsonType, InputAdapterType>::get_token() [with BasicJsonType = nlohmann::basic_json<>; InputAdapterType = nlohmann::detail::iterator_input_adapter<__gnu_cxx::__normal_iterator<const std::byte*, std::span<const std::byte, 16> > >; nlohmann::detail::parser<BasicJsonType, InputAdapterType>::token_type = nlohmann::detail::lexer_base<nlohmann::basic_json<> >::token_type]’
/usr/include/nlohmann/detail/input/parser.hpp:94:28:   required from ‘void nlohmann::detail::parser<BasicJsonType, InputAdapterType>::parse(bool, BasicJsonType&) [with BasicJsonType = nlohmann::basic_json<>; InputAdapterType = nlohmann::detail::iterator_input_adapter<__gnu_cxx::__normal_iterator<const std::byte*, std::span<const std::byte, 16> > >]’
/usr/include/nlohmann/json.hpp:6654:111:   required from ‘static nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer, BinaryType> nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer, BinaryType>::parse(InputType&&, nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer, BinaryType>::parser_callback_t, bool, bool) [with InputType = const std::span<const std::byte, 16>&; ObjectType = std::map; ArrayType = std::vector; StringType = std::__cxx11::basic_string<char>; BooleanType = bool; NumberIntegerType = long int; NumberUnsignedType = long unsigned int; NumberFloatType = double; AllocatorType = std::allocator; JSONSerializer = nlohmann::adl_serializer; BinaryType = std::vector<unsigned char>; nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer, BinaryType>::parser_callback_t = std::function<bool(int, nlohmann::detail::parse_event_t, nlohmann::basic_json<>&)>]’
test.cc:11:41:   required from here
/usr/include/nlohmann/detail/input/lexer.hpp:1544:59: error: cannot convert ‘char’ to ‘std::byte’ in initialization
 1544 |                 std::array<char_type, 4> true_literal = {{'t', 'r', 'u', 'e'}};
      |                                                           ^~~
      |                                                           |
      |                                                           char
...etc

So, bunch of compilation errors related to mismatch between std::bytes to char literals.

Before updating the library from 3.7 to 3.9 this actually used to work.

Is it intentional that parsing containers of std::bytes don't work without casting? std::byte is a very natural model for wire data, and I feel parsing it "should" just work.

[nlohmann]

Thanks for reporting, and sorry for breaking your code.

I have not yet taken C++20 into account.

[nlohmann]

Alright, got it. I could fix the code by adding an explicit conversion:

// literals
case 't':
{
    std::array<char_type, 4> true_literal = {{char_type('t'), char_type('r'), char_type('u'), char_type('e')}};
    return scan_literal(true_literal.data(), true_literal.size(), token_type::literal_true);
}
case 'f':
{
    std::array<char_type, 5> false_literal = {{char_type('f'), char_type('a'), char_type('l'), char_type('s'), char_type('e')}};
    return scan_literal(false_literal.data(), false_literal.size(), token_type::literal_false);
}
case 'n':
{
    std::array<char_type, 4> null_literal = {{char_type('n'), char_type('u'), char_type('l'), char_type('l')}};
    return scan_literal(null_literal.data(), null_literal.size(), token_type::literal_null);
}

I created a PR: #2550

[jasujm]

Nice, thank you :)

[nlohmann]

I had to make some more fixes to compile the tests with C++20. Thanks for reporting this issue - I would never have thought about compiling for C++20 :)

[nlohmann]

Merged.