Skip to content

#770: Radical result iterators rewrite #980

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 17 commits into
base: start-8
Choose a base branch
from
Draft

#770: Radical result iterators rewrite #980

wants to merge 17 commits into from

Conversation

jtv
Copy link
Owner

@jtv jtv commented Mar 30, 2025

Fixes: #770

Major rewrite of the whole row/field/iterator constellation for pqxx::result and its rows. Requires that the original pqxx::result remain valid (and does not move) while the iterations are ongoing.

It's not easy. Things got a little convoluted because:

  1. operator*() is supposed to return a reference to something, but there are no persistent objects to reference.
  2. I used inheritance too much.
  3. The interfaces are pretty rich: i[row][column] indexing, accessing features directly on the iterators without dereferencing...
  4. We had classes that implemented both a container and an iterator on another container, which led to conflicts.

This rewrite should fix that, at the cost of a major compatibility break. This stuff is not easy.

Meant to address:

  • Performance cost of holding a pqxx::result in every single iterator (which is a smart pointer plus extras).
  • Conflicts in member type definitions (between a result iterator's rôles as an iterator and as a container).
  • Row operator[] not having the appropriate definition for a standard iterator.

@jtv jtv marked this pull request as draft March 30, 2025 20:29
@jtv jtv changed the title WIP: #770: Radical result iterators rewrite #770: Radical result iterators rewrite Mar 30, 2025
jtv and others added 15 commits April 10, 2025 16:15
* Retire `binarystring`.

This type has been deprecated since 7.2.0, more than 4 years ago.

* String conversion to `string_view`.  More binary types.

Fixes: #694
Fixes: #827

Making much broader use of concepts.  String conversions now accept any
contiguous range of `std::byte` as binary data.  Traits specialisations
for integer and floating-point types are simpler now.  And you can now
just convert from string to `std::string_view` (or `char const *`), so
long as you don't access it after the original string's lifetime ends.

* Work around Visual Studio 2022 concepts problem.

This compiler was having trouble with the syntax I used to specialise
the generic `string_traits<T>` template to a _concept_ `T` (as opposed
to run-of-the-mill specialisation to a _type_ `T`).

So just for the floating-point string traits, I went back to the old
setup where I had a separate implementation type template
(`string_float_traits`) and derived the `string_traits` implementations
for those types from instantiations of that template.

* Forbid string conversion from `char const *`.

It was stupid of me to allow this.  I hope nobody ever used it.
A quick overview of the changes to the conversion API...
* Pointer/length pairs are now `std::span<char>`.
* Raw pointer return is now `std::size_at`.
* Don't call `pqxx::string_traits<T>::to_buf()` etc.  Call `pqxx::to_buf<T>()` etc. instead.
* There's `pqxx::to_buf()`, `pqxx::into_buf()`, and `pqxx::from_string()`.  They follow the new API.
* The string conversion functions now take a `pqxx::conversion_context` reference, containing:
  * Encoding group.
  * `std::source_location` in case the code needs to throw an exception.
* The `encoding_group` enum is no longer "internal."  Feel free to use it.
* There's a new encoding group, `UNKNOWN`.  This is the default in `pqxx::conversion_context`.
* Code that needs to know the encoding group, such as the array parser, can throw a `pqxx::usage_error` if given an `UNKNOWN` encoding.
* You can now convert a string to a `pqxx::array`.
I'm trying to ensure that every libpqxx function that may throw an exception has a `source_location` parameter, and that it passes this to any functions it calls that may throw.

The end result would ideally be that any exception thrown by libpqxx would include the location of the original call into libpqxx.  (Unless you pass your own choice of source location, of course).

In practice, this ideal is not quite attainable...
* Parameter packs make it hard — you can't just make a function accept any number of arguments of any type, and then add a `source_location` parameter _after_ that.
* With overloaded operators it's completely impossible — there's no way of adding parameters to a function with a fixed, language-dictated signature.
* Destructors take no arguments at all, so they're out as well.
* Callbacks and string conversions are a problem as well, because their existing definitions don't include `source_location`.

I do have a plan for that last item though.  I can probably update the signatures and use compile-time introspection (probably using function concepts) to detect whether the given user-defined function supports the old API or the new API.  That's a separate job though.

Oh, and because I don't want to grow all my function signatures with an additional tedious `std::source_location location = std::source_location::current()` etc, I created a type alias `sl`.  Hate how cryptic this makes the functions, but I really need these to be brief.
jtv added 2 commits April 14, 2025 00:45
We're nowhere near done yet though.  The tests do not compile yet, and
plenty of functions still need implementing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant