Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PCRE2 10.45 release plan #532

Closed
NWilson opened this issue Oct 18, 2024 · 26 comments
Closed

PCRE2 10.45 release plan #532

NWilson opened this issue Oct 18, 2024 · 26 comments
Milestone

Comments

@NWilson
Copy link
Member

NWilson commented Oct 18, 2024

Hi,

Is there a roadmap for the next release? I get the sense that PCRE2 releases happen "as needed" rather than based on specific feature goals or a regular bi-annual cadence.

Are there things that Zoltan or Carlo are planning to get into the next release? Or will it just go out "whenever"?

From my point of view, I'm hoping to finish up my current draft PR on [a-z -- m] extended classes, which I would like to be in the next release. As a stretch goal, I will hopefully one day add support for (?[...]) classes, but that is dependent on when I get some free time to research the precise parsing rules that would be required for accurate Perl compatibility.

Beyond that, other things that I've suggested working on are purely non-observable cleanup or optimisation tasks (like OP_CHAR2, or simplifying/removing PT_CLIST); therefore I don't feel a particular sense of urgency to get them done in order to squeeze them into the next release.

We were hoping that v10.45 might perhaps be released before the end of 2024, if that's feasible. The NEWS file is certainly going to be a very fat one compared to the last couple of entries!

@PhilipHazel
Copy link
Collaborator

You are right about "as needed", though I have tried vaguely over the years to do two releases per year. In the light of the recent burst of activity I have been wondering when the next one should be. Because there have been so many substantial changes, I feel that we ought to put out a Release Candidate and encourage people to test. When I've done this in the past, I have always left at least one month between RC and actual release. Getting something done by the end of 2024 seems very tight.

I hope you (or someone else) will get the Perl $([ syntax working because I think I would feel uncomfortable releasing only the non-Perl extended class syntax. Once you have completed the other syntax I could take a look if nobody else can.

@NWilson
Copy link
Member Author

NWilson commented Oct 18, 2024

OK, I can step up to do the "(?[...])" support. It's just the code to convert the pattern string into the META codes that we need. I don't think the Perl syntax needs any special METAs, and can output them in the same layout as the UTS#18 syntax.

@NWilson
Copy link
Member Author

NWilson commented Oct 18, 2024

I appreciate your careful approach to stable releases, and we won't rush your schedule.

@NWilson NWilson added this to the 10.45-RC1 milestone Dec 6, 2024
@NWilson
Copy link
Member Author

NWilson commented Dec 8, 2024

@zherczeg @carenas @PhilipHazel

I am hoping to put out the 10.45-RC1 release before Christmas. That doesn't seem too rushed a schedule.

  • Test coverage is good
  • We have UTS#18 and Perl extended character classes, and working JIT support for them
  • The backlog looks healthy - not many tickets at all really which demand attention before we do a release

I have tagged open issues here, which I think should be sorted out before the 45-RC1: GitHub issues list

On top of that, I have some non-code "release" tasks to complete (read-through of docs, update ChangeLog, rehearse the release procedures).

Let me know if there's anything else that should be added as a must-do task for the RC1 - code tasks, or additional testing. Current testing will certainly involve building & running the unit tests on a variety of OSes.

@carenas
Copy link
Contributor

carenas commented Dec 8, 2024

  • The backlog looks healthy

problem is we don't usually have a backlog (at least not a visible one), but I think we have only a few blockers for a release IMHO.

is your suggestion to do an RC1 ASAP, a veiled way to ask for a "code freeze"?

@NWilson
Copy link
Member Author

NWilson commented Dec 8, 2024

It's an open way to ask for a "code freeze". I think we're all expecting a release soon. I want to know what you all think - are there any tasks which you think we should complete, before the next release?

@zherczeg
Copy link
Collaborator

zherczeg commented Dec 9, 2024

I am working on some eclass jit improvements at the moment. Smaller scope patches. For example, I want to remove PT_ANY from all matchers, since the parser handles it.

@PhilipHazel
Copy link
Collaborator

Sounds like a reasonable plan. I will let you have a list of contributors scraped from old ChangeLog files later today.

@carenas
Copy link
Contributor

carenas commented Dec 9, 2024

are there any tasks which you think we should complete, before the next release?

since we have never done a "code freeze", if you consider the requirement to be "a fix for a regression that will be otherwise introduced with this release", I think you already have those on your tags, but there might be more (which might not be technically a regression) like:

  • FTBFS in old Solaris with gcc < 5 (loosely connected to x86: improve SIMD detection zherczeg/sljit#257, and likely also affecting old Linux, like CentOS 7)
  • seems at least in AIX a generated snapshot wouldn't run its tests
  • there is an ugly workaround for a macOS libc bug (originally an stackoverflow question) that might need cleanup. This might also improve support in OpenBSD and Linux that use a UTF-8 C locale by default (like Alpine)

Those fixes are not forthcoming though, as I had been unable to secure enough "free" time to work on them (including some on your list)

@NWilson
Copy link
Member Author

NWilson commented Dec 9, 2024

seems at least in AIX a generated snapshot wouldn't run its tests

I saw that failure. It looked like you were running a tarball from "make dist", and then doing "make check". There was indeed a bug, due to the new testdata files not being include in "make dist", and I found & fixed it just yesterday in #603.

Hopefully this one is sorted.

there is an ugly workaround for a macOS libc bug thst might need cleanup. This might also improve support in OpenBSD and Linux that use a UTF-8 C locale by default (like Alpine)

That sounds worrying. Is there a ticket for it?

@NWilson
Copy link
Member Author

NWilson commented Dec 20, 2024

@carenas

there is an ugly workaround for a macOS libc bug thst might need cleanup. This might also improve support in OpenBSD and Linux that use a UTF-8 C locale by default (like Alpine)

I've tried hunting for this issue in the GitHub issues tracker, but I can't find it. I guess I'm using wrong keywords to search? I can look into the issue, if you point me to it.

@NWilson
Copy link
Member Author

NWilson commented Dec 23, 2024

@zherczeg I have no further PRs to open; the two at the moment are my last two for 10.45-RC1.

There's absolutely no rush - have a great Christmas!

@MatthewVernon
Copy link
Contributor

Where are we with the 10.45 release, please? I ask because Debian has just announced the freeze dates for the next release, and if we're going to get 10.45 into that release it'd be good to upload it in good time before the freeze so people can test it and suchlike...

@NWilson
Copy link
Member Author

NWilson commented Jan 23, 2025

We're on track for 10.45. The 10.45-RC1 went out just after Christmas, and I announced that there would be a month for users to provide feedback before I'd release 10.45.

So far... there hasn't been any functional feedback, except for reports of a few compiler warnings.

If nothing is reported, then I'll release 10.45 at the start of Feb.

Is it possible to push RC1 out to Debian sid? Are there other channels that we can use to get pre-release feedback? Or perhaps the lack of bug reports is simply because there aren't any regressions!

@MatthewVernon
Copy link
Contributor

Usually I push a new pcre2 package to the "experimental" distribution, which means the autobuilders, CI, and so on can run and invite people to test it (which has showed up issues before e.g. #1070195), I've not tended to bother with release candidates before, but if you think it would be particularly useful I could try and do so.

@NWilson
Copy link
Member Author

NWilson commented Jan 23, 2025

I don't want to make any extra work for people, and I don't know what the best practice would be (as a new maintainer). I feel if the RC1 is well-tested, then the final release will be good. If clients (like Debian or PHP) don't test the RC1, then we're likely to need a quick 10.46 follow-up release instead.

Philip was realistic, and usually didn't bother with an RC, because it doesn't generate much feedback.

@zherczeg
Copy link
Collaborator

We had RCs in the past, but only a few people tested them, and they were quick, so we never waited longer than two weeks before the final release.

On the other hand, we also never did follow-up releases just because people found some bugs. Maintainers have a way to cherry-pick fixes in their repositories.

@zherczeg
Copy link
Collaborator

(note: now @NWilson is the release guy, so if he wants to change old habits, he has a right to do it)

@NWilson
Copy link
Member Author

NWilson commented Jan 23, 2025

We had RCs in the past, but only a few people tested them, and they were quick, so we never waited longer than two weeks before the final release.

The last RC was 10.43-RC1, on Dec 28, followed by 10.43 on Feb 16. Almost exactly the same proposed dates as for the current release!

On the other hand, we also never did follow-up releases just because people found some bugs. Maintainers have a way to cherry-pick fixes in their repositories.

10.39 was three weeks after 10.38 for a JIT bug. 10.42 was one week after 10.41 for an issue in pcre2posix.h.

I think my plan is unchanged:

  • I'll release 10.45 at the start of Feb
  • Any bugs with the RC1 which are reported in Jan will be fixed in the actual release
  • Any follow-up release will be scheduled on a case-by-case basis depending on whether any bugs are reported, and their severity
  • I probably won't do an RC for every release in future. But since we have a 10.45-RC1... it would be nice if downstream packagers could run their test suites on it!

@MatthewVernon
Copy link
Contributor

OK, well I've pushed 10.45~rc1-1 (the odd spelling is to make sure that 10.45-1 sorts correctly) to Debian experimental, which should at least get CI and suchlike run against it, and interested maintainers of packages that do a lot with pcre2 might be inspired to run tests too.

@MatthewVernon
Copy link
Contributor

Build results look good (assuming riskv64 gets there eventually); I've set off a CI run with glib2.0 (where we've seen incompatibility issues in the past).

@MatthewVernon
Copy link
Contributor

CI also looks good (again, except riskv64 where we're still waiting for the buildd to go, and loong64 where CI hasn't run).

@carenas
Copy link
Contributor

carenas commented Jan 24, 2025

OK, well I've pushed 10.45~rc1-1 (the odd spelling is to make sure that 10.45-1 sorts correctly) to Debian experimental

Does this mean there is a chance that Debian 13 (which should freeze mid March) might include 10.45 if the final release is out by then?

@MatthewVernon
Copy link
Contributor

Maybe :)

Assuming no show-stopping bugs are found, then a package of 10.45 would need to have made it into the "testing" distribution before the freeze; that means it needs to have been uploaded to "unstable" (not "experimental" where I've put the RC) at least 10 days before that. And I'm away for a chunk of time in mid-February.

But if 10.45 is released early Feb, and I can get it into unstable before I go away, then there's a very good chance of 10.45 being in the Debian trixie release.

@zherczeg
Copy link
Collaborator

I would be happy if it would happen.

@MatthewVernon
Copy link
Contributor

Yes, I'm keen that we get 10.45 into Debian trixie, and it feels like it should be doable.
And we've even found a bug from the RC now... :) #673

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants