Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dev/core#2316 - Symfony EventDispatcher dispatch() signature kerfuffle - try 3 #24132

Merged
merged 14 commits into from
Nov 11, 2022

Conversation

demeritcowboy
Copy link
Contributor

@demeritcowboy demeritcowboy commented Aug 3, 2022

Overview

https://lab.civicrm.org/dev/core/-/issues/2316

This is a reworking of #19960 to handle the change in the signature of EventDispatcher::dispatch() across symfony v4/5/6.

Before

Calls to dispatch() all give warnings in drupal 9 because symfony reversed the order of arguments. Then they fail in drupal 10. Also, the Event class disappears in symfony 6, so can no longer reference it.

It's difficult to see these warnings in drupal 9 because it annoyingly uses @trigger_error with the @, which silences the warning. But you can see them in drupal's own test environment - see e.g. this run.

After

No notices. Works in drupal 10.

Technical Details

  • Keep CiviEventDispatcher as close as possible to what it was, so that its signature stays compatible.
  • Replace Event with GenericHookEvent.

Comments

Most of the actual change is in the first 2 commits.

The last commit is because in symfony 4.3 they introduced some fancy optimization code. Something about the test runs makes it cause memory corruption and a segfault. Using a dummy subclass skips the optimization. While it would be nice to track the issue down,

  • (a) I've spent more time than I want to already,
  • (b) It didn't use to do the optimization and nobody complained,
  • (c) my own tests are inconclusive about the performance increase of closures vs real functions in this context, and you'd also need to take into account the extra memory that optimization uses when civi has thousands (millions?) of listeners during a test run.
  • (d) I haven't seen it come up in real life poking around on civi+drupal 10.

@civibot
Copy link

civibot bot commented Aug 3, 2022

(Standard links)

@civibot civibot bot added the master label Aug 3, 2022
@demeritcowboy
Copy link
Contributor Author

demeritcowboy commented Aug 3, 2022

Hmm, memory error still happens

not ok 1379 - Error: E2E\Core\AssetBuilderTest::testGetUrl_cached with data set #2 ('square.js', array(12), 'application/javascript', 'var square=144;')
not ok 1380 - Error: E2E\Core\AssetBuilderTest::testGetUrl_uncached with data set #0 ('square.txt', array(11), 'text/plain', 'Square: 121')
not ok 1381 - Error: E2E\Core\AssetBuilderTest::testGetUrl_uncached with data set #1 ('square.txt', array(12), 'text/plain', 'Square: 144')
not ok 1382 - Error: E2E\Core\AssetBuilderTest::testGetUrl_uncached with data set #2 ('square.js', array(12), 'application/javascript', 'var square=144;')
not ok 1383 - Error: E2E\Core\AssetBuilderTest::testInvalid
Segmentation fault

@demeritcowboy
Copy link
Contributor Author

Interesting. DaveEventDispatcher makes those test fails go away by removing the listener optimizer introduced in 4.3+. Some possible reasons:

  1. A bug in this PR that gets exposed by the optimizer.
  2. Civi is doing something weird.
  3. The optimizer has a bug/limitation that Civi has found.
  4. A bug in php.
  5. Other?

I'm not too concerned about the fails in the recent run - they all seem network-related and are likely either caused by the debug output I've added confusing the network response, or just a network glitch. I can easily test the former by removing the debug output. Let's do that...

@MegaphoneJon
Copy link
Contributor

I just tried civibuild reinstall on a D7 buildkit and got the error below. Is this the right PR to be tracking about this?

> @php -r 'require_once '\''/home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/vendor/autoload.php'\''; Civi\CompilePlugin\TaskTransfer::import(); \CCL\Tasks::template($GLOBALS[\Civi\CompilePlugin\TaskTransfer::GLOBAL_VAR]);'
PHP Fatal error:  Uncaught Error: Class 'Composer\InstalledVersions' not found in /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/guzzle_php81_shim.php:6
Stack trace:
#0 /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/vendor/composer/autoload_real.php(73): require()
#1 /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/vendor/composer/autoload_real.php(63): composerRequire50095186591b60b061f2519f2891543e('59969633dcdb4ee...', '/home/jon/local...')
#2 /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/vendor/autoload.php(7): ComposerAutoloaderInit50095186591b60b061f2519f2891543e::getLoader()
#3 Command line code(1): require_once('/home/jon/local...')
#4 {main}
  thrown in /home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/guzzle_php81_shim.php on line 6
Script @php -r 'require_once '\''/home/jon/local/civicrm-buildkit/build/dmaster/web/sites/all/modules/civicrm/vendor/autoload.php'\''; Civi\CompilePlugin\TaskTransfer::import(); \CCL\Tasks::template($GLOBALS[\Civi\CompilePlugin\TaskTransfer::GLOBAL_VAR]);' handling the shell-runner event returned with error code 255
Subcommand @composer compile  returned with error code 255

@seamuslee001
Copy link
Contributor

@MegaphoneJon no that would be a separate ticket / PR but I would check your composer version see also civicrm/civicrm-buildkit#715

@demeritcowboy demeritcowboy force-pushed the eventdispatcher3 branch 2 times, most recently from 72ece74 to 949358b Compare August 19, 2022 00:19
@demeritcowboy demeritcowboy changed the title [WIP] dev/core#2316 - Symfony EventDispatcher dispatch() signature kerfuffle - try 3 dev/core#2316 - Symfony EventDispatcher dispatch() signature kerfuffle - try 3 Aug 19, 2022
@colemanw
Copy link
Member

I've reviewed the code and I think it's best we merge this now while still early in the release cycle.

@colemanw colemanw merged commit 0c9af4d into civicrm:master Nov 11, 2022
@demeritcowboy
Copy link
Contributor Author

Thanks for giving this a push. Fingers crossed🤞
There was a suggestion earlier to add a warning in dispatch() if someone is directly subclassing Event. It might be very noisy but will put on the todo list.

@demeritcowboy demeritcowboy deleted the eventdispatcher3 branch November 11, 2022 22:31
@kcristiano
Copy link
Member

@demeritcowboy @colemanw

[12-Nov-2022 09:23:11 America/New_York] PHP Fatal error:  Uncaught Error: Class 'Civi\Api4\Subscriber\AutocompleteSubscriber' not found in /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/ext/afform/core/afform.php:60
Stack trace:
#0 /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/CRM/Utils/Hook.php(271): afform_civicrm_config()
#1 /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/CRM/Utils/Hook/WordPress.php(136): CRM_Utils_Hook->runHooks()
#2 /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/Civi/Core/CiviEventDispatcher.php(306): CRM_Utils_Hook_WordPress->invokeViaUF()
#3 /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/vendor/symfony/event-dispatcher/EventDispatcher.php(251): Civi\Core\CiviEventDispatcher::delegateToUF()
#4 /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/vendor/symfony/event-dispatcher/EventDispatcher.php(73): Symfony\Component\EventDispatcher\EventDispatcher->callListeners()
#5 /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/Civi/Cor in /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/ext/afform/core/afform.php on line 60
[12-Nov-2022 14:23:11 UTC] PHP Fatal error:  Uncaught Error: Class 'Civi\Api4\Subscriber\AutocompleteSubscriber' not found in /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/ext/afform/core/afform.php:60
Stack trace:
#0 /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/CRM/Utils/Hook.php(271): afform_civicrm_config()
#1 /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/CRM/Utils/Hook/WordPress.php(136): CRM_Utils_Hook->runHooks()
#2 /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/Civi/Core/CiviEventDispatcher.php(306): CRM_Utils_Hook_WordPress->invokeViaUF()
#3 /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/vendor/symfony/event-dispatcher/EventDispatcher.php(251): Civi\Core\CiviEventDispatcher::delegateToUF()
#4 /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/vendor/symfony/event-dispatcher/EventDispatcher.php(73): Symfony\Component\EventDispatcher\EventDispatcher->callListeners()
#5 /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/Civi/Cor in /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/ext/afform/core/afform.php on line 60

The plugin show enabled. Hmm

  • Attempt to enable any WP plugin that also uses Symfony
  • Fata Error
[12-Nov-2022 14:27:24 UTC] PHP Fatal error:  Uncaught Error: Class 'Civi\Api4\Subscriber\AutocompleteSubscriber' not found in /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/ext/afform/core/afform.php:60
Stack trace:
#0 /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/CRM/Utils/Hook.php(271): afform_civicrm_config()
#1 /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/CRM/Utils/Hook/WordPress.php(136): CRM_Utils_Hook->runHooks()
#2 /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/Civi/Core/CiviEventDispatcher.php(306): CRM_Utils_Hook_WordPress->invokeViaUF()
#3 /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/vendor/symfony/event-dispatcher/EventDispatcher.php(251): Civi\Core\CiviEventDispatcher::delegateToUF()
#4 /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/vendor/symfony/event-dispatcher/EventDispatcher.php(73): Symfony\Component\EventDispatcher\EventDispatcher->callListeners()
#5 /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/Civi/Cor in /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/ext/afform/core/afform.php on line 60
[12-Nov-2022 14:27:25 UTC] PHP Fatal error:  Uncaught Error: Class 'Civi\Api4\Subscriber\AutocompleteSubscriber' not found in /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/ext/afform/core/afform.php:60
Stack trace:
#0 /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/CRM/Utils/Hook.php(271): afform_civicrm_config()
#1 /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/CRM/Utils/Hook/WordPress.php(136): CRM_Utils_Hook->runHooks()
#2 /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/Civi/Core/CiviEventDispatcher.php(306): CRM_Utils_Hook_WordPress->invokeViaUF()
#3 /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/vendor/symfony/event-dispatcher/EventDispatcher.php(251): Civi\Core\CiviEventDispatcher::delegateToUF()
#4 /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/vendor/symfony/event-dispatcher/EventDispatcher.php(73): Symfony\Component\EventDispatcher\EventDispatcher->callListeners()
#5 /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/Civi/Cor in /srv/www/buildkit/wpempty/web/wp-content/plugins/civicrm/civicrm/ext/afform/core/afform.php on line 60

I thought maybe it was the download package, so I rebuilt locally with distmaker. Same results.

wpmaster built earlier this week works without issue.

Let me know if we want a new issue as well on this.

@demeritcowboy
Copy link
Contributor Author

Ok I'll take a look.

@demeritcowboy
Copy link
Contributor Author

I don't think it's this PR because I can reproduce it with 5.56. I know you said it worked last week but I also tried it at http://demo-108-2i2k2.test-3.civicrm.org:8001 which is master and it works ok there, so it maybe depends on how the site is built. I can "fix" it with this patch but that's a giant sledgehammer and it would be better to figure out what is clearing the classloader. There's a lot of clearing going on - it rebuilds it 3 times but then somehow it doesn't contain afform prefixes at the time it calls runHooks.

--- a/CRM/Utils/Hook/WordPress.php
+++ b/CRM/Utils/Hook/WordPress.php
     $this->buildModuleList();

     // Call runHooks the same way Drupal does
+    \CRM_Extension_System::singleton()->getClassLoader()->refresh();
     $moduleResult = $this->runHooks(
       $this->allModules,
       $fnSuffix,

@colemanw
Copy link
Member

@demeritcowboy does this help? #24962

@kcristiano
Copy link
Member

@demeritcowboy I'll rebuild 5.56 (RC) now and see

@kcristiano
Copy link
Member

OK - I can reproduce in a new 5.55.1 install as well - so it's 100% not this. Maybe I did an upgrade on master last week - sorry about taking us down the wrong path

@colemanw
Copy link
Member

@kcristiano can you try with #24962?

@demeritcowboy
Copy link
Contributor Author

I just tried the patch in #24962 - works to solve the install.

@kcristiano
Copy link
Member

@colemanw @demeritcowboy Can you also test 5.55.1 - I get a fatal when enabling afform on new installs. A modified version of #24962 does fix it

diff --git a/ext/afform/core/Civi/Api4/Subscriber/AutocompleteSubscriber.php b/ext/afform/core/Civi/Api4/Subscriber/AutocompleteSubscriber.php
index 685a99c95..9f88e3cb2 100644
--- a/ext/afform/core/Civi/Api4/Subscriber/AutocompleteSubscriber.php
+++ b/ext/afform/core/Civi/Api4/Subscriber/AutocompleteSubscriber.php
@@ -11,6 +11,7 @@
 
 namespace Civi\Api4\Subscriber;
 
+use Civi\Core\Service\AutoService;
 use Civi\Afform\FormDataModel;
 use Civi\API\Events;
 use Civi\Api4\Afform;
@@ -18,8 +19,10 @@ use Symfony\Component\EventDispatcher\EventSubscriberInterface;
 
 /**
  * Preprocess api autocomplete requests
+ * @service
+ * @internal
  */
-class AutocompleteSubscriber implements EventSubscriberInterface {
+class AfformAutocompleteSubscriber extends AutoService implements EventSubscriberInterface {
 
   /**
    * @return array
@@ -34,7 +37,7 @@ class AutocompleteSubscriber implements EventSubscriberInterface {
    * @param \Civi\API\Event\PrepareEvent $event
    *   API preparation event.
    */
-  public function onApiPrepare(\Civi\API\Event\PrepareEvent $event) {
+  public function onApiPrepare(\Civi\API\Event\PrepareEvent $event): void {
     $apiRequest = $event->getApiRequest();
     if (is_object($apiRequest) && is_a($apiRequest, 'Civi\Api4\Generic\AutocompleteAction')) {
       $formName = $apiRequest->getFormName();
diff --git a/ext/afform/core/afform.php b/ext/afform/core/afform.php
index 9aae5d998..7f5473923 100644
--- a/ext/afform/core/afform.php
+++ b/ext/afform/core/afform.php
@@ -56,7 +56,6 @@ function afform_civicrm_config(&$config) {
   $dispatcher->addListener('hook_civicrm_alterAngular', ['\Civi\Afform\AfformMetadataInjector', 'preprocess']);
   $dispatcher->addListener('hook_civicrm_check', ['\Civi\Afform\StatusChecks', 'hook_civicrm_check']);
   $dispatcher->addListener('civi.afform.get', ['\Civi\Api4\Action\Afform\Get', 'getCustomGroupBlocks']);
-  $dispatcher->addSubscriber(new \Civi\Api4\Subscriber\AutocompleteSubscriber());
 
   // Register support for email tokens
   if (CRM_Extension_System::singleton()->getMapper()->isActiveModule('authx')) {
``

@herbdool
Copy link
Contributor

herbdool commented Nov 20, 2022

Not sure if this is related, but upgrading from 5.54.0 directly to 5.55.2 gives me this fatal error:

In ContainerBuilder.php line 1030:
You have requested a non-existent service "afform_scanner".

But with the same site, upgrading from 5.54.0 (or could have been 5.54.1). to 5.55.0 worked.

@demeritcowboy
Copy link
Contributor Author

@herbdool This PR is only in 5.57. Do you have a stack trace from the fail?

@herbdool
Copy link
Contributor

@demeritcowboy I meant to post this on the backport to 5.55.

@demeritcowboy
Copy link
Contributor Author

Ah ok for #24964. Cool.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants