.env processing with symfony using frankenphp worker & runtime

[PhilETaylor]

What happened?

This has been talked about this before with no concrete solution or reproducer. The variables_order = EGPCS talked about before doesn't seem to fix this.

• Inconsistency in environment variables #972
• [Help] $_ENV gets overridden #1061 (comment)

Finally got a night to play and produce a reproducer to explain the issue for you.

Assuming that the Frankenphp runtime is a drop in replacement, and should "just work" then this needs addressing.

The repo at https://github.com/PhilETaylor/env-reproducer can be

1. git cloned
2. composer install
3. docker-compose up (check FRANKENPHP_CONFIG and APP_RUNTIME commented in/out state in docker-compose.yml)
4. visit https://localhost/

There is one controller action, at / which gives a var_dump of $_ENV The reproducer is nothing more than composer require runtime/frankenphp-symfony on top of symfony new . --version="7.2.x" --webapp and then adding HTTP_PROXY to .env, and adding the runtime to the docker-compose

The problem is that when running with:

      FRANKENPHP_CONFIG: "worker ./public/index.php"
      APP_RUNTIME: "Runtime\\FrankenPhpSymfony\\Runtime"

the $_ENV is missing most of the values from .env - this means the app breaks because the $_ENV is not correctly populated

The values from .env are in the $_SERVER array, but they are not in the $_ENV array when running the frankenphp runtime.

If you then comment out the worker mode in the docker-compose, and then restart your container (which then just runs image: dunglas/frankenphp) then compare the dumped $_ENV from the https://localhost/ you can see the difference, and you can see symfony values such as MESSENGER_TRANSPORT_DSN and my own test of HTTP_PROXY

Without Worker Mode Enabled:

array(31) {
  ["HOSTNAME"]=>
  string(12) "72dc79c75bd5"
  ["PHP_INI_DIR"]=>
  string(18) "/usr/local/etc/php"
  ["HOME"]=>
  string(5) "/root"
  ["GODEBUG"]=>
  string(10) "cgocheck=0"
  ["PHP_LDFLAGS"]=>
  string(12) "-Wl,-O1 -pie"
  ["ENV"]=>
  string(3) "dev"
  ["PHP_CFLAGS"]=>
  string(83) "-fstack-protector-strong -fpic -fpie -O2 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64"
  ["CADDY_SERVER_EXTRA_DIRECTIVES"]=>
  string(17) "tls internal
log
"
  ["PHP_VERSION"]=>
  string(5) "8.4.4"
  ["GPG_KEYS"]=>
  string(122) "AFD8691FDAEDF03BDF6E460563F15A9B715376CA 9D7F99A0CB8F05C8A6958D6256A97AF7600A39A6 0616E93D95AF471243E26761770426E17EBBB3DD"
  ["PHP_CPPFLAGS"]=>
  string(83) "-fstack-protector-strong -fpic -fpie -O2 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64"
  ["PHP_ASC_URL"]=>
  string(54) "https://www.php.net/distributions/php-8.4.4.tar.xz.asc"
  ["PHP_URL"]=>
  string(50) "https://www.php.net/distributions/php-8.4.4.tar.xz"
  ["PATH"]=>
  string(60) "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
  ["XDG_CONFIG_HOME"]=>
  string(7) "/config"
  ["XDG_DATA_HOME"]=>
  string(5) "/data"
  ["APP_ENV"]=>
  string(3) "dev"
  ["PHPIZE_DEPS"]=>
  string(76) "autoconf 		dpkg-dev 		file 		g++ 		gcc 		libc-dev 		make 		pkg-config 		re2c"
  ["PWD"]=>
  string(4) "/app"
  ["PHP_SHA256"]=>
  string(64) "05a6c9a2cc894dd8be719ecab221b311886d5e0c02cb6fac648dd9b3459681ac"
  ["SERVER_NAME"]=>
  string(9) "localhost"
  ["SYMFONY_DOTENV_PATH"]=>
  string(9) "/app/.env"
  ["APP_SECRET"]=>
  string(32) "d18b6d021424276c6937ca8c6648b921"
  ["DATABASE_URL"]=>
  string(76) "postgresql://app:!ChangeMe!@127.0.0.1:5432/app?serverVersion=16&charset=utf8"
  ["MESSENGER_TRANSPORT_DSN"]=>
  string(31) "doctrine://default?auto_setup=0"
  ["MAILER_DSN"]=>
  string(11) "null://null"
  ["HTTP_PROXY"]=>
  string(14) "127.0.0.1:8888"
  ["SYMFONY_DOTENV_VARS"]=>
  string(69) "APP_SECRET,DATABASE_URL,MESSENGER_TRANSPORT_DSN,MAILER_DSN,HTTP_PROXY"
  ["APP_DEBUG"]=>
  string(1) "1"
  ["SHELL_VERBOSITY"]=>
  int(3)
  ["DOCTRINE_DEPRECATIONS"]=>
  string(7) "trigger"
}

With Worker Mode enabled

array(25) {
  ["HOSTNAME"]=>
  string(12) "15d133604fcd"
  ["PHP_INI_DIR"]=>
  string(18) "/usr/local/etc/php"
  ["HOME"]=>
  string(5) "/root"
  ["GODEBUG"]=>
  string(10) "cgocheck=0"
  ["PHP_LDFLAGS"]=>
  string(12) "-Wl,-O1 -pie"
  ["ENV"]=>
  string(3) "dev"
  ["PHP_CFLAGS"]=>
  string(83) "-fstack-protector-strong -fpic -fpie -O2 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64"
  ["CADDY_SERVER_EXTRA_DIRECTIVES"]=>
  string(17) "tls internal
log
"
  ["PHP_VERSION"]=>
  string(5) "8.4.4"
  ["GPG_KEYS"]=>
  string(122) "AFD8691FDAEDF03BDF6E460563F15A9B715376CA 9D7F99A0CB8F05C8A6958D6256A97AF7600A39A6 0616E93D95AF471243E26761770426E17EBBB3DD"
  ["PHP_CPPFLAGS"]=>
  string(83) "-fstack-protector-strong -fpic -fpie -O2 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64"
  ["PHP_ASC_URL"]=>
  string(54) "https://www.php.net/distributions/php-8.4.4.tar.xz.asc"
  ["PHP_URL"]=>
  string(50) "https://www.php.net/distributions/php-8.4.4.tar.xz"
  ["PATH"]=>
  string(60) "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
  ["FRANKENPHP_CONFIG"]=>
  string(25) "worker ./public/index.php"
  ["XDG_CONFIG_HOME"]=>
  string(7) "/config"
  ["XDG_DATA_HOME"]=>
  string(5) "/data"
  ["APP_ENV"]=>
  string(3) "dev"
  ["PHPIZE_DEPS"]=>
  string(76) "autoconf 		dpkg-dev 		file 		g++ 		gcc 		libc-dev 		make 		pkg-config 		re2c"
  ["PWD"]=>
  string(4) "/app"
  ["PHP_SHA256"]=>
  string(64) "05a6c9a2cc894dd8be719ecab221b311886d5e0c02cb6fac648dd9b3459681ac"
  ["APP_RUNTIME"]=>
  string(33) "Runtime\FrankenPhpSymfony\Runtime"
  ["SERVER_NAME"]=>
  string(9) "localhost"
  ["SHELL_VERBOSITY"]=>
  string(1) "3"
  ["DOCTRINE_DEPRECATIONS"]=>
  string(7) "trigger"
}

Visually the changes are:

Other interesting notes

With standard EGPCS when you load the reproducer at https://localhost/ the FIRST TIME your $_ENV contains ["DOCTRINE_DEPRECATIONS"]=> string(7) "trigger"

and on the SECOND page refresh its still there. And on the THIRD page refresh IT DISAPPEARS!!!!! STRANGE!!!

If you uncomment the variables_order=GPCS in the /php.ini in the reproducer and restart the container, even stranger, you ONLY get the DOCTRINE_DEPRECATIONS env var when dumping the $_ENV and no other values!?!?!? and on the THIRD PAGE RELOAD IT DISAPPEARS leaving an empty $_ENV array dumped.

Hope this helps.

Build Type

Docker (Debian Bookworm)

Worker Mode

Yes

Operating System

GNU/Linux

CPU Architecture

Apple Silicon

PHP configuration

see reproducer

Relevant log output

[AlliBalliBaba]

Does Symfony's .env populate the $_ENV on each request or just on startup? I think this might be tied to how FrankenPHP resets $_ENV between requests, I'll have a look at your reproducer. In the meantime I would recommend using $_SERVER instead when in worker mode (it's also more efficient).

If you uncomment the variables_order=GPCS in the /php.ini in the reproducer and restart the container, even stranger, you ONLY get the DOCTRINE_DEPRECATIONS env var when dumping the $_ENV and no other values!?!?!? and on the THIRD PAGE RELOAD IT DISAPPEARS leaving an empty $_ENV array dumped.

The $_ENV being empty is to be expected with GPCS . In this case it's only Symfony's .env that will write to the global array.

[PhilETaylor]

The whole point is that a worker runtime should be a drop in replacement and "just work".

With symfony there are also Environment Variable Processors helpers like ->proxy('%env(HTTP_PROXY)%') which compile and pass the values through, so simply moving to $_SERVER is not really an option.

Ironically, today in my live product (not the reproducer), when in worker mode, now global twig helpers, like the mercure and global symfony app twig objects are missing :( (as soon as Im not using worker mode, these work (and are working in production without worker mode))

[dunglas]

This may also be a bug in the Symfony FrankenPHP Runtime instead of in FrankenPHP.

For the helpers, could you please report to Symfony?

I'll try to take a look next week.

The worker mode tries to be as close as possible of the normal mode, but as it is using a totally different paradigm (not fire and forget anymore), both mode can and do slightly differ with edge cases like this one. Using $_SERVER is a Symfony best practice anyway.

[PhilETaylor]

This may also be a bug in the Symfony FrankenPHP Runtime instead of in FrankenPHP.

Yup, I only posted here because thats where the other conversations about the $_ENV were, but you are right it could be either, or both

For the helpers, could you please report to Symfony?

As soon as I can get a reproducer for this specific issue, yup I'll report it.

Does Symfony's .env populate the $_ENV on each request or just on startup?

https://symfony.com/doc/current/configuration.html#configuring-environment-variables-in-env-files states "The .env file is read and parsed on every request and its env vars are added to the $_ENV & $_SERVER PHP variables. Any existing env vars are never overwritten by the values defined in .env, so you can combine both."

[withinboredom]

I have a suspicious feeling that this is a weird bug with $_ENV vanishing during autoloading. I have a reproducer and have stepped through the php interpreter many times, but eventually put it on a shelf.

It only happens on zts builds.

I will open a php-src bug this weekend since I have a pretty solid reproducer now.

[AlliBalliBaba]

The issue seems to be that Symfony's .env only loads the environment once at startup. If $_ENV is later discovered in a different script, it triggers the autoglobals JIT and the original $_ENV gets reloaded and overwritten,

Symfony's .env will populate properly in worker mode if you add this configuration to the php.ini:
auto_globals_jit=Off

It might make sense to generally not flush $_ENV between requests in worker mode or at least prevent the autoglobals jit from triggering.

[PhilETaylor]

auto_globals_jit=Off seems to fix the twig globals, but doesnt seem to fix the env vars

If I remove all env vars from my symfony configs (like HTTP_PROXY in the httpclient) then I can, for the first time (auto_globals_jit=Off) load the pages of my application and navigate it in worker mode, until I reach a page that needs an env var, and then it dies again.