Evaluate symlinks when comparing $BUILD_DIR against /app

[kamaln7]

This updates the $BUILD_DIR check to evaluate symlinks before comparing to /app.

In our build environment we have /app symlinked to /workspace to accommodate other buildpacks. $BUILD_DIR is set to /workspace. That causes this buildpack to fail to build because the paths aren't equal so the .heroku/* symlinks are created and in a later step the buildpack tries to install python over the symlink.

2020-07-08T15:36:14.47487514Z -----> Installing python-3.6.10
2020-07-08T15:36:14.493172053Z mkdir: cannot create directory ‘.heroku/python’: File exists
2020-07-08T15:36:31.569316508Z  !     Requested runtime (python-3.6.10) is not available for this stack (heroku-18).
2020-07-08T15:36:31.569494894Z  !     Aborting.  More info: 

[kamaln7]

Hmmm I feel like Travis CI is failing because of an unrelated issue?

$ bundle exec hatchet ci:setup
/home/travis/build/heroku/heroku-buildpack-python/vendor/bundle/ruby/2.6.0/gems/heroku_hatchet-5.0.3/etc/ci_setup.rb:11:in `fetch': key not found: "HEROKU_API_USER" (KeyError)
	from /home/travis/build/heroku/heroku-buildpack-python/vendor/bundle/ruby/2.6.0/gems/heroku_hatchet-5.0.3/etc/ci_setup.rb:11:in `block in <main>'
	from /home/travis/build/heroku/heroku-buildpack-python/vendor/bundle/ruby/2.6.0/gems/heroku_hatchet-5.0.3/etc/ci_setup.rb:8:in `open'
	from /home/travis/build/heroku/heroku-buildpack-python/vendor/bundle/ruby/2.6.0/gems/heroku_hatchet-5.0.3/etc/ci_setup.rb:8:in `<main>'
/home/travis/build/heroku/heroku-buildpack-python/vendor/bundle/ruby/2.6.0/gems/heroku_hatchet-5.0.3/bin/hatchet:174:in `cmd': Command /home/travis/build/heroku/heroku-buildpack-python/vendor/bundle/ruby/2.6.0/gems/heroku_hatchet-5.0.3/etc/ci_setup.rb failed: (RuntimeError)
== Setting Up CI ==
	from /home/travis/build/heroku/heroku-buildpack-python/vendor/bundle/ruby/2.6.0/gems/heroku_hatchet-5.0.3/bin/hatchet:35:in `block in <class:HatchetCLI>'
	from /home/travis/build/heroku/heroku-buildpack-python/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor/command.rb:27:in `run'
	from /home/travis/build/heroku/heroku-buildpack-python/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor/invocation.rb:126:in `invoke_command'
	from /home/travis/build/heroku/heroku-buildpack-python/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor.rb:387:in `dispatch'
	from /home/travis/build/heroku/heroku-buildpack-python/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor/base.rb:466:in `start'
	from /home/travis/build/heroku/heroku-buildpack-python/vendor/bundle/ruby/2.6.0/gems/heroku_hatchet-5.0.3/bin/hatchet:179:in `<top (required)>'
	from /home/travis/build/heroku/heroku-buildpack-python/vendor/bundle/ruby/2.6.0/bin/hatchet:23:in `load'
	from /home/travis/build/heroku/heroku-buildpack-python/vendor/bundle/ruby/2.6.0/bin/hatchet:23:in `<main>'
	from /home/travis/build/heroku/heroku-buildpack-python/vendor/bundle/ruby/2.6.0/bin/ruby_executable_hooks:24:in `eval'
	from /home/travis/build/heroku/heroku-buildpack-python/vendor/bundle/ruby/2.6.0/bin/ruby_executable_hooks:24:in `<main>'
The command "bundle exec hatchet ci:setup" exited with 1.

key not found: "HEROKU_API_USER" (KeyError)

    file.write <<-EOF
machine git.heroku.com
login #{ENV.fetch('HEROKU_API_USER')}
password #{ENV.fetch('HEROKU_API_KEY')}
EOF

[edmorley]

@kamaln7 Hi! Thank you for the PR. The reason Travis is failing is since those credentials are only set for non-fork PRs.

This is on my list to review/trigger a full CI run with creds next week - in the meantime could you rebase on master and fix the merge conflict in CHANGELOG.MD? :-)

[kamaln7]

Thank you, @edmorley! That explains it :)

[edmorley]

@kamaln7 Hi! Just wanted to let you know I haven't forgotten about this, but that #1007 and related changes took up most of my time this week. I have this down on my list to review next week.

[edmorley]

Hmm so the unit tests fail on Cedar-14 (eg https://travis-ci.com/github/heroku/heroku-buildpack-python/jobs/369755972), since for Ubuntu 14.04 coreutils doesn't include realpath and instead the realpath package must be installed explicitly.

Our options are to either update https://github.com/heroku/stack-images to install the realpath package (and wait for the next stack image release), or else switch to readlink -f instead.

Researching the difference between the two I found:
https://unix.stackexchange.com/questions/136494/whats-the-difference-between-realpath-and-readlink-f

...which muddies the water further by suggesting that readlink -e is in fact the more accurate replacement for realpath.

[kamaln7]

Ah thanks @edmorley I admit that I had only tested this on 18.04. I agree that if we should avoid modifying the stack images if we can. readlink -e makes sense, if either /app or $BUILD_DIR don't exist we would want the build to fail because that's a bigger problem there 😅 I can make this change Monday morning if that sounds good to you!

[edmorley]

That change sounds good - thank you! Marking this as changes required for now to remove it from my pending reviews list :-)

[edmorley]

I've opened heroku/base-images/pull/176 to add realpath to Cedar-14 (since it seems useful to have it available on that stack for consistency between stacks, regardless of what we choose here).

[kamaln7]

Hi @edmorley very sorry for the delay here. I've just pushed a commit updating the code to use readlink -e. heroku/base-images#176 sounds good 👍🏼

[edmorley]

@kamaln7 Sorry for the delay here. In the meantime, the Cedar-14 stack has been sunset, meaning all buildable stacks now support realpath. As such, I've reverted the commit to switch to readlink. I've rebased the PR, updated another BUILD_DIR comparison to use realpath too, and updated the changelog entry for the new style. I've triggered CI (now using Circle) which is passing.

[edmorley]

Meant to say - in the future I hope to remove the BUILD_DIR to /app comparisons entirely, once the Heroku build system begins building in /app (meaning the build time and run time paths no longer differ). Rolling out this change is slow work since there are buildpacks that will break with such a change, so it's required a bunch of outreach.

[kamaln7]

Hey @edmorley, thanks for the context & the additional fix!