Make create, update, build and run automatically execute any of the earlier steps which are out of date

[mhsmith]

Describe the bug

On Windows, briefcase create immediately followed by briefcase run skips the build step:

(bw-dev) smith@xanadu:~/git/beeware/hello$ briefcase create

Application 'helloworld' already exists; overwrite [y/N]? y

[helloworld] Removing old application bundle...

[helloworld] Generating application template...
Using app template: https://github.com/beeware/briefcase-windows-app-template.git, branch 3.8
Using existing template (sha e5e330faf36c2490c74a5459c30951b516b41fa1, updated Mon Aug  1 15:58:39 2022)
Assigning helloworld an application GUID of 93bbf2ae-12d0-580c-aeb0-965dc4c04b69

[helloworld] Installing support package...
Using support package https://briefcase-support.org/python?platform=windows&version=3.8&arch=amd64
... using most recent revision
python-3.8.3-embed-amd64.zip already downloaded
Unpacking support package... done

[helloworld] Installing dependencies...
Collecting toga-core>=0.3.0.dev35
  Using cached toga_core-0.3.0.dev35-py3-none-any.whl (491 kB)
Collecting toga-winforms>=0.3.0.dev35
  Using cached toga_winforms-0.3.0.dev35-py3-none-any.whl (326 kB)
Collecting travertino>=0.1.3
  Using cached travertino-0.1.3-py3-none-any.whl (15 kB)
Collecting pythonnet>=3.0.0a1
  Using cached pythonnet-3.0.0rc4-py3-none-any.whl (250 kB)
Collecting clr-loader>=0.1.7
  Using cached clr_loader-0.1.7-py3-none-any.whl (45 kB)
Collecting cffi>=1.13
  Using cached cffi-1.15.1-cp38-cp38-win_amd64.whl (178 kB)
Collecting pycparser
  Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Installing collected packages: travertino, pycparser, toga-core, cffi, clr-loader, pythonnet, toga-winforms
Successfully installed cffi-1.15.1 clr-loader-0.1.7 pycparser-2.21 pythonnet-3.0.0rc4 toga-core-0.3.0.dev35 toga-winforms-0.3.0.dev35 travertino-0.1.3
Installing app dependencies... done

[helloworld] Installing application code...
Installing src/helloworld... done

[helloworld] Installing application resources...
Installing src/helloworld/resources/helloworld.ico as application icon... done

[helloworld] Created windows\app\Hello World
(bw-dev) smith@xanadu:~/git/beeware/hello$ briefcase run

[helloworld] Starting app...
===========================================================================
Log started: 2022-08-01 18:28:22Z
PreInitializing Python runtime...
PythonHome: C:\Users\smith\cygwin\git\beeware\hello\windows\app\Hello World\src
PYTHONPATH:
- C:\Users\smith\cygwin\git\beeware\hello\windows\app\Hello World\src\python38.zip
- C:\Users\smith\cygwin\git\beeware\hello\windows\app\Hello World\src
- C:\Users\smith\cygwin\git\beeware\hello\windows\app\Hello World\src\app_packages
- C:\Users\smith\cygwin\git\beeware\hello\windows\app\Hello World\src\app
Configure argc/argv...
Initializing Python runtime...
Running app module: stub
---------------------------------------------------------------------------
Traceback (most recent call last):
  File "runpy.py", line 185, in _run_module_as_main
  File "runpy.py", line 138, in _get_module_details
runpy._Error: No module named stub

During handling of the above exception, another exception occurred:

SystemExit: \Hello World.exe: No module named stub

This is because RunCommand contains the code:

        elif not binary_file.exists():
            state = self.build_command(app, **options)

But on Windows, the executable comes with the template, so it always exists. So briefcase create immediately followed by briefcase run will never do the build step, which is responsible for setting the main module name in the EXE's version information.

Expected behavior
briefcase run should do a build if it hasn't already happened.

Environment:

• Operating System: Windows 10
• Python version: 3.8
• Software versions:
  • Briefcase: 8df2d11

[freakboy3742]

This has been observed on Windows, but variations on this theme exist on almost every platform (e.g., on macOS, the binary exists in the template; the build step does adhoc signing, so create then run will result in an unsigned app that won't run on M1 hardware).

There's also some overlap with #411; in that case, an incomplete create hasn't been identified, and therefore build fails.

The more general problem that needs a solution is "dirty" state tracking. My best thought at present is that the last step in create, update, build etc should be to write a state file in the root of the bundle path that tracks the date/time and/or a useful hash to identify when create/build/update was last successfully completed. The app can then check whether build has been executed, rather than purely looking for the existence of the executable.

It could also potentially remove the need for explicit calls to update. By checking for modification dates on the sources folders, or the hash of requires definitions, or changes to the support_package or template declarations, we'd be able to identify when an update is required, and automatically perform that update.

[mhsmith]

It could also potentially remove the need for explicit calls to update. By checking for modification dates on the sources folders, or the hash of requires definitions, or changes to the support_package or template declarations, we'd be able to identify when an update is required, and automatically perform that update.

That's a great idea: I'm constantly forgetting to run update, and I'm sure the users are as well.

[jgilmore]

Particularly unexpected for me (a new user, just going through the tutorials) was that "briefcase update" does NOT UPDATE mobile or other packages, JUST the one for the current system type.

I had to figure out myself that I needed to run "briefcase update android" and THEN "briefcase build android" as separate commands. It not only doesn't update automatically when changes are detected, but update must be run for each platform separately.

That's pretty crazy. Build processes are supposed to be more automatic than that these days, aren't they?

[freakboy3742]

@jgilmore An interesting (and not unreasonable) perspective; however, it's a moot point if the idea proposed by this ticket is addressed. If an update is automatically performed when required as a result of invoking run/build on a specific platform, then "update all platforms" isn't necessary.

FWIW, the current behaviour is an optimization. I could easily have a dozen active platforms (macOS, Windows, iOS, Android, web, Flatpak, multiple linux distributions,... ), and update isn't a "zero time" operation. If update updated every platform, it could take quite some time. It also breaks the symmetry with other commands - run macOS runs on macOS. build macOS builds on macOS - and so, update macOS updates on macOS.