Merge tag 'v2020.06.02'

🔖 v2020.06.02

* Adds missing requests sub dependencies (kivy#2221)
* Bumps to Gradle 6.4.1 (kivy#2222)
* Bumps to Cython==0.29.19 (kivy#2220)
* Updates install and troubleshooting docs (kivy#2219)
* Bumps to Ubuntu 20.04 (kivy#2218)
* Attempt to improve the issue template (kivy#2217)
* Add `opencv_extras` recipe (kivy#2209)
* Split logic for build modes & debug symbols (kivy#2213)
* Troubleshoot SSL error (kivy#2205)
* Remove superfluous recipes fixes (kivy#2202)
* Add tests for hostpython3 recipe (kivy#2196)
* Fix for 'cannot find setuptools module' (kivy#2195)
* Rename `Hostpython3Recipe` class to camel case (kivy#2194)
* Fix `test_should_build` (kivy#2193)
* Add initial tests for python3 recipe (kivy#2192)
* PythonActivityUtil helper for unpacking data (kivy#2189)
* Fixes flake8 errors post update (kivy#2191)
* Share PythonUtil.java between bootstraps (kivy#2188)
* Java code linting using PMD 6.23.0 (kivy#2187)
* Deletes deprecated renpy Python{Activity,Service}.java (kivy#2186)
* Removes java concurrency/ folder (kivy#2185)
* Reuse common AssetExtract.java (kivy#2182)
* Use common Hardware.java (kivy#2183)
* Moves kamranzafar/ java directory to common/ (kivy#2184)
* Updates release documentation (kivy#2177)
* Fixes service only unittest loading (kivy#2181)
* Narrows some context manager scopes (kivy#2179)
* Downgrades to SDL2 2.0.9 (kivy#2180)
* Bump to SDL2 2.0.10 & extract .java from SDL2 tarball (kivy#2113)
* Adds pygame recipe (kivy#2164)
* Adds macOS install instructions (2165)
* Removed python2 support mention from README (kivy#2162)
* Adding more assets (kivy#2132)
* Get --add-source working for dirs in Gradle builds (kivy#2156)
* Fixes python build with macOS venv (kivy#2159)

.github/ISSUE_TEMPLATE.md

@@ -1,23 +1,24 @@
 <!--
-The issue tracker is a tool to address bugs.
-Please use the Disord community or Stack Overflow for support questions,
+The issue tracker is a tool to address bugs NOT a support platform.
+Please use the Discord community or Stack Overflow for support questions,
 more information at https://github.com/kivy/python-for-android#support
-
-Before opening a new issue, make sure you do the following:
-    * check that your issue isn't already filed: https://github.com/kivy/python-for-android/issues
-    * prepare a short, runnable example that reproduces the issue
-    * make sure to have `log_level = 2` in your `buildozer.spec`
-    * reproduce the problem with the latest development version (`p4a.branch = master`)
-    * double-check that the issue is indeed a bug and not a support request
-    * please use backticks to format code or logs
 -->
 
+### Checklist
+
+- [ ] the issue is indeed a bug and not a support request
+- [ ] issue doesn't already exist: https://github.com/kivy/python-for-android/issues
+- [ ] I have a short, runnable example that reproduces the issue
+- [ ] I reproduced the problem with the latest development version (`p4a.branch = develop`)
+- [ ] I used the grave accent (aka backticks) to format code or logs when appropriated
+
 ### Versions
 
-* Python:
-* OS:
-* Kivy:
-* Cython:
+- Python:
+- OS:
+- Kivy:
+- Cython:
+- OpenJDK:
 
 ### Description
 
@@ -28,6 +29,7 @@ Before opening a new issue, make sure you do the following:
 Command:
 ```sh
 // REPLACE ME: buildozer command ran? e.g. buildozer android debug
+// Keep the triple grave accent (aka backquote/backtick) to have the code formatted
 ```
 
 Spec file:
@@ -38,5 +40,6 @@ Spec file:
 ### Logs
 
 ```
-// REPLACE ME: Paste the build ouput containing the error
+// REPLACE ME: Paste the build output containing the error
+// Keep the triple grave accent (a.k.a. backquote/backtick) to have the code formatted
 ```

Dockerfile

@@ -15,17 +15,19 @@
 #     Use 'docker run' without '--rm' flag for keeping the container and use
 #     'docker commit <container hash> <new image>' to extend the original image
 
-FROM ubuntu:18.04
+FROM ubuntu:20.04
 
 # configure locale
-RUN apt update -qq > /dev/null && apt install -qq --yes --no-install-recommends \
+RUN apt -y update -qq > /dev/null \
+    && DEBIAN_FRONTEND=noninteractive apt install -qq --yes --no-install-recommends \
     locales && \
     locale-gen en_US.UTF-8
 ENV LANG="en_US.UTF-8" \
     LANGUAGE="en_US.UTF-8" \
     LC_ALL="en_US.UTF-8"
 
-RUN apt -y update -qq > /dev/null && apt -y install -qq --no-install-recommends \
+RUN apt -y update -qq > /dev/null \
+    && DEBIAN_FRONTEND=noninteractive apt install -qq --yes --no-install-recommends \
 	ca-certificates \
     curl \
     && apt -y autoremove \
@@ -49,7 +51,7 @@ ENV WORK_DIR="${HOME_DIR}/app" \
 # install system dependencies
 RUN dpkg --add-architecture i386 \
     && ${RETRY} apt -y update -qq > /dev/null \
-    && ${RETRY} apt -y install -qq --no-install-recommends \
+    && ${RETRY} DEBIAN_FRONTEND=noninteractive apt install -qq --yes --no-install-recommends \
     autoconf \
     automake \
     autopoint \
@@ -64,8 +66,6 @@ RUN dpkg --add-architecture i386 \
     libidn11:i386 \
     libltdl-dev \
     libncurses5:i386 \
-    libpangox-1.0-0:i386 \
-    libpangoxft-1.0-0:i386 \
     libssl-dev \
     libstdc++6:i386 \
     libtool \

Makefile

@@ -20,7 +20,7 @@ all: virtualenv
 
 $(VIRTUAL_ENV):
 	python3 -m venv $(VIRTUAL_ENV)
-	$(PIP) install Cython==0.28.6
+	$(PIP) install Cython==0.29.19
 	$(PIP) install -e .
 
 virtualenv: $(VIRTUAL_ENV)
@@ -38,7 +38,7 @@ testapps-with-numpy/%: virtualenv
 	$(eval $@_APP_ARCH := $(shell basename $*))
 	. $(ACTIVATE) && cd testapps/on_device_unit_tests/ && \
     python setup.py apk --sdk-dir $(ANDROID_SDK_HOME) --ndk-dir $(ANDROID_NDK_HOME) \
-    --requirements libffi,sdl2,pyjnius,kivy,python3,openssl,requests,sqlite3,setuptools,numpy \
+    --requirements libffi,sdl2,pyjnius,kivy,python3,openssl,requests,urllib3,chardet,idna,sqlite3,setuptools,numpy \
     --arch=$($@_APP_ARCH)
 
 testapps/%: virtualenv

README.md

@@ -13,7 +13,6 @@ dependencies you want, and bundle it in an APK along with your own code.
 
 Features include:
 
--  Support for building with both Python 2 and Python 3.
 -  Different app backends including Kivy, PySDL2, and a WebView with
    Python webserver.
 -  Automatic support for most pure Python modules, and built in support

ci/makefiles/osx.mk

@@ -11,7 +11,7 @@ install_java:
 	/usr/libexec/java_home -V
 
 upgrade_cython:
-	pip3 install --upgrade Cython==0.28.6
+	pip3 install --upgrade Cython==0.29.19
 
 install_android_ndk_sdk:
 	mkdir -p $(ANDROID_HOME)

doc/source/contribute.rst

@@ -55,8 +55,8 @@ At this point, wait for reviewer approval and conclude any discussion that arise
 - Merge the release branch to the ``master`` branch.
 - Also merge the release branch to the ``develop`` branch.
 - Tag the release commit in ``master``, with tag ``vYYYY.MM.DD``. Include a short summary of the changes.
-- Create the release distributions: ``python3 setup.py sdist bdist_wheel``
-- Upload the release to pypi: ``python3 -m twine upload``.
+- Release distributions and PyPI upload should be `handled by the CI
+  <https://github.com/kivy/python-for-android/blob/v2020.04.29/.travis.yml#L60-L70>`_.
 - Add to the Github release page (see e.g. `this example <https://github.com/kivy/python-for-android/releases/tag/v2019.06.06>`__):
   - The python-for-android README summary
   - A short list of major changes in this release, if any
@@ -67,15 +67,18 @@ At this point, wait for reviewer approval and conclude any discussion that arise
 
 Release checklist
 ~~~~~~~~~~~~~~~~~
-
-- [ ] Check that the [build is passing](https://travis-ci.org/kivy/python-for-android)
-- [ ] Run the tests locally via `tox`: this performs some long-running tests that are skipped on Travis.
-- [ ] Build and run the [on_device_unit_tests](https://github.com/kivy/python-for-android/tree/master/testapps/on_device_unit_tests) app using buildozer. Check that they all pass.
-- [ ] Build and run the following [testapps](https://github.com/kivy/python-for-android/tree/master/testapps) for arch `armeabi-v7a` and `arm64-v8a`:
-  - `python3 setup_testapp_python3_sqlite_openssl.py apk`
-    - [ ] `armeabi-v7a`
-    - [ ] `arm64-v8a`
-- [ ] Check that the version number is correct
+::
+
+    - [ ] Check that the builds are passing
+      - [ ] [Travis build](https://travis-ci.org/kivy/python-for-android)
+      - [ ] [GitHub Action](https://github.com/kivy/python-for-android/actions)
+    - [ ] Run the tests locally via `tox`: this performs some long-running tests that are skipped on Travis.
+    - [ ] Build and run the [on_device_unit_tests](https://github.com/kivy/python-for-android/tree/master/testapps/on_device_unit_tests) app using buildozer. Check that they all pass.
+    - [ ] Build (or download from github actions) and run the following [testapps](https://github.com/kivy/python-for-android/tree/master/testapps/on_device_unit_tests) for arch `armeabi-v7a` and `arm64-v8a`:
+      - [ ] on_device_unit_tests
+        - [ ] `armeabi-v7a` (`cd testapps/on_device_unit_tests && PYTHONPATH=.:../../ python3 setup.py apk  --ndk-dir=<your-ndk-dir> --sdk-dir=<your-sdk-dir> --arch=armeabi-v7a --debug`)
+        - [ ] `arm64-v8a` (`cd testapps/on_device_unit_tests && PYTHONPATH=.:../../ python3 setup.py apk  --ndk-dir=<your-ndk-dir> --sdk-dir=<your-sdk-dir> --arch=arm64-v8a --debug`)
+    - [ ] Check that the version number is correct
 
 
 

doc/source/troubleshooting.rst

@@ -202,3 +202,20 @@ ModuleNotFoundError: No module named '_ctypes'
 You do not have the libffi headers available to python-for-android, so you need to install them. On Ubuntu and derivatives these come from the `libffi-dev` package.
 
 After installing the headers, clean the build (`p4a clean builds`, or with buildozer delete the `.buildozer` directory within your app directory) and run python-for-android again.
+
+SSLError("Can't connect to HTTPS URL because the SSL module is not available.")
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Your `hostpython3` was compiled without SSL support. You need to install the SSL development files before rebuilding the `hostpython3` recipe.
+Remember to always clean the build before rebuilding (`p4a clean builds`, or with buildozer `buildozer android clean`).
+
+On Ubuntu and derivatives::
+
+    apt install libssl-dev
+    p4a clean builds # or with: buildozer `buildozer android clean
+
+On macOS::
+
+    brew install openssl
+    sudo ln -sfn /usr/local/opt/openssl /usr/local/ssl
+    p4a clean builds # or with: buildozer `buildozer android clean

pythonforandroid/__init__.py

@@ -1 +1 @@
-__version__ = '2020.03.30'
+__version__ = '2020.06.02'

pythonforandroid/archs.py

@@ -145,9 +145,9 @@ def get_env(self, with_flags_in_cc=True):
             + " ".join(
                 [
                     "-L'"
-                    + l.replace("'", "'\"'\"'")
+                    + link_path.replace("'", "'\"'\"'")
                     + "'"  # no shlex.quote in py2
-                    for l in self.extra_global_link_paths
+                    for link_path in self.extra_global_link_paths
                 ]
             )
             + ' ' + ' '.join(self.common_ldflags).format(

pythonforandroid/bootstrap.py

@@ -132,9 +132,6 @@ def get_build_dir(self):
     def get_dist_dir(self, name):
         return join(self.ctx.dist_dir, name)
 
-    def get_common_dir(self):
-        return os.path.abspath(join(self.bootstrap_dir, "..", 'common'))
-
     @property
     def name(self):
         modname = self.__class__.__module__
@@ -152,6 +149,23 @@ def get_bootstrap_dirs(self):
         ]
         return bootstrap_dirs
 
+    def _copy_in_final_files(self):
+        if self.name == "sdl2":
+            # Get the paths for copying SDL2's java source code:
+            sdl2_recipe = Recipe.get_recipe("sdl2", self.ctx)
+            sdl2_build_dir = sdl2_recipe.get_jni_dir()
+            src_dir = join(sdl2_build_dir, "SDL", "android-project",
+                           "app", "src", "main", "java",
+                           "org", "libsdl", "app")
+            target_dir = join(self.dist_dir, 'src', 'main', 'java', 'org',
+                              'libsdl', 'app')
+
+            # Do actual copying:
+            info('Copying in SDL2 .java files from: ' + str(src_dir))
+            if not os.path.exists(target_dir):
+                os.makedirs(target_dir)
+            copy_files(src_dir, target_dir, override=True)
+
     def prepare_build_dir(self):
         """Ensure that a build dir exists for the recipe. This same single
         dir will be used for building all different archs."""
@@ -168,7 +182,12 @@ def prepare_build_dir(self):
     def prepare_dist_dir(self):
         ensure_dir(self.dist_dir)
 
-    def run_distribute(self):
+    def assemble_distribution(self):
+        ''' Copies all the files into the distribution (this function is
+            overridden by the specific bootstrap classes to do this)
+            and add in the distribution info.
+        '''
+        self._copy_in_final_files()
         self.distribution.save_info(self.dist_dir)
 
     @classmethod

pythonforandroid/bootstraps/common/build/build.py

@@ -530,24 +530,28 @@ def make_package(args):
         for patch_name in os.listdir(join('src', 'patches')):
             patch_path = join('src', 'patches', patch_name)
             print("Applying patch: " + str(patch_path))
+
+            # -N: insist this is FORWARD patch, don't reverse apply
+            # -p1: strip first path component
+            # -t: batch mode, don't ask questions
+            patch_command = ["patch", "-N", "-p1", "-t", "-i", patch_path]
+
             try:
-                subprocess.check_output([
-                    # -N: insist this is FORWARd patch, don't reverse apply
-                    # -p1: strip first path component
-                    # -t: batch mode, don't ask questions
-                    "patch", "-N", "-p1", "-t", "-i", patch_path
-                ])
+                # Use a dry run to establish whether the patch is already applied.
+                # If we don't check this, the patch may be partially applied (which is bad!)
+                subprocess.check_output(patch_command + ["--dry-run"])
             except subprocess.CalledProcessError as e:
                 if e.returncode == 1:
-                    # Return code 1 means it didn't apply, this will
-                    # usually mean it is already applied.
-                    print("Warning: failed to apply patch (" +
-                          "exit code 1), " +
-                          "assuming it is already applied: " +
-                          str(patch_path)
-                         )
+                    # Return code 1 means not all hunks could be applied, this usually
+                    # means the patch is already applied.
+                    print("Warning: failed to apply patch (exit code 1), "
+                          "assuming it is already applied: ",
+                          str(patch_path))
                 else:
                     raise e
+            else:
+                # The dry run worked, so do the real thing
+                subprocess.check_output(patch_command)
 
 
 def parse_args_and_make_package(args=None):
@@ -676,6 +680,9 @@ def parse_args_and_make_package(args=None):
                     const='release', default='debug',
                     help='Build your app as a non-debug release build. '
                          '(Disables gdb debugging among other things)')
+    ap.add_argument('--with-debug-symbols', dest='with_debug_symbols',
+                    action='store_const', const=True, default=False,
+                    help='Will keep debug symbols from `.so` files.')
     ap.add_argument('--add-jar', dest='add_jar', action='append',
                     help=('Add a Java .jar to the libs, so you can access its '
                           'classes with pyjnius. You can specify this '

pythonforandroid/bootstraps/common/build/gradle/wrapper/gradle-wrapper.properties

@@ -3,4 +3,4 @@ distributionBase=GRADLE_USER_HOME
 distributionPath=wrapper/dists
 zipStoreBase=GRADLE_USER_HOME
 zipStorePath=wrapper/dists
-distributionUrl=https\://services.gradle.org/distributions/gradle-4.4-all.zip
+distributionUrl=https\://services.gradle.org/distributions/gradle-6.4.1-all.zip