Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-114099: Additions to standard library to support iOS #117052

Merged
merged 39 commits into from
Mar 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
760751b
Mark a concurrency test as requiring fork.
freakboy3742 Mar 16, 2024
b41f9a4
Lower the marshalling stack depth for iOS.
freakboy3742 Mar 16, 2024
0f3e9bc
Add iOS webbrowser support.
freakboy3742 Mar 19, 2024
1a9d965
Updates to site and sysconfig modules for iOS support.
freakboy3742 Mar 19, 2024
c7fe185
Update platform module to provide ios_ver.
freakboy3742 Mar 19, 2024
732c4da
Add changenote.
freakboy3742 Mar 19, 2024
97a081d
Correct some consistency issues.
freakboy3742 Mar 19, 2024
95c367d
Modify testios Makefile target to output failures once.
freakboy3742 Mar 19, 2024
3d6d875
Avoid calling iOS APIs on tvOS/watchOS.
freakboy3742 Mar 19, 2024
d12cfa0
Add a record for the new stdlib module.
freakboy3742 Mar 20, 2024
95d11fb
Add protection against running platform.ios_ver on macOS/Linux, which…
freakboy3742 Mar 20, 2024
48f4c1a
Improve markup in the description of ios_ver()
freakboy3742 Mar 20, 2024
f584d29
Use a namedtuple for ios_ver().
freakboy3742 Mar 20, 2024
9515f75
Ensure the minimum iOS version is used in sysconfig.get_platform()
freakboy3742 Mar 20, 2024
cf0b5ff
Apply suggestions from code review
freakboy3742 Mar 21, 2024
96aa042
Correct the documentation of ios_ver()
freakboy3742 Mar 22, 2024
24b3662
Clarified some discrepancies in the platform module.
freakboy3742 Mar 22, 2024
a4e09c9
Simplify getpath handling.
freakboy3742 Mar 22, 2024
41a3c1a
Modify webbrowser module to use _ios_support as a helper.
freakboy3742 Mar 22, 2024
84ba760
Simplify sysconfig schemes for iOS.
freakboy3742 Mar 22, 2024
4194849
Apply suggestions from code review
freakboy3742 Mar 24, 2024
9a91933
More documentation tweaks.
freakboy3742 Mar 24, 2024
e6550b7
Add protection for missing ctypes to webbrowser module.
freakboy3742 Mar 24, 2024
8654376
Simplifications and clarifications to sysconfig.
freakboy3742 Mar 25, 2024
7a4dcaf
Merge branch 'main' into ios-platform-changes
freakboy3742 Mar 25, 2024
4451326
Correct the naming of the IPHONEOS_DEPLOYMENT_TARGET variable.
freakboy3742 Mar 25, 2024
4386a7a
Added clarifying comment around IPHONEOS_DEPLOYMENT_TARGET
freakboy3742 Mar 25, 2024
c1a1f0b
Merge branch 'main' into ios-platform-changes
freakboy3742 Mar 27, 2024
61559ac
Removed the hard-coded development team.
freakboy3742 Mar 27, 2024
abc2034
Account for some testing edge cases picked up in review.
freakboy3742 Mar 27, 2024
096078a
Disable --with-ensurepip for iOS.
freakboy3742 Mar 27, 2024
44bbf79
Merge branch 'main' into ios-platform-changes
freakboy3742 Mar 27, 2024
7419002
Correct merge of test_platform.
freakboy3742 Mar 27, 2024
c121d5f
Always display iOS test output.
freakboy3742 Mar 28, 2024
1ac4b26
Make test suite resilient to the absence of ctypes.
freakboy3742 Mar 28, 2024
857a0c3
Make webbrowser skip resilient across platforms.
freakboy3742 Mar 28, 2024
aa65dc2
Account for ABI suffix in header directory
ned-deily Mar 28, 2024
61e51ff
test_refcount_errors requires subprocess
ned-deily Mar 28, 2024
2ee4aba
Merge branch 'main' into ios-platform-changes
ned-deily Mar 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions Doc/library/os.rst
Original file line number Diff line number Diff line change
Expand Up @@ -784,6 +784,11 @@ process and user.
:func:`socket.gethostname` or even
``socket.gethostbyaddr(socket.gethostname())``.

On macOS, iOS and Android, this returns the *kernel* name and version (i.e.,
``'Darwin'`` on macOS and iOS; ``'Linux'`` on Android). :func:`platform.uname()`
can be used to get the user-facing operating system name and version on iOS and
Android.

.. availability:: Unix.

.. versionchanged:: 3.3
Expand Down
24 changes: 23 additions & 1 deletion Doc/library/platform.rst
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,9 @@ Cross Platform
Returns the system/OS name, such as ``'Linux'``, ``'Darwin'``, ``'Java'``,
``'Windows'``. An empty string is returned if the value cannot be determined.

On iOS and Android, this returns the user-facing OS name (i.e, ``'iOS``,
``'iPadOS'`` or ``'Android'``). To obtain the kernel name (``'Darwin'`` or
``'Linux'``), use :func:`os.uname()`.

.. function:: system_alias(system, release, version)

Expand All @@ -161,6 +164,8 @@ Cross Platform
Returns the system's release version, e.g. ``'#3 on degas'``. An empty string is
returned if the value cannot be determined.

On iOS and Android, this is the user-facing OS version. To obtain the
Darwin or Linux kernel version, use :func:`os.uname()`.

.. function:: uname()

Expand Down Expand Up @@ -238,7 +243,6 @@ Windows Platform
macOS Platform
--------------


.. function:: mac_ver(release='', versioninfo=('','',''), machine='')

Get macOS version information and return it as tuple ``(release, versioninfo,
Expand All @@ -248,6 +252,24 @@ macOS Platform
Entries which cannot be determined are set to ``''``. All tuple entries are
strings.

iOS Platform
------------

.. function:: ios_ver(system='', release='', model='', is_simulator=False)

Get iOS version information and return it as a
:func:`~collections.namedtuple` with the following attributes:

* ``system`` is the OS name; either ``'iOS'`` or ``'iPadOS'``.
* ``release`` is the iOS version number as a string (e.g., ``'17.2'``).
* ``model`` is the device model identifier; this will be a string like
``'iPhone13,2'`` for a physical device, or ``'iPhone'`` on a simulator.
* ``is_simulator`` is a boolean describing if the app is running on a
simulator or a physical device.

Entries which cannot be determined are set to the defaults given as
parameters.


Unix Platforms
--------------
Expand Down
17 changes: 16 additions & 1 deletion Doc/library/webbrowser.rst
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,13 @@ allow the remote browser to maintain its own windows on the display. If remote
browsers are not available on Unix, the controlling process will launch a new
browser and wait.

On iOS, the :envvar:`BROWSER` environment variable, as well as any arguments
controlling autoraise, browser preference, and new tab/window creation will be
ignored. Web pages will *always* be opened in the user's preferred browser, in
a new tab, with the browser being brought to the foreground. The use of the
:mod:`webbrowser` module on iOS requires the :mod:`ctypes` module. If
:mod:`ctypes` isn't available, calls to :func:`.open` will fail.

The script :program:`webbrowser` can be used as a command-line interface for the
module. It accepts a URL as the argument. It accepts the following optional
parameters: ``-n`` opens the URL in a new browser window, if possible;
Expand Down Expand Up @@ -147,6 +154,8 @@ for the controller classes, all defined in this module.
+------------------------+-----------------------------------------+-------+
| ``'chromium-browser'`` | ``Chromium('chromium-browser')`` | |
+------------------------+-----------------------------------------+-------+
| ``'iosbrowser'`` | ``IOSBrowser`` | \(4) |
+------------------------+-----------------------------------------+-------+

Notes:

Expand All @@ -161,7 +170,10 @@ Notes:
Only on Windows platforms.

(3)
Only on macOS platform.
Only on macOS.

(4)
Only on iOS.

.. versionadded:: 3.2
A new :class:`!MacOSXOSAScript` class has been added
Expand All @@ -176,6 +188,9 @@ Notes:
Removed browsers include Grail, Mosaic, Netscape, Galeon,
Skipstone, Iceape, and Firefox versions 35 and below.

.. versionchanged:: 3.13
Support for iOS has been added.

Here are some simple examples::

url = 'https://docs.python.org/'
Expand Down
71 changes: 71 additions & 0 deletions Lib/_ios_support.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
import sys
try:
from ctypes import cdll, c_void_p, c_char_p, util
except ImportError:
# ctypes is an optional module. If it's not present, we're limited in what
# we can tell about the system, but we don't want to prevent the module
# from working.
print("ctypes isn't available; iOS system calls will not be available")
objc = None
else:
# ctypes is available. Load the ObjC library, and wrap the objc_getClass,
# sel_registerName methods
lib = util.find_library("objc")
if lib is None:
# Failed to load the objc library
raise RuntimeError("ObjC runtime library couldn't be loaded")

objc = cdll.LoadLibrary(lib)
objc.objc_getClass.restype = c_void_p
objc.objc_getClass.argtypes = [c_char_p]
objc.sel_registerName.restype = c_void_p
objc.sel_registerName.argtypes = [c_char_p]


def get_platform_ios():
# Determine if this is a simulator using the multiarch value
is_simulator = sys.implementation._multiarch.endswith("simulator")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
is_simulator = sys.implementation._multiarch.endswith("simulator")
is_simulator = hasttr(sys.implementation, '_multiarch') and sys.implementation._multiarch.endswith("simulator")

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As above: iOS always has _multiarch, and get_platform_ios() is only invoked by platform.ios_ver(), after being gated to be iOS-specific.


# We can't use ctypes; abort
if not objc:
return None

# Most of the methods return ObjC objects
objc.objc_msgSend.restype = c_void_p
# All the methods used have no arguments.
objc.objc_msgSend.argtypes = [c_void_p, c_void_p]

# Equivalent of:
# device = [UIDevice currentDevice]
UIDevice = objc.objc_getClass(b"UIDevice")
SEL_currentDevice = objc.sel_registerName(b"currentDevice")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

snake_case for this and the other variables here, or is it more important to mirror the objc names?

https://peps.python.org/pep-0008/#function-and-variable-names

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - the intention was to mirror the objc naming, so that it's a little easier to see that the name comes from an "alien space", and might have some additional considerations - a sort of very weak Hungarian notation.

In terms of pure functionality, it doesn't have any impact on the operation of the code. Any variable name will do; this naming convention is entirely as an affordance to future readers. For my money, the "hey, that's weird - they're not camel case" reaction is mildly desirable in this case.

device = objc.objc_msgSend(UIDevice, SEL_currentDevice)

# Equivalent of:
# device_systemVersion = [device systemVersion]
SEL_systemVersion = objc.sel_registerName(b"systemVersion")
device_systemVersion = objc.objc_msgSend(device, SEL_systemVersion)

# Equivalent of:
# device_systemName = [device systemName]
SEL_systemName = objc.sel_registerName(b"systemName")
device_systemName = objc.objc_msgSend(device, SEL_systemName)

# Equivalent of:
# device_model = [device model]
SEL_model = objc.sel_registerName(b"model")
device_model = objc.objc_msgSend(device, SEL_model)

# UTF8String returns a const char*;
SEL_UTF8String = objc.sel_registerName(b"UTF8String")
objc.objc_msgSend.restype = c_char_p

# Equivalent of:
# system = [device_systemName UTF8String]
# release = [device_systemVersion UTF8String]
# model = [device_model UTF8String]
system = objc.objc_msgSend(device_systemName, SEL_UTF8String).decode()
release = objc.objc_msgSend(device_systemVersion, SEL_UTF8String).decode()
model = objc.objc_msgSend(device_model, SEL_UTF8String).decode()

return system, release, model, is_simulator
53 changes: 46 additions & 7 deletions Lib/platform.py
Original file line number Diff line number Diff line change
Expand Up @@ -496,6 +496,30 @@ def mac_ver(release='', versioninfo=('', '', ''), machine=''):
# If that also doesn't work return the default values
return release, versioninfo, machine


# A namedtuple for iOS version information.
IOSVersionInfo = collections.namedtuple(
"IOSVersionInfo",
["system", "release", "model", "is_simulator"]
)


def ios_ver(system="", release="", model="", is_simulator=False):
"""Get iOS version information, and return it as a namedtuple:
(system, release, model, is_simulator).

If values can't be determined, they are set to values provided as
parameters.
"""
if sys.platform == "ios":
import _ios_support
result = _ios_support.get_platform_ios()
if result is not None:
return IOSVersionInfo(*result)

return IOSVersionInfo(system, release, model, is_simulator)


def _java_getprop(name, default):
"""This private helper is deprecated in 3.13 and will be removed in 3.15"""
from java.lang import System
Expand Down Expand Up @@ -654,7 +678,7 @@ def _platform(*args):
if cleaned == platform:
break
platform = cleaned
while platform[-1] == '-':
while platform and platform[-1] == '-':
platform = platform[:-1]

return platform
Expand Down Expand Up @@ -695,7 +719,7 @@ def _syscmd_file(target, default=''):
default in case the command should fail.

"""
if sys.platform in ('dos', 'win32', 'win16'):
if sys.platform in {'dos', 'win32', 'win16', 'ios', 'tvos', 'watchos'}:
# XXX Others too ?
return default

Expand Down Expand Up @@ -859,6 +883,14 @@ def get_OpenVMS():
csid, cpu_number = vms_lib.getsyi('SYI$_CPU', 0)
return 'Alpha' if cpu_number >= 128 else 'VAX'

# On the iOS simulator, os.uname returns the architecture as uname.machine.
# On device it returns the model name for some reason; but there's only one
# CPU architecture for iOS devices, so we know the right answer.
def get_ios():
if sys.implementation._multiarch.endswith("simulator"):
return os.uname().machine
return 'arm64'
Comment on lines +890 to +892
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if sys.implementation._multiarch.endswith("simulator"):
return os.uname().machine
return 'arm64'
if hasattr(sys.implementation, '_multiarch') and sys.implementation._multiarch.endswith("simulator"):
return os.uname().machine
return 'arm64'

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because of the way the build works for iOS, I think we can assume that _multiarch is always present.

Copy link
Contributor Author

@freakboy3742 freakboy3742 Mar 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirming: iOS builds will always have _multiarch set (because of this section of configure.ac); and this block of logic will only be invoked on iOS as it's part of a get_ios() method.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is true for CPython, but it may change for different implementations that keep our stdlib. It's not a big deal, but generally I avoid adding a hard dependency on implementation details. This is certainly not a blocker, just a possible improvement.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoiding hard dependencies on implementation details definitely makes sense; However, I'm not sure if using a fallback is entirely safe. Without any additional changes, using a fallback value for _multiarch will result in:

  • All iOS code identifying as ARM64 on a physical device, regardless whether it is or not
  • Attempting to load binary modules with the tag ios-12.0-, regardless of architecture or simulator status

The second point would at least complicate, and possibly prevent any binary module from loading; the first would lead to misleading or incorrect behavior.

On that basis, I'd argue it would be safer to fail early, providing a clear indicator that something isn't configured correctly (or, at least, not configured the way that the CPython implementation expects). However, it's also a relatively straightforward change, so I'm happy to be overruled on this.


def from_subprocess():
"""
Fall back to `uname -p`
Expand Down Expand Up @@ -1018,6 +1050,10 @@ def uname():
system = 'Android'
release = android_ver().release

# Normalize responses on iOS
if sys.platform == 'ios':
system, release, _, _ = ios_ver()

vals = system, node, release, version, machine
# Replace 'unknown' values with the more portable ''
_uname_cache = uname_result(*map(_unknown_as_blank, vals))
Expand Down Expand Up @@ -1297,11 +1333,14 @@ def platform(aliased=False, terse=False):
system, release, version = system_alias(system, release, version)

if system == 'Darwin':
# macOS (darwin kernel)
macos_release = mac_ver()[0]
if macos_release:
system = 'macOS'
release = macos_release
# macOS and iOS both report as a "Darwin" kernel
if sys.platform == "ios":
system, release, _, _ = ios_ver()
else:
macos_release = mac_ver()[0]
if macos_release:
system = 'macOS'
release = macos_release

if system == 'Windows':
# MS platforms
Expand Down
4 changes: 2 additions & 2 deletions Lib/site.py
Original file line number Diff line number Diff line change
Expand Up @@ -280,8 +280,8 @@ def _getuserbase():
if env_base:
return env_base

# Emscripten, VxWorks, and WASI have no home directories
if sys.platform in {"emscripten", "vxworks", "wasi"}:
# Emscripten, iOS, tvOS, VxWorks, WASI, and watchOS have no home directories
if sys.platform in {"emscripten", "ios", "tvos", "vxworks", "wasi", "watchos"}:
return None

def joinuser(*args):
Expand Down
20 changes: 14 additions & 6 deletions Lib/sysconfig/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@

# Keys for get_config_var() that are never converted to Python integers.
_ALWAYS_STR = {
'IPHONEOS_DEPLOYMENT_TARGET',
'MACOSX_DEPLOYMENT_TARGET',
}

Expand Down Expand Up @@ -57,6 +58,7 @@
'scripts': '{base}/Scripts',
'data': '{base}',
},

# Downstream distributors can overwrite the default install scheme.
# This is done to support downstream modifications where distributors change
# the installation layout (eg. different site-packages directory).
Expand Down Expand Up @@ -114,8 +116,8 @@ def _getuserbase():
if env_base:
return env_base

# Emscripten, VxWorks, and WASI have no home directories
if sys.platform in {"emscripten", "vxworks", "wasi"}:
# Emscripten, iOS, tvOS, VxWorks, WASI, and watchOS have no home directories
if sys.platform in {"emscripten", "ios", "tvos", "vxworks", "wasi", "watchos"}:
return None

def joinuser(*args):
Expand Down Expand Up @@ -290,6 +292,7 @@ def _get_preferred_schemes():
'home': 'posix_home',
'user': 'osx_framework_user',
}

return {
'prefix': 'posix_prefix',
'home': 'posix_home',
Expand Down Expand Up @@ -623,10 +626,15 @@ def get_platform():
if m:
release = m.group()
elif osname[:6] == "darwin":
import _osx_support
osname, release, machine = _osx_support.get_platform_osx(
get_config_vars(),
osname, release, machine)
if sys.platform == "ios":
release = get_config_vars().get("IPHONEOS_DEPLOYMENT_TARGET", "12.0")
osname = sys.platform
machine = sys.implementation._multiarch
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
machine = sys.implementation._multiarch
machine = getattr(sys.implementation, '_multiarch', '')

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again; this is iOS specific code.

else:
import _osx_support
osname, release, machine = _osx_support.get_platform_osx(
get_config_vars(),
osname, release, machine)

return f"{osname}-{release}-{machine}"

Expand Down
1 change: 1 addition & 0 deletions Lib/test/pythoninfo.py
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,7 @@ def format_groups(groups):
"HOMEDRIVE",
"HOMEPATH",
"IDLESTARTUP",
"IPHONEOS_DEPLOYMENT_TARGET",
"LANG",
"LDFLAGS",
"LDSHARED",
Expand Down
1 change: 1 addition & 0 deletions Lib/test/test_concurrent_futures/test_thread_pool.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ def test_idle_thread_reuse(self):
self.assertEqual(len(executor._threads), 1)
executor.shutdown(wait=True)

@support.requires_fork()
@unittest.skipUnless(hasattr(os, 'register_at_fork'), 'need os.register_at_fork')
@support.requires_resource('cpu')
def test_hang_global_shutdown_lock(self):
Expand Down
1 change: 1 addition & 0 deletions Lib/test/test_gc.py
Original file line number Diff line number Diff line change
Expand Up @@ -1223,6 +1223,7 @@ def test_collect_garbage(self):
self.assertEqual(len(gc.garbage), 0)


@requires_subprocess()
@unittest.skipIf(BUILD_WITH_NDEBUG,
'built with -NDEBUG')
def test_refcount_errors(self):
Expand Down
Loading
Loading