Skip to content

Introduction

Jump to bottom
Chris Birkhold edited this page Aug 30, 2018 · 2 revisions

OpenVRMetal is a collection of utility classes to make it easier to work with OpenVR + Metal on macOS. As of August 2018 Vive/SteamVR is the only officially supported combination of HMD/runtime for VR on macOS. Therefore it is assumed in the following that OpenVR is implemented in terms of SteamVR.

OpenVR + Metal

Apple/Valve announced SteamVR/Vive support for macOS High Sierra at WWDC 2017 and again at WWDC 2018 with improved Metal support for macOS Mojave. Because of those advanced features, the platform specific Metal advantages over OpenGL, as well as Apple retiring OpenGL support without getting behind Vulkan, Metal is the choice for any new VR development for macOS. With that in mind let's take a look at the Metal specific and Metal related parts of the OpenVR API.

Metal Specifics

The Metal specific part of OpenVR is quite small:

  1. IVRCompositor::Submit accepts Texture_t descriptors of type:
    1. TextureType_IOSurface (macOS 11.13)
    2. TextureType_Metal (macOS 10.14+)
  2. IVRSystem::GetOutputDevice returns the Metal device representing the GPU the HMD is connected to.

Though however small the API surfaces may be there is a few details to be aware of. The following is not based on direct knowledge of the internal workings of SteamVR but rather on analysis, observations and experiments.

TextureType_IOSurface

  • The kIOSurfaceIsGlobal property must be set to true/yes on creation of the IOSurface. While kIOSurfaceIsGlobal was deprecated in favor of using IOSurfaceCreateMachPort or IOSurfaceCreateXPCObject, as of August 2018 SteamVR still uses good old shared memory to submit IOSurface backed textures to the compositor so this property must be set for the compositor process to be able to access the surface.
  • The same two IOSurface backed Metal texture (one for each eye) must be used for all frames submitted to the compositor. If this is not done SteamVR appears to ignore all but the first submitted texture or present them at the wrong times. For instance submitting a triple-buffered texture appears to result in only index zero being displayed across three frames (effectively cutting the frame rate down to a third). Even though the same textures will be used every frame IVRCompositor::Submit must still be called once per frame for each eye. Of course if either distortion or color space conversion is left to the SteamVR runtime then the submitted textures could not be used for direct-to-display read-out anyways.
  • The two supported IOSurface pixel formats appear to be 'RGBA' (know as k32RGBAPixelFormat, kIO32RGBAPixelFormat or kCVPixelFormatType_32RGBA) and 'BGRA' (k32BGRAPixelFormat, kIO32BGRAPixelFormat, kCVPixelFormatType_32BGRA, ...). These map to MTLPixelFormatRGBA8Unorm, MTLPixelFormatRGBA8Unorm_sRGB, MTLPixelFormatRGBA8Uint and MTLPixelFormatBGRA8Unorm, MTLPixelFormatBGRA8Unorm_sRGB respectively.

TextureType_Metal

  • As of August 2018 (SteamVR mainline, macOS 10.14 Beta) support for this feature is incomplete.
  • Textures must be created using MTLDevice::newSharedTextureWithDescriptor. The Metal framework creates an IOSurface internally but does not set the public iosurface property as would be the case for a Metal texture created directly from an IOSurface using MTLDevice::newTextureWithDescriptor. For the curious the private iosurface property can still be read for analyzing the properties of the backing IOSurface.
  • Supports MTLTextureType2DArray.

Metal Related

There are a few generic OpenVR types that need to be converted for use with the Metal framework:

  1. Math Types (vectors, matrices, ...)
  2. Hidden Area Meshes
  3. Render Models
Clone this wiki locally