Does swap chain image count include images required by the driver to implement a given present mode?

[Overv]

Let's say I create a swap chain with presentMode = VK_PRESENT_MODE_MAILBOX_KHR and minImageCount = 3. Which of the following two behaviors will I get?

1. I can acquire 3 images with vkAcquireNextImageKHR simultaneously and the driver has a 2 additional images internally to implement the queue and presentation engine.
2. I can acquire only 1 image at a time because the driver uses the other 2 to implement the queue and presentation engine.

With my NVIDIA driver I appear to get 2 simultaneously acquirable images in this case, which is yet another scenario inbetween these. Does the spec guarantee certain behavior in this regard or is this an implementation detail and should I assume the worst case (scenario 2)?

[krOoze]

I was answering this somewhere already... hmm can't find it.

Neither. You have to let driver keep VkSurfaceCapabilitiesKHR::minImageCount - 1. Therefore if as you say your swapchain has 3 images and VkSurfaceCapabilitiesKHR::minImageCount is 2 then you are allowed to acquire 2 images. You are allowed to acquire more if you do not cause a deadlock.

To be precise if VkSurfaceCapabilitiesKHR::minImageCount == 2 and swapchain has 2 images:

vkAcquireNextImage(UINT32_MAX);
vkAcquireNextImage(UINT32_MAX);
// ^ invalid; 2 swapchain images, 2 minImages -> 
// only allowed to acquire one image with endless wait. Undefined behavior (VU violation)

but

vkAcquireNextImage(UINT32_MAX);
vkAcquireNextImage(1 s); // OK, but may fail

and

vkAcquireNextImage(UINT32_MAX);
vkQueuePresent();
vkAcquireNextImage(UINT32_MAX);
// ^ OK, `vkQueuePresent`ed image always counts as released (irrespective of the semaphore wait)

PS:

Let's say I create a swap chain with presentMode = VK_PRESENT_MODE_MAILBOX_KHR and minImageCount = 3.

Technically the count vkGetSwapchainImagesKHR returns counts. vkCreateSwapchain can create more.

PS2:
This applies to all present modes really.

PS3:
Whether vkAcquireNextImage gives you the image immediatelly OR lets you wait for it is a different matter.

[Overv]

Thank you for clarifying, but this seems to be inconsistent with the comments on the VK_KHR_swapchain extension. They state that minImageCount + 1 is required whereas your answer implies that only minImageCount is required (assuming my application never holds more than 1 image).

[krOoze]

@Overv Yes, that somewhat relates to PS3 above. Guaranteeing non-blocking behavior, and guaranteeng something is valid in the first place are two different things.

[Overv]

@krOoze Ah, alright, thanks.

[krOoze]

minImageCount + 1 makes sense. minImageCount means the PE can only give you one image. Assuming that image was previously vkQueuePresented, that means it may still be in the process of being copied into the real compositor buffer. Ergo acquire may theoretically block (though copies can be fast). If there are no worries over RAM then why not have minImageCount + 1.

Then again drivers may report overinflated VkSurfaceCapabilitiesKHR::minImageCount so the above may mean wasted memory. Indeed some platforms allow you to get all the swapchain images without problems, so they should report VkSurfaceCapabilitiesKHR::minImageCount == 1 and not more.
PS: though Vulkan driver may be innocent here and may not know itself. IIRC e.g. X may work any way and does not tell either whether it needs some images forever OR if all are acquirable.

[ianelliottus]

I haven't read this conversation carefully, but want to make sure there are no outstanding questions ...

One thing I'll point out from the spec (re: VkSwapchainCreateInfoKHR::minImageCount) is:

minImageCount is the minimum number of presentable images that the application needs. The implementation will either create the swapchain with at least that many images, or it will fail to create the swapchain.

You might get more images than minImageCount. Therefore, you need to base your code on what's returned by vkGetSwapchainImagesKHR (I mention this because of seeing code that didn't do that:-).

I'll also state what I think you both know: the implementation may create different numbers of images depending on the present mode. For example, on Android, if you ask for 2 images AND MAILBOX, you'll get 3 images. This is as suggested:

• 1 to be displayed to the user
• 1 to be queued
• 1 to be acquired by the application, that can be rendered to

A final point is that, for the sake of portability and longevity, you shouldn't assume a compositor (or lack thereof), nor a particular set of semantics. I've seen some implementations change their behavior overtime, or in certain cases (e.g. if they notice your window is the full size of the screen, they may use HW instead of a compositor).

Hope this is helpful. I'll try to look to see if there are follow-up questions.

[marco1475]

@ianelliottus Can the number of swapchain images returned by vkGetSwapchainImagesKHR vary between devices even if the settings are the same?

For example, on Android, if you ask for 2 images AND MAILBOX, you'll get 3 images.

On Android with:

• VK_PRESENT_MODE_MAILBOX_KHR,
• VkSurfaceCapabilitiesKHR::minImageCount = 2, and
• VkSurfaceCapabilitiesKHR::maxImageCount = 3

I'm seeing vkGetSwapchainImagesKHR return 3 as I would expect on a Mali-based device, but return 4 on NVIDIA's Shield (which has a Tegra).

[ianelliottus]

Sorry for not seeing this sooner. Yes, I believe you can see different numbers returned. The Android implementation is supposed to be common, but I believe we changed things at one of the recent dessert releases. I also know that OEMs (e.g. NVIDIA for the Shield) can change things, or may be running a newer/older version of Android than another OEM's device.

[danielkeller]

@ianelliottus The spec for vkAcquireNextImageKHR says that

An image will eventually be acquired if the number of images that the application has currently acquired (but not yet presented) is less than or equal to the difference between the number of images in swapchain and the value of VkSurfaceCapabilitiesKHR::minImageCount.

Does this mean in that case, when minImageCount = 2 and the present mode is mailbox, I can actually acquire both images that aren't being displayed?

[krOoze]

In other words, you generally can acquire swapchainImageCount - minImageCount + 1 number of images. If the swapchain has 3+ images, then yes, if it has 2 then no.

[plasmacel]

In other words, you generally can acquire swapchainImageCount - minImageCount + 1 number of images. If the swapchain has 3+ images, then yes, if it has 2 then no.

Just to be clear, by minImageCount I guess you mean VkSurfaceCapabilitiesKHR::minImageCount, not VkSwapchainCreateInfoKHR::minImageCount.

[krOoze]

@plasmacel Yes, VkSurfaceCapabilitiesKHR::minImageCount. VkSwapchainCreateInfoKHR::minImageCount is related to swapchainImageCount above ( swapchainImageCount ≥ VkSwapchainCreateInfoKHR::minImageCount )

[plasmacel]

@krOoze Then, if I'm right, for N frames in flight, to achieve non-blocking acquire behavior you actually need VkSwapchainCreateInfoKHR::minImageCount = K + N - 1 images, where K = max(VkSurfaceCapabilitiesKHR::minImageCount, (VkSwapchainCreateInfoKHR::presentMode == VK_PRESENT_MODE_IMMEDIATE_KHR) ? 2 : 3). This works even if the driver reports VkSurfaceCapabilitiesKHR::minImageCount == 1.

[krOoze]

@plasmacel You are making humane assumptions that are not necessarily contained in the dry math.

Firstly, you could have trillion images, and yet Vulkan does not give explicit non-blocking guarantees.

Secondly, N is a very generic number. Say it is 420. Surely, by frame 420, the frame 1 is long done irrespective of any blocking and stuff. As such it would just be wasteful of memory. I am not entirely convinced N larger than 2 makes sense outside very specific niche usages.

Thirdly I am partly as lost as you. That's why I made #1137. Still kinda feel like it is more of a guesswork rather than something you can choose rationally, timelessly, and multiplatformly.

[plasmacel]

@krOoze I have the feeling that the VK_KHR_swapchain extension is not designed well.

First, VkSurfaceCapabilitiesKHR::minImageCount is somewhat pointless, since it is the minimum among the minimums of all presentation modes, while the actual minimum of a specific presentation mode can and will be higher, still it is hidden by the API - that's why I introduced K into my formula, which is a guess for the actual minimum requirement of the presentation engine.

Also, the definition of VkSwapchainCreateInfoKHR::minImageCount is ambiguous, and as you also mention in #1137, it's not clear whether it includes the count of images the presentation engine reserves for itself or not - if you want to be sure, then there is a potential for wasting memory.

Then, the issues of synchronization primitives during swapchain recreation, specifically the potential memory leak which comes from the fact that it is impossible to tell whether the previously used vkQueuePresentKHR semaphores can be safely destroyed or not.

These bug me, since Vulkan was created to be explicit and to avoid the hints and guesses we had to deal with using OpenGL, still such a fundamental extension is broken by design. I hope these will be addressed by Khronos in the close future.

I do agree that N > 2 is rarely useful and even in those cases 3 should be enough.

Thanks for the linked issue though.

[krOoze]

@plasmacel Sometimes, and sometimes the underlying presentation system is the problem. The Vulkan things are just a wrapper for the OS specific stuff, and the OS specific stuff is often not all that great.

I mean, I have a Vulkan DXGI swapchain impl for fun. All it tells you is that you should have between two to sixteen buffers. If I try to give it 1, what happens? Who knows. Reading the MS docs I have only the vaguest and ambiguous notions how their presentation engine works. I also read bit of X spec and Mesa implementation, and it is as bad, if not worse.

First, VkSurfaceCapabilitiesKHR::minImageCount is somewhat pointless, since it is the minimum among the minimums of all presentation modes, while the actual minimum of a specific presentation mode can and will be higher, still it is hidden by the API

It is not so bad, because you learn the actual minimum soon enough when creating the swapchain. What is proplematic here is drivers "cheating" and not necessarily giving you the bearest brutalest minimum, but they sometimes hand-hold you and give you the optimum anyway (I think particularly for MAILBOX). That goes against explicitness, but obviously vendors try to avoid looking like they are low perf defensively.

Also, the definition of VkSwapchainCreateInfoKHR::minImageCount is ambiguous, and as you also mention in #1137, it's not clear whether it includes the count of images the presentation engine reserves for itself or not

That as I recall is e.g. the property of the underlying X. I think the driver itself does not know. What you do is create Pixbufs which are all for the purposes of transfer between the app and the OS. Whether they are used directly or the OS has its own extra scratchpad is largely unspecified. The only information the X volunteers is whether particular image is idle at a particular time, and that's all.

Then, the issues of synchronization primitives during swapchain recreation, specifically the potential memory leak which comes from the fact that it is impossible to tell whether the previously used vkQueuePresentKHR semaphores can be safely destroyed or not.

Yea, that is annoying. Khronos' organizational structure might be the problem here. Vulkan is a design by stakeholders, as it should be. But as such it lacks the single and only great property a design-by-dictator has: top-down, longterm, wholistic, and esthetical thinking. Unless some specific issue is not picked up by a specific volunteer or is a painful immediate issue, it has tendency to rot until it becomes a bigger problem. Not sure how to bring up this metaissue without sounding more overbearing than I already am. Feel free to give it a try. Vulkan could use more responsiveness to the more wholistic or forgotten "everyone's and no one's problems", and less frankensteinish solutions that come too late.

I do agree that N > 2 is rarely useful and even in those cases 3 should be enough.

Yeah. Architecturally 1 (or something less integery) might even make sense. For something like VR, the input latency is at premium, rather than the raw throughput. Meanwhile 1 is also a degenarate case that could potentially simplify the app architecture a bit.