Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable application volume and mute features when sound split is off or when wasapi is off #16404

Closed
wants to merge 2 commits into from

Conversation

mltony
Copy link
Contributor

@mltony mltony commented Apr 16, 2024

Link to issue number:

Closes #16402
Closes #16409

Summary of the issue:

Mute applications command works incorrectly when sound split is off. or when wasapi is off.

Description of user facing changes

Now muting other apps or adjusting other apps volume only works when sound split is not set to off.
Also speaking error message when wasapi is off.

Description of development approach

Since #16287 I had to introduce explicit sound split off mode, which is not supposed to touch volumes of other applications through wasapi. Since adjusting and muting volume of other apps done through the same logic, disabling it for sound split off mode.

Testing strategy:

Tested the following conditions:

  • Muting and adjusting volumes of other apps doesn't work when sound split is off.
  • muting is temporary when sound split is enabled. That is after restart all apps are unmuted.
  • Adjusting volumes of other apps is permanent across restarts - as expected.

Known issues with pull request:

N/A

Code Review Checklist:

  • Documentation:
    • Change log entry
    • User Documentation
    • Developer / Technical Documentation
    • Context sensitive help for GUI changes
  • Testing:
    • Unit tests
    • System (end to end) tests
    • Manual testing
  • UX of all users considered:
    • Speech
    • Braille
    • Low Vision
    • Different web browsers
    • Localization in other languages / culture than English
  • API is compatible with existing add-ons.
  • Security precautions taken.

@mltony mltony marked this pull request as ready for review April 16, 2024 18:35
@mltony mltony requested a review from a team as a code owner April 16, 2024 18:35
@mltony mltony requested a review from gerald-hartig April 16, 2024 18:35
@AppVeyorBot
Copy link

See test results for failed build of commit 5f97586b17

@CyrilleB79
Copy link
Collaborator

@mltony, this PR totally defeats the use of the mute other apps feature, at least in the majority of its use case.

The users will use NVDA with sound split disabled most of the time. But during this time, they may still want to mute other apps quickly with a handy shortcut.

I understand that there is a correlation in your implementation between sound split and other apps muting or volume changing. But as already written elsewhere, I really think that you should de-correlate these two features. And if possible link other apps volume setting or muting to Windows Volume Mixer instead, while keeping sound split internal to NVDA.

I guess that we should really re-clarify why the mute other apps feature was requested, but probably not in this PR...

@mltony
Copy link
Contributor Author

mltony commented Apr 16, 2024

Decorellating is hard. Since sound split and apps volume use the same API and if we separate them in NVDA codebase, we'll have to deal with all the pain from their inevitable interference.
And there's been a strong push by @LeonarddeR et al to have soundSplit off = don't touch anything via wasapi. I would think that implies not muting other applications.
So it seems to me "you can't both have a cake and eat it" type of situation.
Also in order to mute apps you can switch sound split to NVDA_BOTH_APPS_BOTH - which is in most situations equivalent to sound split off.

@CyrilleB79
Copy link
Collaborator

Decorellating is hard. Since sound split and apps volume use the same API and if we separate them in NVDA codebase, we'll have to deal with all the pain from their inevitable interference.

I think a path to investigate is to see if it's possible to modify apps mute/volume so that it is reflected in Windows Volume Mixer.
But if there is no other way to do it, I really think that dealing with the interference of both features is needed.

And there's been a strong push by @LeonarddeR et al to have soundSplit off = don't touch anything via wasapi. I would think that implies not muting other applications.

Why do you think that the same reason apply for other apps muting. As I describe in #16402, the use case between other apps muting and sound split are not the same IMO.
Muting or modifying volume of other apps should be considered as a way to control Windows Volume Mixer thanks to shortcuts implemented in NVDA. Whereas sound split should not touch Windows Volume Mixer at all and should remain internal to NVDA.

Also in order to mute apps you can switch sound split to NVDA_BOTH_APPS_BOTH - which is in most situations equivalent to sound split off.

Yes, I know, but NVDA_BOTH_APPS_BOTH is not the default setting; and I do not say that it should be.

Another strange thing is that if your other apps are muted and you turn back to sound split off, the other apps are unmuted. That's quite a strange and bad UX.

@mltony
Copy link
Contributor Author

mltony commented Apr 16, 2024

Shall we drop apps mute then? Looks like you're not happy with this implementation and originally I only intended to add adjusting other apps volume specifically for sound split. I only added apps mute as per your request, but now it seems like it might not be that good of a fit here in the end. Then perhaps you or someone else can implement apps mute later using some other technique. WDYT?

@LeonarddeR
Copy link
Collaborator

Shall we drop apps mute then?

I would at least take it back to the drawing board. Note that #16273 was merged without some comments in #16042 being addressed. In my opinion, there is also a task for NV Access to manage these types of processes and to be very cautious in merging features about which there is doubt among part of the core community. afterwards.

@XLTechie
Copy link
Collaborator

XLTechie commented Apr 17, 2024 via email

@amirsol81
Copy link

@mltony Honestly muting other apps when Sound Split is disabled is what, IMO, most users might want to do, so if muting would be disabled when Sound Split is off, it'll lose most of its usefulness and appeal. As such, I think if we are supposed to keep muting other apps, it should work regardless of whether or not Sound Split is enabled. If these can't be separated or if issues like #16402 can't be fixed easily, I suggest removing the mute feature and retaining the volume adjustment.

@amirsol81
Copy link

@XLTechie Oddly enough, I think volume adjustment makes more sense compared with muting, but I understand that this is absolutely personal. Ideally both should be retained if issues can be resolved - I mean stuff like restoring the mute state upon exiting, or restarting, NVDA. But if one is to be retained, I think it should be app volume adjustment.

@CyrilleB79
Copy link
Collaborator

IMO the discussion here is not should we keep mute or volume? Both feature are quite similar and the issues found with one of them will probably be found with the other one.

I guess that most of us also agree or at least accept that these features be integrated in core since they answer to the need of many people.

Though, there is an on-going discussion in #16402 about reverting the whole feature. And it should be clarified before going further with this PR.

@CyrilleB79
Copy link
Collaborator

@mltony for another technique to control per-app volume modification or muting, you can have a look at NVDA Sound Manager add-on from @yplassiard. With this add-on, if you modify the volume of other apps or mute them, it is reflected in Windows Volume Manager.
What do you think?

@mltony
Copy link
Contributor Author

mltony commented Apr 17, 2024

@CyrilleB79, I checked source code of Sound Manager, they just use simple audio volume. I think you're right in the sense that this might allow us to decouple apps volume adjusting from sound splitting. I would need to make a prototype to confirm that ISimpleAudioVolume and IChannelAudioVolume interfaces don't interfere with each other in some weird unexpected way.
There is going to be new interference with Sound Manager add-on though. But I can make it so that when volume=100% and mute=False, the feature will be effectivly off and so in this mode Sound Manager or any other addons that use ISimpleAudioVolume should work without problems.

@Adriani90
Copy link
Collaborator

Adriani90 commented Apr 17, 2024

It was clear from the beginning of the feature design that the use case is in situations where sound split is used and still the other apps are too loud. There was never the discussion to control Windows volume mixer via NVDA. So I think there was always actually the understanding that these settings are linked to the sound split which is restored after NVDA exit.
I don't agree with controlling the volume mixer via NVDA.
Use case reported by @CyrilleB79:

I use NVDA and have set my other apps volume quite low (e.g. 5%) to continue having music played in the background. During my tasks, I install an add-on and restart NVDA. As soon as NVDA exits, the music begins shouting loud.

I understand this use case, but this happens with audio ducking set to always as well. The volume is restored. At the other end if you are using a computer together with a person with hearing problems, and this person needs the loudness for its tasks, it is really not a good idea to let any negative traces after using NVDA.
I suggest @mltony continues with his solution and I propose to restore the volume of apps as well after NVDA exits.
I think NVDA could remember the volume of apps as it remembers the sound split setting when starting NVDA, but the muted / unmuted state should still remain temporary.

@Adriani90
Copy link
Collaborator

Why is there the desire to control the volume mixer at all via NVDA when the mixer itself is accessible? The use case for these settings in NVDA are for on demand purpose and are only linked to the NVDA user. I don't see any arguments to link the volume mixer here.

@Adriani90
Copy link
Collaborator

Adriani90 commented Apr 17, 2024

@CyrilleB79 wrote:

The users will use NVDA with sound split disabled most of the time. But during this time, they may still want to mute other apps quickly with a handy shortcut.

I don't think this is true. If users will discover the benefits of using sound split, they will rather use that. And I think even having this feature to mute other apps when sound split is on it will make this feature more attractive anyway.
Users who are not using sound split can still use the audio ducking feature for less volume, or use the accessible mixer for muting the apps.

But yeah, I am not against having that decoupled from sound split if possible as long as it stays a temporary setting until NVDA exits.

@amirsol81
Copy link

@Adriani90 I have discovered the benefits of the Sound Split, but they really don't fit into my daily use case. I guess there will be many users like me. However, I do want to be able to alter the volume or mute other apps via NVDA's easy-to-access key strokes when the Sound Split is disabled. As @CyrilleB79 mentioned, the feature will be utterly defeated if it fails to work with Sound Split disabled.

@Adriani90
Copy link
Collaborator

@amirsol81 as I said, I am not against decoupling as long as it restores after exiting NVDA.
But if this is not technically possible at this stage, this should not block this PR. And still lots of users will benefit from these settings, even though it works only with sound split on.
Making this work with sound split disabled can be handled in a separate PR if it turns out to be technically too complex at this stage.

@XLTechie
Copy link
Collaborator

XLTechie commented Apr 17, 2024

@Adriani90 asked:

Why is there the desire to control the volume mixer at all via NVDA when the mixer itself is accessible?

At least for me, the reason is this. Consider the following circumstance:

  1. NVDA and sighted user share the same computer.
  2. NVDA user mutes other apps while playing a video in Firefox.
  3. NVDA user quits NVDA and leaves.
  4. Sighted user tries to play a video in Firefox, but finds it muted.
  5. Sighted user goes to volume mixer to unmute it, only to find that it is already unmuted in volume mixer. In fact, pressing mute/unmute has no effect.

How does the sighted user get audio back? The same question can be asked for volume changes.

Further, if an inexperienced NVDA user mutes audio/changes volume accidentally, how is a sighted user (or Microsoft accessibility support, etc.) likely to help, with or without NVDA? By going to the system volume mixer, I would think.

So not having these changes synchronized with the standard way that Windows manipulates audio, seems like a degraded UX all around.

As an aside: in my other life, I am an audio technician. When there are problems with audio, you always start solving them by following the signal chain. From input to output. In this case, the input is Firefox playing a video, and the output is the speakers. What's between them, is Windows volume mixer.
Now we suddenly put something else into the signal chain that can do the same things as Windows volume mixer, but in an obscure way that some users of the machine can't even find, and don't know to expect.
So now there are two mixers in the chain, one of which can be invisible, and overrides the other, or makes its settings relative instead of absolute. It seems needlessly complex.
On top of that, since NVDA can make these changes and then be shut down with no obvious way to reverse or even view the changes it made, it makes the process doubly obscure.

@seanbudd seanbudd added this to the 2024.2 milestone Apr 18, 2024
@seanbudd
Copy link
Member

Can you please re-target this PR to beta

@mltony
Copy link
Contributor Author

mltony commented Apr 18, 2024

@seanbudd ,
Sorry for noob question - shall I run something like git merge upstream/beta?
Also it seems like there is no consensus on whether this is the right direction, but it seems to me we should still squeeze this one into beta and then have a discussion on whether decoupling appVolumes and soundSplit is the right direction.

@seanbudd seanbudd changed the base branch from master to beta April 18, 2024 03:53
@seanbudd
Copy link
Member

Generally you would have to do git rebase -i beta. I would encourage reading up on this and testing it out first on a separate branch.

In this case I just needed to retarget via the GitHub interface, as master is now identical to beta, so no need to take further action.

@seanbudd seanbudd added the blocked/needs-product-decision A product decision needs to be made. Decisions about NVDA UX or supported use-cases. label Apr 18, 2024
@Adriani90
Copy link
Collaborator

@XLTechie

How do sighted people get volume back?

NVDA restores it after exit.

@mltony
Not restoring things after NVDA exit will introduce inconsistencies e.g. with audio ducking feature and will break user experience. @seanbudd this PR fixes this, because the feature as is now in alpha is not acceptable. So I would say it is important to remove the blocked label and continue working on this.

re decoupling in my view it is out of scope for this PR and can be handled in a separate discussion.

@LeonarddeR
Copy link
Collaborator

@Adriani90

The feature as is now in alpha is not acceptable. So I would say it is important to remove the blocked label and continue working on this.

If the feature as is is not acceptable and the current pr needs a product decision (i agree with both), then the preferable solution is reverting the feature and providing a new pr that is in line with the product decision to be made.

@Adriani90
Copy link
Collaborator

Adriani90 commented Apr 18, 2024 via email

@cary-rowen
Copy link
Contributor

I also agree that sound split and application volume should be decoupled.

@Adriani90
Copy link
Collaborator

Adriani90 commented Apr 18, 2024 via email

@XLTechie
Copy link
Collaborator

XLTechie commented Apr 18, 2024 via email

@XLTechie
Copy link
Collaborator

@Adriani90 you wrote something puzzling:

Not restoring things after NVDA exit will introduce inconsistencies e.g. with audio ducking feature and will break user experience. @seanbudd this PR fixes this, because the feature as is now in alpha is not acceptable. So I would say it is important to remove the blocked label and continue working on this.

All this PR does is make mute/volume features only work with sound split enabled. It does not claim to do anything about restoring states after NVDA closes. Can you explain why you say it fixes that issue?

Personally, I agree with those who believe that this PR should be closed--if muting and volume changes exist in NVDA, they should work with or without sound split.

@Adriani90
Copy link
Collaborator

Adriani90 commented Apr 18, 2024 via email

@XLTechie
Copy link
Collaborator

@Adriani90

This will be solved by this PR as far as I understand

The title and description of this PR say otherwise. The code, which I have read, doesn't appear to contain any change that would do what you are saying as far as I can tell.

@CyrilleB79
Copy link
Collaborator

There are many discussions and various open issues/PRs following the mute and volume control for other apps introduced in #16273. This leads to various impacts of this feature being discussed in various places, which is inefficient. This reveals that this feature is not mature enough or that the design and the use cases have not been correctly explained and listed.

I think that @LeonarddeR suggestion to revert the feature until a design with clear use cases is discussed and understood by all is the best course of action.

I am quite disappointed to request this revert because @mltony has done a great job to push this in core and because his availability period may reach its end, so there is a risk that this feature may not be reintegrated soon if he is not available to work on it anymore.
But this seems the best solution if we want a clear and centralized discussion on this feature.

@seanbudd, @gerald-hartig what do you think?

@Adriani90
Copy link
Collaborator

@XLTechie

Can you explain why you say it fixes that issue?

See description of user facing changes. @mltony writes:

Now muting other apps or adjusting other apps volume only works when sound split is not set to off.

And sound split restores apps volume after NVDA exits as far as I understand. @mltony can you comment on this?

Personally, I agree with those who believe that this PR should be closed--if muting and volume changes exist in NVDA, they should work with or without sound split.

I disagree. Decoupling from sound split can be handled afterwards at a later stage, we don't have to wait for the perfect world until something is merged in my view at least. If decoupling is the only point we have to solve, this can be discussed separately.

This leads to various impacts of this feature being discussed in various places, which is inefficient. This reveals that this feature is not mature enough or that the design and the use cases have not been correctly explained and listed.

This will be the case for lots of new features which might come into NVDA in the future. Things will be discussed in different places. These discussions have been taken place because of 3 things:

  1. Reverting the feature (which is not really needed in my view because this feature doesn't degradate user experience, but it improves it)
  2. Decoupling from sound split (which is a different issue and can be handled separately and should not block this feature from being merged)
  3. And restoring volume and mute / unmute state (which will be added after merging this PR and which is the expected behavior in line with audio ducking feature).

I am quite disappointed to request this revert because @mltony has done a great job to push this in core and because his availability period may reach its end, so there is a risk that this feature may not be reintegrated soon if he is not available to work on it anymore.
But this seems the best solution if we want a clear and centralized discussion on this feature.

I strongly disagree with reverting this. Further improvements can be discussed separately and there are alot of features which were introduced in NVDA earlier and did not have enough discussions with the community, still they are accepted and improvement requests are in place. For example OCR feature, Add-on store, object navigation etc.

@josephsl
Copy link
Collaborator

josephsl commented Apr 18, 2024 via email

@mltony mltony changed the title Disable application volume and mute features when sound split is off Disable application volume and mute features when sound split is off or when wasapi is off Apr 18, 2024
@mltony
Copy link
Contributor Author

mltony commented Apr 18, 2024

I just squeezed in a fix for#16409 as well here - hope that's ok.

As for decoupling aplication volume from sound split, it seems to me that more people are in favor of decoupling. Personally I am biased towards decoupling too: I especially like the argument that by switching to ISimpleAudioVolume mute/volume changes will be reflected in Windows Volume mixer. Unless there are any concerns, I'll try to prepare anotehr PR for beta branch in the next couple of days. Or please LMK if that shouldn't go into beta.

@LeonarddeR

I would at least take it back to the drawing board.

It has been at the drawing board stage for quite a while - #16052 was opened in January. It would seem counterproductive to me to roll back features because of concerns (unless such concerns are critical). Open source projects typically don't move very fast to begin with, so we don't want to have analysis paralysis.

@Adriani90
Copy link
Collaborator

Adriani90 commented Apr 18, 2024 via email

@CyrilleB79
Copy link
Collaborator

Tony, as long as the reflected changes in the volume mixer are restored after NVDA exit, I don’t see any issue with that. I think the audio ducking feature reflects also changes there but restores them after NVDA exit.

No, the audio ducking has no impact on Windows Volume Mixer. It's considered a change internal to the screen reader, not from a coding point of view, but in consideration to the fact that the audio ducking volume lowering disappears when the screen reader is exited.

There is no automatic process modifying the Volume Mixer today, and that's why I am advocating for the volume mixer not being modified automatically when NVDA exits. Since they will be reflected in the Volume Mixer, the other apps volume or mute commands should just be considered convenience shortcut to modify settings in the Volume Mixer in one keystroke. This makes it the simplest and clearest user experience.

@mltony by the way, since the volume / mute other apps become convenience shortcuts to control Windows Volume Mixer, I wonders if it's still useful to keep the associated controls in NVDA's GUI (audio panel). Keeping them leads to a greater risk of synchronization issues with the Volume Mixer. You also do not need to consider profile switch or config reset anymore.

@Adriani90
Copy link
Collaborator

Adriani90 commented Apr 21, 2024 via email

@seanbudd
Copy link
Member

Closing in favour of reverting the mute application feature

@mltony
Copy link
Contributor Author

mltony commented Apr 24, 2024

@seanbudd,
I would urge you to reconsider. This feature is an integral part of sound split and without this feature sound split functionality is going to be inadequate - since we'll be able to only split volumes, but not adjust left and right volumes separately.
I think my mistake was to spin it off as a separate feature which causes a lot of confusion.
And BTW this particular bug is fixed in

@cary-rowen
Copy link
Contributor

I also hope to reconsider this. The community has a need for this, and Tony is willing to implement this. If further discussion is needed, please let the community know.

@gerald-hartig
Copy link
Collaborator

Hello everyone. With the public holiday calendar in Australia and various ongoing discussions this week, we have not been able to respond in detail to this issue or #16440. Rather than rush something out and risk a misunderstanding, I wanted to let you know that it's on our list and we'll respond shortly.

@gerald-hartig
Copy link
Collaborator

@cary-rowen I've updated the description of #16440 and put a comment in there.

seanbudd added a commit that referenced this pull request May 10, 2024
Reverts PR
Reverts #16273

Issues fixed
Fixes #16409
Fixes #16402

Issues reopened
#16052

Brief reason for revert
We started with mltony creating:
#16051 Feature request: Sound split

Which was a duplicate of:
#12985 Audio settings with stereo headset (or speakers) - Send NVDA sounds on one side and the rest of Windows sounds on the other side

And then implemented by:
#16071 Sound split

mltony also created:
#16052 Feature request: add command to adjust volume of all applications except for NVDA

Which was implemented in:
#16273 Keystrokes to adjust applications volume and mute

This PR was approved and merged and was then found to cause issues:
#16402 Unmuting other apps does not work as expected
#16409 Apps mute and volume features work very unexpectedly with WASAPI disabled

Due to these issues and the considerable debate on the approach, the above PR #16273 was reverted by:
#16440

As an alternative to the revert #16440 to resolve the 2 issues (#16402, #16409) and keep #16273, mltony created:
#16404

The question now becomes, how do we proceed from here?

NV Access's position is that the sound split functionality (#16051) is a useful feature to add into core. However, due to the following reasons, we believe that further work on the volume adjustment features (#16273, #16404) to improve the UX is required on a branch (off master/alpha) before it can be added back in:

Windows sound mixer has reasonable accessibility.
Sound split on its own provides value to users.
The UX of swapping between NVDA volume control and windows volume control needs to be resolved.
The UX of resolving volume issues due to NVDA crashes needs to be resolved.
As one of the contributors on the threads said, "So now there are two mixers in the chain, one of which can be invisible, and overrides the other, or makes its settings relative instead of absolute."
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocked/needs-product-decision A product decision needs to be made. Decisions about NVDA UX or supported use-cases.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unmuting other apps does not work as expected