Audio settings with stereo headset (or speakers) - Send NVDA sounds on one side and the rest of Windows sounds on the other side #12985
Comments
|
|
I think this would also be a cool feature to have. However I suspect it requires some significant interaction with the windows system or similar API or something like that. I'm not sure how JAWS got around the problem, but there's probably a way.. |
|
I've actually had this in my plans to implement this in one of my add-ons when time permits. |
|
Implemented in Tony's enhancements add-on. Sound splitting can be toggled by pressing NVDA+Alt+S. |
|
Thanks so much @mltony for this. |
|
Thanks to tony’s efforts, it seems that the sound can only be fixed to the right at the moment,Can the left and right sides be realized? |
|
@cary-rowen yes, if you go to NVDA's settings, under the Tony's enhancements panel, check the checkbox named Switch left and right during sound split. This will revert it, meaning that NVDA will be on the left and everything else on the right. |
|
|
In the next version I will make it restore sound before quitting NVDA.
…On 11/13/2021 4:53 PM, nidza07 wrote:
Thanks so much @mltony <https://github.com/mltony> for this.
Might be better to move to your repo and create a separate issue, but
there is currently one case you are not handling.
If you enable sound split, and then quit NVDA, the rest of the audio
remains on one side instead of being restored.
Now, if you start NVDA, NVDA goes to center obviously since sound
split is no longer enabled, but the rest remains on one side.
Hope that is clear enough and that you can fix this in a future build.
Thanks once again for taking this up.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#12985 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AIJRDHFADDW6GAHCGBWREYLUL4CBXANCNFSM5GVCIK2Q>.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
|
|
Hi, Partly resolved with add-ons from Tony and yours truly. Thanks. |
|
Hi, @josephsl Thanks |
|
Hi, I have no plans to work further on Sound Splitter, nor plan to bring this to NVDA Core as I’m about to accelerate my transition away from NVDA Core contributions. Thanks.
|
|
Then what was the point of forking from Tony's enhancements? Initially
you said that you'd fork specifically to prepare for merging into NVDA
core PR.
…On 5/2/2022 1:17 AM, Joseph Lee wrote:
Hi, I have no plans to work further on Sound Splitter, nor plan to
bring this to NVDA Core as I’m about to accelerate my transition away
from NVDA Core contributions. Thanks.
—
Reply to this email directly, view it on GitHub
<#12985 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AIJRDHDNKCHPFAO2OPQTLFDVH6FQRANCNFSM5GVCIK2Q>.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
Hi, indeed, the original intention was to test the feasibility of merging into Core if possible. But I realized that we can run into dependency issues (psutil, for instance), something we can work around by reminding ourselves to update dependencies annually if possible; a better solution is changing audio volume levels without use of additional dependencies. The bigger reason for not continuing with this work was due to life priorities – I’ve been letting go of add-ons to reduce workload. But the fundamentals of bringing sound splitting functionality to NVDA Core is present in our add-ons: app volume manipulation, nvwave changes, handling profile switches directly from nvwave module, and incorporating add-on settings to appropriate settings panel (likely speech panel). Thanks for your concerns (I’m sorry for bringing this news).
|
|
@CyrilleB79 is this somehow a duplicate of #13818? I think there is more technical discussion going on, I wonder if we can close this one in favor of #13818. |
I agree that the implementation of both issues may be based on the same functions. But the use cases are different. |
|
There are already three add-ons that provide this feature: Sound Splitter (cc @XLTechie), Tony's Enhancements (cc @cary-rowen, @mltony) and NVDAExtensionGlobalPlugin (cc @paulber19). WASAPI is now integrated in alpha and is activated by default. Unfortunately, from what I know, the authors or contributors of these add-ons have not yet found a compatible way to implement the sound splitting feature. If no solution is found, there is the risk that many user disable the WASAPI checkbox. In addition, unfortunately, we do not have a FeatureFlag on this feature to manage it more precisely. Would this be the opportunity to include sound split it in NVDA? Is this something hard to do or not? If not and since you have worked recently on WASAPI, @jcsteh would you be interested in implementing it? @seanbudd and @michaelDCurran what do |
|
I'm not willing to implement the feature in its entirety, as that means figuring out the right user experience and interface and I just don't have the time to go back and forth on that. However, I might be able to make it possible for add-ons to do this easily. I could provide a setChannelVolume method on WavePlayer instances or something like that. I'd also need to know what extensionPoints would be required for an add-on to implement this without monkey patching. For example, I'm guessing we'd need actions for when a stream is opened, when it is stopped, etc. Alternatively, I could implement the backend parts of this (setChannelVolume or similar) and someone else could implement the UX/UI for channel splitting, working with others to figure out the best UX/UI. |
I think that would be nice too, that's enough, someone will be willing to put in the effort on the UI/UX side. |
@jcsteh thanks for this proposal. Such contribution would be very helpful. Add-on authors or anyone wanting to implement it in core will be able to provide a GUI without any problem. To help you, maybe add-on authors (@cary-rowen, @XLTechie and @paulber19) can indicate the functions and extension points taht they would need in the API. That is indicate the private functions they use in their add-on and the functions that they are monkey patching. |
|
jcsteh Thank you for your consideration. We already have UI for this behavior, extant in at least three add-ons; though one of them, Sound Splitter, is an updated version of what was already being done in one of the others. @josephsl did that work, but I recently started maintaining it. Based on my grasp of what's going on in the code used, we would definitely need channel based volume control as you mentioned, and probably the session notification Actions you talked about. I haven't looked into how WASAPI differs in the way it handles audio, so I don't know what else we would need, but that sounds like the minimum (and maybe also the maximum). Below, I have pasted the most important code that is used to patch stable NVDA to accomplish this, minus the obvious parts that you can infer. The full sourcecode is available here. Code Excerpts...import ctypes
import types
import os
import core
import nvwave
winmm = ctypes.windll.winmm
originalWaveOpen = None
def noWASAPI_preWaveOpen(selfself, *args, **kwargs):
global originalWaveOpen
result = originalWaveOpen(selfself, *args, **kwargs)
# All we care about is splitting sounds, so set volume to 100 percent always.
volume = 100
volume2 = int(0xFFFF * (volume / 100))
if not config.conf["soundSplitter"]["soundSplit"]:
volume2 = volume2 | (volume2 << 16)
else:
if config.conf["soundSplitter"]["soundSplitLeft"]:
pass
else:
volume2 = (volume2 << 16)
winmm.waveOutSetVolume(selfself._waveout, volume2)
return result
if not isUsingWASAPI():
preWaveOpen = noWASAPI_preWaveOpen
def setAppsVolume(volumes=None, exit=False):
from . pycaw.pycaw import AudioUtilities, IChannelAudioVolume
if volumes is not None:
leftVolume, rightVolume = volumes
else:
volume = 100
# By default, if enabled, NVDA is on the right, left only is on the left
if config.conf["soundSplitter"]["soundSplit"]:
# If this option is set, NVDA is on the left, and right sounds are on the right
if config.conf["soundSplitter"]["soundSplitLeft"]:
leftVolume = 0
rightVolume = volume
else:
leftVolume = volume
rightVolume = 0
else:
leftVolume = rightVolume = volume
leftVolume /= 100.0
rightVolume /= 100.0
audioSessions = AudioUtilities.GetAllSessions()
for s in audioSessions:
if (
not exit
and s.Process is not None
and s.ProcessId == os.getpid() # FixMe should this be here? (Luke)
):
continue
channelVolume = s._ctl.QueryInterface(IChannelAudioVolume)
if channelVolume.GetChannelCount() == 2:
channelVolume.SetChannelVolume(0, leftVolume, None)
channelVolume.SetChannelVolume(1, rightVolume, None)
soundSplitterMonitorCounter = 0
def soundSplitterMonitorThread(localSoundSplitterMonitorCounter):
global soundSplitterMonitorCounter
while localSoundSplitterMonitorCounter == soundSplitterMonitorCounter:
if (
not config.conf["soundSplitter"]["soundSplit"]
or isUsingWASAPI()
):
return
setAppsVolume()
yield 1000
def updateSoundSplitterMonitorThread(exit=False):
global soundSplitterMonitorCounter
soundSplitterMonitorCounter += 1
if exit:
setAppsVolume((100, 100), exit=True)
return
ss = config.conf["soundSplitter"]["soundSplit"]
if ss:
executeAsynchronously(soundSplitterMonitorThread(soundSplitterMonitorCounter))
else:
setAppsVolume()
def executeAsynchronously(gen):
"""This function executes a generator in such a manner, that allows updates
from the operating system to be processed during execution.
Specifically, every time the generator yields a positive number,,
the rest of the generator function will be executed from within wx.CallLater().
If the generator yields a value of 0, then the rest of the generator
will be executed from within wx.CallAfter().
This allows clear and simple expression of the logic inside the generator body,
while still allowing NVDA to process update events from the operating system.
Essentially the generator will be paused every time it calls yield, then the updates will be
processed by NVDA and then the remainder of generator function will continue executing.
"""
if not isinstance(gen, types.GeneratorType):
raise Exception("A generator is required")
try:
value = gen.__next__()
except StopIteration:
return
# FixMe: The below line may require a rewrite as multiple Flake8/lint errors are raised.
l = lambda gen=gen: executeAsynchronously(gen) # NOQA
core.callLater(value, executeAsynchronously, gen)
updateSoundSplitterMonitorThread()
class GlobalPlugin(globalPluginHandler.GlobalPlugin):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
global originalWaveOpen
if not isUsingWASAPI():
originalWaveOpen = nvwave.WavePlayer.open
nvwave.WavePlayer.open = preWaveOpen
def terminate(self):
global originalWaveOpen
updateSoundSplitterMonitorThread(exit=True)
if not isUsingWASAPI():
nvwave.WavePlayer.open = originalWaveOpen
super().terminate() # Probably unnecessary but maybe needed in the future |
|
It looks like you already have everything you need in the setAppsVolume function. NVDA's main output (usually just speech) and sounds are now in separate sessions. You can get the session GUIDs using nvwave.defaultSession.guid and nvwave.soundsSession.guid. You could adjust the channels for those sessions as needed. That said, it'd obviously be better if this wasn't done with monkey patching in future. |
|
Now I'm realising you don't even want to separate NVDA speech from NVDA sounds. But I don't understand why you have the monkey patching winmm path when you have setAppsVolume which can already adjust all channel volumes, including NVDA. Can you explain? |
|
@josephsl can you explain, as you were the one who (re)wrote this code?
I am a bit perplexed about why some of this is being done. I know what's being
done, but not why it has to be.
And I haven't had time since taking over to take the code back down to bedrock,
and figure out what breaks when it's done in a more straight forward way. The
code documentation is somewhat lacking.
|
|
Hi, the code was originally written by Tony Malykh as part of his Tony’s Enhancements add-on and I simplified it somewhat. I know that it patches parts of nvwave to provide channel splitting functionality by manipulating app volume. Thanks.
|
|
So the question is why do we need monkey patching in sound splitter, right? |
|
Quote from @mltony in #16051 regarding UX and code design for the implementation.
|
|
The current Sound Splitter add-on (which I have been slow to update for WASAPI),
has the ability to use the keysstroke to cycle between off, left, or right.
I suggest, for the audio settings panel interface, a combo box with the
following options:
* Off
* NVDA on the right
* NVDA on the left
* Selected by keyboard
As an end result, the toggle/cycle key will either switch between off and left,
off and right, or off, left, right, as it does now in the add-on.
Alternatively, this could be done with checkboxes:
* Enable NVDA on the right
* Enable NVDA on the left
If both are checked, then the shortcut works to cycle through all three options.
Otherwise it is a toggle between off and the selected option.
Note: I don't care about these names, they are only for example to make clear
what I am suggesting.
I thought of doing something like this in the add-on when I updated it, but
depending on when this feature appears, I likely won't bother.
|
|
@michaelDCurran, @seanbudd, since guys closed my other issue, I would like to align with you here on the following two issues before I can start implementing this:
So do I have green light on copying Com APIs from PyCaw library and to create a new thread within NVDA - that thread would only be active as long as sound split is active? |
|
Yes, we would accept the comtypes COM interface definitions from pyCaw. As long as the file clearly stated where it came from. |
|
@mltony Have you looked at how NVDA Global Extension add-on is doing this for
WASAPI? I believe @CyrilleB79 suggested I use the same method in Sound Splitter,
and perhaps there is an approach there you can utilize some part of.
|
|
@XLTechie, what exactly are you talking about? I looked at global extension a while ago and the guy just copied relevant code from Tony's Enhancements. So yeah, I'm familiar with that approach because I wrote it in the first place. |
In the meantime, NVDA Global Extensions' sound splitting capability works both with WASAPI. If I am not mistaken, Sound Splitter and Tony's Enhancements instead have only sound splitting working when WASAPI is disabled. Cc @paulber19 in case you have some advice to give. |
|
All I can specify is that today, the "NVDAExtensionGlobalPlugin" add-on
no longer includes a single line of code from the "TonyEnhancements"
add-on and that only the pycaw library is used to control the volume of
NVDA channels and applications.
The add-on works with Wasapi enabled or not.
Le 17/01/2024 22:45, Cyrille Bougot a écrit :
…
@XLTechie <https://github.com/XLTechie>, what exactly are you
talking about? I looked at global extension a while ago and the
guy just copied relevant code from Tony's Enhancements. So yeah,
I'm familiar with that approach because I wrote it in the first place.
In the meantime, NVDA Global Extensions' sound splitting capability
works both with wASAPI. If I am not mistaken, Sound Splitter and
Tony's Enhancements instead have only sound splitting working when
WASAPI is disabled.
Hence the suggestion to look at NVDA Global Extensions add-on.
Cc @paulber19 <https://github.com/paulber19> in case you have some
advice to give.
—
Reply to this email directly, view it on GitHub
<#12985 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADZLFFFX2BS6CWQRSDSNK7TYPBA7RAVCNFSM5GVCIK22U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBZGY4TCNZZGY2Q>.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Fixes #12985 Summary of the issue: Feature request: sound split. Splits system sound into two channels: NVDA speaks in one channel (e.g. left), while all other applications play their sound in the other channel (e.g. right). Description of user facing changes Added global command NVDA+alt+s that toggles sound split between off, NVDA on the left and NVDA on the right (default behavior). Added combo box on Audio panel in NVDA settings that also allows to switch between Sound split modes. Added list of checkboxes in Audio panel, that allows to change behavior of NVDA+alt+s command: it allows to select all modes that the global command will cycle through. Description of development approach Added pycaw library as a dependency. Created file source\audio\soundSplit.py where I implemented all logic. Contrary to what I said before, I managed to implement sound split without creating an extra monitor thread. It works like this: When sound split is toggled, it uses IAudioSessionEnumerator to set volume in all currently active audio sessions. Then it does sessionManager.RegisterSessionNotification() to create a callback that listens for any new audio sessions being created, an it executes the the same volume updating function upon creation. On the next call or on shutdown we unregister the previous notification callback.
Fixes nvaccess#12985 Summary of the issue: Feature request: sound split. Splits system sound into two channels: NVDA speaks in one channel (e.g. left), while all other applications play their sound in the other channel (e.g. right). Description of user facing changes Added global command NVDA+alt+s that toggles sound split between off, NVDA on the left and NVDA on the right (default behavior). Added combo box on Audio panel in NVDA settings that also allows to switch between Sound split modes. Added list of checkboxes in Audio panel, that allows to change behavior of NVDA+alt+s command: it allows to select all modes that the global command will cycle through. Description of development approach Added pycaw library as a dependency. Created file source\audio\soundSplit.py where I implemented all logic. Contrary to what I said before, I managed to implement sound split without creating an extra monitor thread. It works like this: When sound split is toggled, it uses IAudioSessionEnumerator to set volume in all currently active audio sessions. Then it does sessionManager.RegisterSessionNotification() to create a callback that listens for any new audio sessions being created, an it executes the the same volume updating function upon creation. On the next call or on shutdown we unregister the previous notification callback.
Reverts PR Reverts #16273 Issues fixed Fixes #16409 Fixes #16402 Issues reopened #16052 Brief reason for revert We started with mltony creating: #16051 Feature request: Sound split Which was a duplicate of: #12985 Audio settings with stereo headset (or speakers) - Send NVDA sounds on one side and the rest of Windows sounds on the other side And then implemented by: #16071 Sound split mltony also created: #16052 Feature request: add command to adjust volume of all applications except for NVDA Which was implemented in: #16273 Keystrokes to adjust applications volume and mute This PR was approved and merged and was then found to cause issues: #16402 Unmuting other apps does not work as expected #16409 Apps mute and volume features work very unexpectedly with WASAPI disabled Due to these issues and the considerable debate on the approach, the above PR #16273 was reverted by: #16440 As an alternative to the revert #16440 to resolve the 2 issues (#16402, #16409) and keep #16273, mltony created: #16404 The question now becomes, how do we proceed from here? NV Access's position is that the sound split functionality (#16051) is a useful feature to add into core. However, due to the following reasons, we believe that further work on the volume adjustment features (#16273, #16404) to improve the UX is required on a branch (off master/alpha) before it can be added back in: Windows sound mixer has reasonable accessibility. Sound split on its own provides value to users. The UX of swapping between NVDA volume control and windows volume control needs to be resolved. The UX of resolving volume issues due to NVDA crashes needs to be resolved. As one of the contributors on the threads said, "So now there are two mixers in the chain, one of which can be invisible, and overrides the other, or makes its settings relative instead of absolute."
I have had this feature request in mind since many years. But I finally open this ticket thanks to another person asking it on a French mailing list.
Is your feature request related to a problem? Please describe.
When participating to audio-conferences on the computer, it's often confusing to have NVDA's vocal feedback mixed up with the audio stream of the conference.
Describe the solution you'd like
NVDA audio stream is produced by synthesizer output but also beeps (progress bars, indent reporting) as well as other sounds such as focus/browse mode and spelling mistakes detection.
With a stereo headset or stereo speakers, it would be handy to have an option allowing to split the audio stream in two parts:
Describe alternatives you've considered
But IMO it would be valuable to have this feature integrated in NVDA or in an add-on rather that an external application. It would allow many people to discover this feature even if they do not express the need clearly.
Also it would be a plus to have this feature available in a portable version of NVDA; I do not know if it is technically doable however.
Additional context
Jaws 2022 will provide this feature (and that's what convinced me that this feature could be embedded in a screen reader). Below is a quote of its release notes (What's new):
I do not advocate for copying all features of Jaws in NVDA. But I think in this case that having this feature integrated in NVDA is a real plus.
The text was updated successfully, but these errors were encountered: