Adjust controller polling frequency with the audio buffer

[mixxxbot]

Reported by: albeu
Date: 2012-04-29T10:16:32Z
Status: Triaged
Importance: High
Launchpad Issue: lp990992
Tags: midi, performance
Attachments: controller-polling-frequency.patch

---

In the new controller manager the polling frequency is hardcoded to 1KHz, however my system (ubuntu 11.10) use HZ=250. As the timer frequency is higher than the system time slice it completely hog the cpu rendering controllers unusable. In the old system the poll frequency of PortMidi was 250Hz.

[mixxxbot]

Commented by: noname2-deactivatedaccount
Date: 2012-05-04T03:28:12Z

---

wow my laptop is getting hot over 100% cpu usage while doing nothing

[mixxxbot]

Commented by: albeu
Date: 2012-05-06T09:27:35Z
Attachments: controller-polling-frequency.patch

---

Here is the patch to get the polling frequency back to the old setting.

[mixxxbot]

Commented by: Pegasus-RPG
Date: 2012-05-07T07:18:09Z

---

I'd like Mixxx to auto-detect whether a system can support a 1ms timer and automatically use it if so. Does anyone know of a way to do this? Qt doesn't seem to have any mechanism.

[mixxxbot]

Commented by: Pegasus-RPG
Date: 2012-05-07T07:19:03Z

---

Or, ideally, we patch PortMIDI to use a callback-based system, truely solving the problem.

[mixxxbot]

Commented by: albeu
Date: 2012-05-07T07:57:31Z

---

Moving away from polling would be the ideal solution, but I don't know if the PortMIDI API allow it. Otherwise, regarding linux, getting the HZ value of the currently running kernel doesn't seems to be straightforward. Dunno how it's like on other systems, but making the poll frequency configurable is probably the simplest.

[mixxxbot]

Commented by: pwhelan
Date: 2012-05-07T09:27:43Z

---

Neither PortMidi nor RtMidi support any of the event based I/O APIs on any of the platforms or a blocking poll call. This might actually be a portability problem.

As quoted from the PortMidi Doxygen docs at http://portmedia.sourceforge.net/portmidi/doxygen/group__grp__io.html#ge88e6c682ee3550defdd0c74bcbdd508:

"with a lock-free fifo, which is pretty much what we're stuck with to enable portability to the Mac, it's tricky for the producer and consumer to synchronously reset the buffer and resume normal operation."

[mixxxbot]

Commented by: Pegasus-RPG
Date: 2012-05-12T17:29:16Z

---

Does a lock-free FIFO prevent blocking (for a set amount of time) until a message is received like HIDAPI does?

[mixxxbot]

Commented by: raskolnikov
Date: 2012-05-17T17:25:23Z

---

I suffer the same problem, and I am running on a 1000HZ custom kernel.

[mixxxbot]

Commented by: bkgood
Date: 2012-05-21T17:46:32Z

---

I went looking at portaudio's alsa implementation and rtmidi's alsa implementation. Both use polling without burning up users' systems. They both do this by sleeping for a very short (1ms) amount of time when no data is available after a poll() call.

I've implemented something similar to this at lp:~mixxxdevelopers/mixxx/fixes_midi_burnout. Results, in CPU usage:

                                 | Under load | Idle |
Without sleep (current behavior) |  150%      | 110% |
With sleep (as revised)          |   70%      |  35% |

I'll be doing a merge request shortly.

[mixxxbot]

Commented by: bkgood
Date: 2012-05-21T17:48:24Z

---

Bleh so much for my lovely table. Basically, without my changes, I see 150%/110% CPU usage under load/idle. With the patch, it's 70%/35% (the under load figure doesn't include any analyzers).

[mixxxbot]

Commented by: bkgood
Date: 2012-05-21T17:54:14Z

---

If someone whose system is noticeably heated by the unrestricted polling can test this it'd be much appreciated, I'm in the process of moving country so I just had enough time to write a fix and watch htop for a few minutes.

[mixxxbot]

Commented by: rryan
Date: 2012-05-21T18:27:42Z

---

Can you give the load/idle numbers for no MIDI device enabled? (that way we
see a baseline for no polling going on at all)

On Mon, May 21, 2012 at 1:54 PM, Bill Good wrote:

If someone whose system is noticeably heated by the unrestricted polling
can test this it'd be much appreciated, I'm in the process of moving
country so I just had enough time to write a fix and watch htop for a
few minutes.

--
You received this bug notification because you are a member of Mixxx
Development Team, which is subscribed to Mixxx.
https://bugs.launchpad.net/bugs/990992

Title:
The new controller polling frequency is too high for desktop systems

To manage notifications about this bug go to:
https://bugs.launchpad.net/mixxx/+bug/990992/+subscriptions

[mixxxbot]

Commented by: xorik
Date: 2012-05-21T18:31:16Z

---

I can confirm, mixxx from lp:~mixxxdevelopers/mixxx/fixes_midi_burnout not uses more then 100% CPU, when controller is pluged in!

[mixxxbot]

Commented by: rryan
Date: 2012-05-21T18:35:06Z

---

Also, I don't really see why you use QThread::msleep(1) instead of just
increasing kPollIntervalMillis to 2 in controllermanager.cpp. We are
polling on a timer in our own poll thread. msleeping a callback in that
thread should be roughly identical to setting the polling interval higher.
Based on that reasoning it seems to make sense that by doing msleep(1) you
cut your CPU usage roughly in half since the poll frequency will roughly
halve

On Mon, May 21, 2012 at 2:27 PM, RJ Ryan wrote:

Can you give the load/idle numbers for no MIDI device enabled? (that way
we see a baseline for no polling going on at all)

On Mon, May 21, 2012 at 1:54 PM, Bill Good wrote:

If someone whose system is noticeably heated by the unrestricted polling
can test this it'd be much appreciated, I'm in the process of moving
country so I just had enough time to write a fix and watch htop for a
few minutes.

--
You received this bug notification because you are a member of Mixxx
Development Team, which is subscribed to Mixxx.
https://bugs.launchpad.net/bugs/990992

Title:
The new controller polling frequency is too high for desktop systems

To manage notifications about this bug go to:
https://bugs.launchpad.net/mixxx/+bug/990992/+subscriptions

[mixxxbot]

Commented by: noname2-deactivatedaccount
Date: 2012-05-21T18:50:23Z

---

come on guys just revert it to 250hz or lower. it should be enough

[mixxxbot]

Commented by: bkgood
Date: 2012-05-22T00:46:00Z

---

Can you give the load/idle numbers for no MIDI device enabled? (that way we
see a baseline for no polling going on at all)

12% idle/55% load.

I have no issue with simply increasing the polling interval.

Also, I don't really see why you use QThread::msleep(1) instead of just
increasing kPollIntervalMillis to 2 in controllermanager.cpp.

It only sleeps when the poll didn't see any data. So if mixxx starts taking in a stream of data (scratching), it'll hopefully not sleep. It does increase latency from no data to data though, which will be common scenario (no midi input to hitting a button, for instance). Honestly I thought there were limitations of midi over usb that prevented data from coming in at 1ms intervals so this could all be for naught.

[mixxxbot]

Commented by: albeu
Date: 2012-05-22T08:39:12Z

---

On Mon, 21 May 2012 18:50:23 -0000
julius von kohout <email address hidden> wrote:

come on guys just revert it to 250hz or lower. it should be enough

I fully agree. I'm using the current trunk with 250Hz and it work
great. Keep it simple!

Alban

[mixxxbot]

Commented by: zestoi
Date: 2012-05-22T18:21:01Z

---

i like bill's method - it does seem to be exactly how rtmidi handles the same thing, to emulate blocking nreads and callbacks.

[mixxxbot]

Commented by: rryan
Date: 2012-05-24T15:11:44Z

---

There are other parts of Mixxx that we use a 1ms timer, for example in the script-engine scratch timers.

@juliusvonkohout we are searching for the right fix. "just reverting it" doesn't give us any useful information about how to avoid issues like this in the future. High-frequency timers are a very necessary part of getting rid of latency in Mixxx and making it more responsive.

@Bill -- we must be missing something... if RtMidi sleeps for 1ms when no input is available then we are sleeping strictly /more/ often than RtMidi (we sleep 1ms regardless of whether input is available) so where is the difference?

[mixxxbot]

Commented by: albeu
Date: 2012-05-24T16:22:56Z

---

On Thu, 24 May 2012 15:11:44 -0000
RJ Ryan <email address hidden> wrote:

@juliusvonkohout we are searching for the right fix. "just reverting it"
doesn't give us any useful information about how to avoid issues like
this in the future. High-frequency timers are a very necessary part of
getting rid of latency in Mixxx and making it more responsive.

Reliable timer with 1ms accuracy are just not possible without a
realtime OS. In my experience anything under 10ms is not reliable
enough to be really useful on a desktop OS.

If the design really require such high frequency timer it is a big flaw
IMHO. One must use OS events (select() and the like) to efficiently
implement anything that would otherwise need polling with more than
100Hz. Desktop OS are just no realtime OS with guaranteed timing
constraints!

Alban

[mixxxbot]

Commented by: rryan
Date: 2012-05-24T16:43:07Z

---

Hi Alban -- while it's true that without a realtime kernel you lose out on
a lot of guarantees about scheduling, scheduling is /unrelated/ to timing
accuracy in this case. Scheduling describes when your handler for a timer
is scheduled, not when the timer itself is triggered as complete by the OS.

Using basic timing primitives like select() is bad because you tie yourself
to the system timer and your timer can't complete between jiffies. It's not
true that those are the only option. Linux has a high-resolution timer
framework that doesn't require having a realtime kernel. In fact, Ubuntu
comes with the HRT system enabled by default. This high-res timer framework
allows your timer to trigger between jiffies. I think we should use this
instead of QTimer simply because it's more accurate.

To address your point about scheduling.. we strongly suggest people enable
realtime scheduling by tweaking their /etc/security/limits.conf file on
Linux (note we don't suggest people use a realtime kernel since there are
many stability issues with realtime kernels). This allows Mixxx to set its
mixing thread to have higher priorities over its other threads. It's not
true realtime but it's much better than the default.

High-resolution, accurate timing is a requirement of any DSP-heavy or
latency-sensitive application. Even just processing jog-wheels accurately
and with no drift requires accurate timing. There's no way around this. :)

On Thu, May 24, 2012 at 12:22 PM, Alban wrote:

On Thu, 24 May 2012 15:11:44 -0000
RJ Ryan wrote:

@juliusvonkohout we are searching for the right fix. "just reverting it"
doesn't give us any useful information about how to avoid issues like
this in the future. High-frequency timers are a very necessary part of
getting rid of latency in Mixxx and making it more responsive.

Reliable timer with 1ms accuracy are just not possible without a
realtime OS. In my experience anything under 10ms is not reliable
enough to be really useful on a desktop OS.

If the design really require such high frequency timer it is a big flaw
IMHO. One must use OS events (select() and the like) to efficiently
implement anything that would otherwise need polling with more than
100Hz. Desktop OS are just no realtime OS with guaranteed timing
constraints!

Alban

--
You received this bug notification because you are a member of Mixxx
Development Team, which is subscribed to Mixxx.
https://bugs.launchpad.net/bugs/990992

Title:
The new controller polling frequency is too high for desktop systems

To manage notifications about this bug go to:
https://bugs.launchpad.net/mixxx/+bug/990992/+subscriptions

[mixxxbot]

Commented by: rryan
Date: 2012-05-24T16:53:15Z

---

In general it doesn't make sense that scheduling a timer for 1ms when the
system tick is 250Hz would inherently cause any additional CPU burden than
if the system tick were 1000Hz. The only thing that should happen in that
case is that the timer will be triggered later than 1ms -- potentially 4ms
later.

Since we're asking for a 1ms repeated timer from QTimer, is it possible
that when the system tick expires, QTimer realizes that 4 timer events
should have happened and calls them all at once? This would cause us to
issue 4 polls every 4ms instead of 1 poll every 1ms.

On Thu, May 24, 2012 at 12:43 PM, RJ Ryan wrote:

Hi Alban -- while it's true that without a realtime kernel you lose out on
a lot of guarantees about scheduling, scheduling is /unrelated/ to timing
accuracy in this case. Scheduling describes when your handler for a timer
is scheduled, not when the timer itself is triggered as complete by the OS.

Using basic timing primitives like select() is bad because you tie
yourself to the system timer and your timer can't complete between jiffies.
It's not true that those are the only option. Linux has a high-resolution
timer framework that doesn't require having a realtime kernel. In fact,
Ubuntu comes with the HRT system enabled by default. This high-res timer
framework allows your timer to trigger between jiffies. I think we should
use this instead of QTimer simply because it's more accurate.

To address your point about scheduling.. we strongly suggest people enable
realtime scheduling by tweaking their /etc/security/limits.conf file on
Linux (note we don't suggest people use a realtime kernel since there are
many stability issues with realtime kernels). This allows Mixxx to set its
mixing thread to have higher priorities over its other threads. It's not
true realtime but it's much better than the default.

High-resolution, accurate timing is a requirement of any DSP-heavy or
latency-sensitive application. Even just processing jog-wheels accurately
and with no drift requires accurate timing. There's no way around this. :)

On Thu, May 24, 2012 at 12:22 PM, Alban wrote:

On Thu, 24 May 2012 15:11:44 -0000
RJ Ryan wrote:

@juliusvonkohout we are searching for the right fix. "just reverting it"
doesn't give us any useful information about how to avoid issues like
this in the future. High-frequency timers are a very necessary part of
getting rid of latency in Mixxx and making it more responsive.

Reliable timer with 1ms accuracy are just not possible without a
realtime OS. In my experience anything under 10ms is not reliable
enough to be really useful on a desktop OS.

If the design really require such high frequency timer it is a big flaw
IMHO. One must use OS events (select() and the like) to efficiently
implement anything that would otherwise need polling with more than
100Hz. Desktop OS are just no realtime OS with guaranteed timing
constraints!

Alban

--
You received this bug notification because you are a member of Mixxx
Development Team, which is subscribed to Mixxx.
https://bugs.launchpad.net/bugs/990992

Title:
The new controller polling frequency is too high for desktop systems

To manage notifications about this bug go to:
https://bugs.launchpad.net/mixxx/+bug/990992/+subscriptions

[mixxxbot]

Commented by: Pegasus-RPG
Date: 2012-05-24T17:34:49Z

---

In addition, see #⁠8: someone with a 1000Hz kernel has the same problem. There's something else mysterious going on here. I just wish one of our devs had the problem. raskolnikov (or anyone affected by this): are you gather some profiling data for us? We would need you to build Mixxx from source with profiling=1, run it, try to use the MIDI controller for about a minute, then close Mixxx. Then run gprof ./mixxx and attach the output to this bug.

[mixxxbot]

Commented by: rryan
Date: 2012-05-24T18:58:55Z

---

I dug into the QEventDispatcher code a little bit and it looks like my
hypothesis that multiple timer invocations come after a tick is wrong. On
UNIX, Qt checks whether multiple timer periods have elapsed and only sends
one timer event. Timer waits are accomplished by setting the timeout of the
event loop select() to the time to the next timer event, so this is
inherently limited to boundaries of jiffies.

Other interesting tidbits from reading the Qt source is that for timers
below 20ms on Windows, Qt uses the WinMM timeSetEvent method which provides
millisecond accuracy but from various Googling appears to be able to
reliably deliver 1ms callbacks with low jitter. Good to know.

On Thu, May 24, 2012 at 1:34 PM, Sean M. Pappalardo <
<email address hidden>> wrote:

In addition, see #⁠8: someone with a 1000Hz kernel has the same problem.
There's something else mysterious going on here. I just wish one of our
devs had the problem. raskolnikov (or anyone affected by this): are you
gather some profiling data for us? We would need you to build Mixxx from
source with profiling=1, run it, try to use the MIDI controller for
about a minute, then close Mixxx. Then run gprof ./mixxx and attach the
output to this bug.

--
You received this bug notification because you are a member of Mixxx
Development Team, which is subscribed to Mixxx.
https://bugs.launchpad.net/bugs/990992

Title:
The new controller polling frequency is too high for desktop systems

To manage notifications about this bug go to:
https://bugs.launchpad.net/mixxx/+bug/990992/+subscriptions

[mixxxbot]

Commented by: albeu
Date: 2012-05-25T11:37:08Z

---

On Thu, 24 May 2012 16:43:07 -0000
RJ Ryan <email address hidden> wrote:

Hi Alban -- while it's true that without a realtime kernel you lose
out on a lot of guarantees about scheduling, scheduling
is /unrelated/ to timing accuracy in this case. Scheduling describes
when your handler for a timer is scheduled, not when the timer itself
is triggered as complete by the OS.

Assuming the handler thread is sleeping, waiting for some event or
sleeping, it still need to be rescheduled by the kernel, and there it
will most likely have to wait for the next jiffy.

Using basic timing primitives like select() is bad because you tie
yourself to the system timer and your timer can't complete between
jiffies. It's not true that those are the only option. Linux has a
high-resolution timer framework that doesn't require having a
realtime kernel. In fact, Ubuntu comes with the HRT system enabled by
default. This high-res timer framework allows your timer to trigger
between jiffies. I think we should use this instead of QTimer simply
because it's more accurate.

High resolution doesn't imply that you can use high frequency. You
can schedule with better accuracy, but it doesn't change much on how
often you can schedule a given action. The CPU just can run that much
code in a given time, at some point too much task switching is just
wasting time.

To address your point about scheduling.. we strongly suggest people
enable realtime scheduling by tweaking
their /etc/security/limits.conf file on Linux (note we don't suggest
people use a realtime kernel since there are many stability issues
with realtime kernels). This allows Mixxx to set its mixing thread to
have higher priorities over its other threads. It's not true realtime
but it's much better than the default.

This only allow fine tuning the priority between the threads, a good
negative nice get one a bit further to avoiding being unscheduled, but
it will still have to happen sometimes.

High-resolution, accurate timing is a requirement of any DSP-heavy or
latency-sensitive application. Even just processing jog-wheels
accurately and with no drift requires accurate timing. There's no way
around this. :)

Sure high resolution help a lot, but we are talking about wasting
time checking for events that the kernel probably knew weren't there in
the first place.

Alban

[mixxxbot]

Commented by: rryan
Date: 2012-06-14T17:37:59Z

---

Bill -- Hm, sleeping the controller manager thread isn't going to work since that also blocks other processing done in that thread for 1ms (e.g. midi script timers, control object changes, etc.). For 1.11.0 beta1 I'm just going to ifdef the polling frequency to 5ms for Linux.

I do like your branch's changes though.. except that I think if we processed events while polling we should immediately poll again (this matches RtMidi's behavior and our 1.10.0 and below behavior). So I'm merging it now. Thanks!

[mixxxbot]

Commented by: daschuer
Date: 2013-03-20T07:46:16Z

---

This bug is fixed for 1.11.0 isn't it?
I have opened a new bug Bug #⁠1157579 with a different approach to sync the polling with the audio callback

[mixxxbot]

Commented by: Pegasus-RPG
Date: 2013-07-12T06:50:27Z

---

This is not fixed in 1.11, as there's just an IFDEF that sets the polling frequency to 5ms on Linux only. Syncing the polling with the audio callback is an interesting idea, so we should explore that too.

[daschuer]

Duplicates: #6952