H.264 over slow link may be unusable with some apps

[totaam]

Issue migrated from trac ticket # 522

component: core | priority: major | resolution: worksforme

2014-02-18 04:34:26: onlyjob created the issue

---

Recently I tried 0.11.3 over (slow) WAN connection and there was a problem with H.264 encoding on some applications. Notably I was trying to work with kmail with minimum_speed=low_bandwidth and minimum_quality either "low" or "average". As expected initially app. window rendered with very low quality (unreadable text) which improved after several seconds. The problem was that full window refresh were kicking again by merely moving mouse cursor over app. window so visual presentation was degrading window-wide making all text unreadable then seconds later improving to the point when text became readable again only for few seconds then cycle repeated over and over.
Bad-goog-bad cycle forced me to switch encoding to JPEG where situation was slightly better because minimum quality did not make text completely unreadable. There were seemingly unnecessary full window redraws but I could still read the text.

I'm not sure whether the issue is due to applications doing full window redraw or due to (incorrect?) detection of new content or because of too low minimum quality settings in H.264. It could be combination of any of those three or perhaps the lack of detection for changes to avoid re-sending the same data when window simply re-painted exactly the same content...

[totaam]

2014-02-18 04:48:44: totaam commented

---

It is difficult to say, I don't have kmail setup to try it out.

It does sound like kmail is redrawing parts of the screen unnecessarily (many applications do this unfortunately), 0.12 / trunk has a few changes in this area so it is worth a try to see if it improves things:

• it tries harder to use jpeg/png for regions before doing a fullscreen video refresh
• it will try to detect where to use the video encoding to get the best compression (better sub-window encoding: detect regions and use sub video encoder #410: mostly useful for browsers with a flash video region)

If JPEG gives you a better experience, then I think you should still be able to go with h264 and just bump the minimum quality higher. Bandwidth consumption should be lower than with JPEG, it is just a case of finding the quality level that makes this usable. (and this varies widely: text based applications have a very different profile than say, games or graphical apps)

It is also worth noting that by setting the speed to low-bandwidth, things may actually get worse for some applications (especially those that send lots of small updates): as it takes much longer to compress each frame, more updates have time to accumulate, when we do get around to processing them there are too many and we choose a fullscreen update to try to optimize...
Were the results on default settings worse?

There may be more information in xpra info for the window you are interested in: actual encodings used, etc.

[totaam]

2014-02-20 23:57:27: onlyjob commented

---

Apparently kate have a similar problem. To reproduce paste some text, select some and move mouse over the window. Selected area will appear with degraded quality, which will improve and then degrade again etc.

Bumping minimum h264 quality helps. Default settings are not much better than "low-bandwidth" -- perhaps there is small benefit in regards to bandwidth as it feels like quality improves slightly faster. The problem though is that as soon as quality improves it goes back to bad (unreadable) state to repeat redraw cycle all over...

I didn't have a chance to try 0.12 yet...

[totaam]

2014-03-03 04:49:45: totaam changed owner from antoine to onlyjob

[totaam]

2014-03-03 04:49:45: totaam commented

---

Please capture xpra info when the window is shown with degraded quality.

[totaam]

2014-03-03 14:56:26: totaam commented

---

kate does repaint a lot of things on screen, and on a slow link (where the batch delay is going to be high) you are more likely to get a full screen refresh, and therefore more likely to see a subsampled frame. As I said above, v0.12 should be better at this.

[[BR]]

But overall I don't think this is a bug: on a slow link, something has to give. By default we lower both the speed (higher latency to get better compression) and the quality (lower quality to get better compression). If you want to keep a higher quality, the option is there. Some users do want to sacrifice quality for better framerate and others do not, that's why the options are there..

Can you suggest what we should be doing different?

[totaam]

2014-03-18 03:05:26: totaam commented

---

Can I close this? (see previous comments)

[totaam]

2014-03-18 03:43:51: onlyjob commented

---

Yes please close. I didn't have a chance to try 0.12 yet... Is it detect (and skip) refreshes with no changes?

[totaam]

2014-03-18 08:31:38: totaam commented

---

Detecting "no change" is hard, video encodings do it for us as part of their motion detection (no motion in this case). But the problem is that by the time you get there, you've already decided to send a video frame, and so you may already be committed to subsampling its pixels, which means that you will gain nothing.

What 0.12 does is:

• try harder to use lossless regions before switching to video encoding (hopefully enough so that applications that refresh too much, like kate, end up using a lossless encoding more often before switching to video)
• detect video sub regions: when a window refreshes a lot, it is often limited to a sub-region of that window, we try to detect that and send only the fast-refreshing part using video encoding. (this one is probably of a more limited benefit to applications like kate or kmail)

Feel free to re-open if you feel that we should do better on automatic mode. Though it can be argued that problematic applications (kate has been added to [/wiki/ApplicationQuircks Application Quircks]) should be handled manually rather than risking degrading the default behaviour.

[totaam]

2014-03-18 13:57:01: onlyjob commented

---

Makes sense. Thank you for detailed explanation -- it helps to understand what's going on and the challenges. Let's close unless there are any new improvement ideas.

[totaam]

2014-03-18 13:59:05: totaam changed status from new to closed

[totaam]

2014-03-18 13:59:05: totaam changed resolution from ** to worksforme

[totaam]

2014-03-18 13:59:05: totaam commented

---

I'm closing as "worksforme" for lack of a better option, things aren't fixed, just a tad better hopefully.