rpi-4.19-rt

[antonellocaroli]

there's going to be a 4.19 real time version?

[pelwell]

I hope so. We are currently relying on a volunteer - Tiejun Chen - to maintain the branch, and other commitments have meant that he hasn't had time recently.

[paul-1]

The RT patches apply cleanly to the 4.19.y branch. Been running pretty solid.

[vanfanel]

@paul-1 : I am trying to apply the RT patch to 4.19.y branch, too.
Looking at the contents of the patch TAR in , I see there is a list of patches available:

0009-kthread-convert-worker-lock-to-raw-spinlock.patch
0012-arm-Convert-arm-boot_lock-to-raw.patch
0023-EXP-rcu-Revert-expedited-GP-parallelization-cleverne.patch
0028-sched-migrate_disable-Add-export_symbol_gpl-for-__mi.patch
0032-signal-Revert-ptrace-preempt-magic.patch
0036-rt-Provide-PREEMPT_RT_BASE-config-switch.patch
0052-x86-Use-generic-rwsem_spinlocks-on-rt.patch
0059-preempt-Provide-preempt_-_-no-rt-variants.patch
0061-rt-Add-local-irq-locks.patch
0066-buffer_head-Replace-bh_uptodate_lock-for-rt.patch
0067-fs-jbd-jbd2-Make-state-lock-and-journal-head-lock-rt.patch
0070-genirq-Disable-irqpoll-on-rt.patch
0076-mm-page_alloc-rt-friendly-per-cpu-pages.patch
0077-mm-swap-Convert-to-percpu-locked.patch
0098-time-hrtimer-avoid-schedule_work-with-interrupts-dis.patch
0099-hrtimer-consolidate-hrtimer_init-hrtimer_init_sleepe.patch
0100-hrtimers-Prepare-full-preemption.patch
0101-hrtimer-by-timers-by-default-into-the-softirq-contex.patch
0102-sched-fair-Make-the-hrtimers-non-hard-again.patch
0103-hrtimer-Move-schedule_work-call-to-helper-thread.patch
0104-hrtimer-move-state-change-before-hrtimer_cancel-in-d.patch
0105-posix-timers-Thread-posix-cpu-timers-on-rt.patch
0115-rt-Increase-decrease-the-nr-of-migratory-tasks-when-.patch
0128-rtmutex-trylock-is-okay-on-RT.patch
0130-rtmutex-Handle-the-various-new-futex-race-conditions.patch
0135-locking-locktorture-Do-NOT-include-rwlock.h-directly.patch
0136-rtmutex-Add-rtmutex_lock_killable.patch
0137-rtmutex-Make-lock_killable-work.patch
0139-rtmutex-Avoid-include-hell.patch
0141-rtmutex-Provide-rt_mutex_slowlock_locked.patch
0142-rtmutex-export-lockdep-less-version-of-rt_mutex-s-lo.patch
0143-rtmutex-add-sleeping-lock-implementation.patch
0144-rtmutex-add-mutex-implementation-based-on-rtmutex.patch
0145-rtmutex-add-rwsem-implementation-based-on-rtmutex.patch
0146-rtmutex-add-rwlock-implementation-based-on-rtmutex.patch
0147-rtmutex-rwlock-preserve-state-like-a-sleeping-lock.patch
0148-rtmutex-wire-up-RT-s-locking.patch
0149-rtmutex-add-ww_mutex-addon-for-mutex-rt.patch
0151-locking-rt-mutex-fix-deadlock-in-device-mapper-block.patch
0152-locking-rt-mutex-Flush-block-plug-on-__down_read.patch
0153-locking-rtmutex-re-init-the-wait_lock-in-rt_mutex_in.patch
0155-rtmutex-annotate-sleeping-lock-context.patch
0168-rt-Improve-the-serial-console-PASS_LIMIT.patch
0183-rt-Introduce-cpu_chill.patch
0184-hrtimer-Don-t-lose-state-in-cpu_chill.patch
0185-hrtimer-cpu_chill-save-task-state-in-saved_state.patch
0196-seqlock-Prevent-rt-starvation.patch
0197-sunrpc-Make-svc_xprt_do_enqueue-use-get_cpu_light.patch
0207-printk-Make-rt-aware.patch
0214-kgdb-serial-Short-term-workaround.patch
0216-mm-rt-kmap_atomic-scheduling.patch
0219-arm-Enable-highmem-for-rt.patch
0227-x86-stackprotector-Avoid-random-pool-on-rt.patch
0228-random-Make-it-work-on-rt.patch
0240-sched-Add-support-for-lazy-preemption.patch
0242-x86-Support-for-lazy-preemption.patch
0245-arm-Add-support-for-lazy-preemption.patch
0246-powerpc-Add-support-for-lazy-preemption.patch
0247-arch-arm64-Add-lazy-preempt-support.patch
0249-drivers-block-zram-Replace-bit-spinlocks-with-rtmute.patch
0254-drm-radeon-i915-Use-preempt_disable-enable_rt-where-.patch
0259-cpuset-Convert-callback_lock-to-raw_spinlock_t.patch
0262-signals-Allow-rt-tasks-to-cache-one-sigqueue-struct.patch
0264-Linux-4.19.37-rt19-REBASE.patch

What set of patches do you apply to get a stable RT kernel on the Pi?

[antonellocaroli]

I applied this without any problems:

https://mirrors.edge.kernel.org/pub/linux/kernel/projects/rt/4.19/patch-4.19.37-rt19.patch.gz

[vanfanel]

@antonellocaroli : Great! What other options, appart from GENERAL SETUP->PREEMPTION MODEL can you recommend me to change?
In KERNEL FEATURES->TIMER FREQUENCY, I already have 1000Hz (default in 4.14.y and 4.19.y branches anyway).

[antonellocaroli]

@vanfanel
you just need them...
but take a look at this to find out more:
https://rt.wiki.kernel.org/index.php/Frequently_Asked_Questions

[vanfanel]

@antonellocaroli : Thanks! I read that document time ago.
What I usually do to set realtime priority on a process is:

1. Set the rlimits system-wide accordingly so normal users can set realtime priority.
2. Execute the process with chrt -f 80 <executable name>
   If you have any other observation regarding RT on GNU/Linux, please share it.

[guysoft]

Following, so I can get RealtimePi to point to it when ready

[NikiSchlifke]

+1

[guysoft]

Just realised no one actually mentioned @TiejunChina . So no wonder this has been hanging for 25 days.

[pelwell]

Tiejun is well aware of the situation. Pinging him (which I intentionally avoided) won't help.

[guysoft]

@pelwell Understood, it did not happen here and I could not guess at your private communication. Apologies.

I recommend notifying that next time, since I was starting to speculate on making my own PR branch.

[pelwell]

I was starting to speculate on making my own PR branch.

I wouldn't have a problem with that, if you can follow the same format as Tiejun did. The standard rpi- branches use merge commits to pull in upstream patches so they keep their commit IDs, and cherry-ick/rebase downstream commits to keep the commit history clear - you end up with two parallel streams of commits, "us" and "them".

As I wrote to Tiejun at the time:

"One of our internal rules is that, apart from back-porting patches to older kernels, we never change the upstream commits. This means that our development looks like this:

    L-L-L-L-L-L-L---L-L-L-L-L-L-L     = Upstream Linux
                 \       \       \
                  P-P-P-P-M-P-P---M-P = Downstream rpi- branch
                      ^   ^       ^
                      A   B       C

where L is an upstream Linux commit, P is a downstream Pi commit, and M is a merge commit. A represents the point where we stop rebasing our commits against the new kernel version, which corresponds to the point where it becomes our mainline branch. B and C are minor kernel version updates.

"Perhaps you could structure the -rt branch like this:

    L-L-L-L-L-L-L----L-L-L-L-L-L-L     = Upstream Linux
                \         \       \
                 R-R-R-R-M-M--M----M-M = rpi-rt branch
                        /    /     /
                 P-P-P-P--P-P-----P    = Downstream rpi- branch

"I hope you can understand my diagram. The important point is that the L and P commits in this diagram are identical to those in the previous one, with the same commit hash. The nice thing about this scheme is that by not rebasing your existing commits you only need to work on and think about (and fix) new commits."

If you can follow that pattern and prepare a branch in your repo then I'll be happy to bless it and call it rpi-4.19-rt. If, in doing so, you end up creating any tools to simplify the task then let me know - should it become easy enough then eventually the task may be brought in-house.

[paul-1]

Merging Upstream Linux commits into the RT branch is fun. I've never kept a copy of the rt-kernel git.

When maintaining for myself, when there is a new -rtxx version, I start from a clean rpi branch, and apply the rt patch set, then cherry pick any rpi-rt commits. I know that some don't like constant rebasing, but in my mind keeps the rt branch only a few commits different than the normal rpi branch.

[TheLongAndOnly]

@paul-1 As I understood you managed to get the RT patches to work for 4.19. I tried myself but ran into problems and hope you might have some hints.

As patching 4.19.y HEAD did not work, I pulled the matching commit for 4.19.37. These are the steps I took:

• git clone https://github.com/raspberrypi/linux.git --single-branch --branch rpi-4.19.y
• git checkout 19bb613
  (this is the commit changing to version 4.19.37)
• make bcm2709_defconfig (had to pull it from head, as it is not part of the commit)
  As the debos are not build, I updated to the correct commit of rpi-update
• sudo rpi-update 6cf2788e74b497c109865d052cd6fa68d094cf38

Unfortunately after installation of the new kernel I do not get past the rainbow screen. Changing kernel7.img back to the original one makes the system boot.

[guysoft]

I am not an expert, but looking over the tty serial output might give you more information to what is failing at the rainbow screen.

[paul-1]

RPI is rebasing the 4.19.y tree, you cannot go back like that right now, unless you go and then pick all of the rpi downstream commits. did you try to apply the 4.19.37-rt19 against the current branch? Otherwise it's best to wait until the next rt patch comes out.

[TheLongAndOnly]

Ok, I didn't realize this, but it explains why the configs are missing. Applying 4.19.37-rt19 to the current branch does not work as there have been changes to mainline which makes the build fail.

[paul-1]

4.19.50-rt22 is out, it applies cleanly against the current rpi branch.

[pelwell]

Thanks for the tip.

[guysoft]

@paul-1 Isn't the dwc_otg fix patch also needed?
https://www.osadl.org/Single-View.111+M5c03315dc57.0.html
I have it in source control here:
https://github.com/guysoft/RealtimePi/blob/8b375fd1848276fa22c5c24a490d9696cd4a9e85/src/modules/realtimepi/filesystem/home/pi/patches/0001-usb-dwc_otg-fix-system-lockup-when-interrupts-are-th.patch

[paul-1]

I have not been applying that for a while, and definitely not on the 4.19 branch, I have not noticed a problem. But I've not done a ton of heavy USB testing.

Edit: You may be correct, I think with all the 4.19 changes, I made a mistake in my defconfig.

[pelwell]

Please take a look at https://github.com/pelwell/linux/tree/rpi-4.19.y.rt . It's a simple merge of the current rpi-4.19.y.rt with the RT commits on top. I've also copied across one of Tiejun's commits. Other than building it and booting it on a 3B+ I've not done any real testing, but if people think it's OK then I'm happy to push it as a semi-official, low-priority-supported branch off raspberrypi/linux.

[paul-1]

Should you change the defconfigs in that branch for
CONFIG_PREEMPT_RT_FULL=y

[pelwell]

Good point. I've updated the branch with updated defconfigs, but without the cherry-pick of Tiejun's patch (it was locking up for me with that included).

[pelwell]

I'm also using dtoverlay=dwc2.

[guysoft]

I'll try find the time and make a build or RealtimePi, so we can all be testing the same thing.

[paul-1]

I'm using the standard dwc_otg driver (With fiq & fsm enabled).......I've never had good luck with the dwc2.

I've confirmed that the patch "don't thread shared irqs" from Tiejun is needed if SPI is enabled. I also applied the patch from osadl.org, but there has been changes to the dwc_otg driver that still hangs the system. I believe this commit adds a spot that needs to be locked with the fiq_fsm_spin_lock_irqsave 3ea9af8 after patching that part, kernel has been stable.....but doing more testing now.

[pelwell]

Thanks for the feedback. You could probably save some time and effort if you could submit a Pull Request with the necessary changes, otherwise I'll have another go when I can.

[paul-1]

I will once I do some validation.....I don't want to make mistakes that I made with earlier RT builds.

[guysoft]

This issue can be closed.
Also you can use this image if you like, released :)
https://github.com/guysoft/RealtimePi/releases/tag/0.4.0

[pelwell]

Thanks again, Guy.

[antonellocaroli]

Thanks

[gtrainavicius]

Do you get a warning and callstack printed out when running RT kernel on 3B+, as in the logs attached in this comment: #3172 (comment) ? RPi2 and 4 don't seem to produce it.