Real Time Scheduling Of Threads In Linux

It can be easy to get tripped up when setting up and managing real time threads in linux. In fact, I recently discovered that the thread management calls I was using within the Poco C++ library effectively did nothing. It provides an interface to set a general thread priority, but not its policy and as a result has makes no changes to the thread.

Real Time Policies And The Poco ThreadPool

Linux supports different thread scheduling types, called policies. For linux kernels with real time functionality, there should be support for both FIFO and round robbin real time scheduling. SCHED_FIFO requires the thread to yield to the processor, guaranteeing that it completes its operations before another thread takes over. SCHED_RR uses a specified time slice for the thread.

I was using the Poco::ThreadPool library to set priority at thread start. This can be misleading and ineffective because it uses the SCHED_OTHER policy only, which is not a real time policy. Looking at the source code, it pulls a range of valid priority numbers from the kernel. In the case of arch linux, it turns out that only priority 0 is valid, meaning that setting a different priority has no effect.

Real Time Thread Priority Numbers

As mentioned earlier, when setting a priority, the maximum and minimum valid priority for a given policy must be checked. This can be done in Poco using Poco::Thread::getMaxOSPriority(<Policy>) / Poco::Thread::getMinOSPriority(<Policy>)  or using the direct pthread call sched_get_priority_max(<Policy>) / sched_get_priority_min(SCHED_FIFO)
pthread_setschedparam can be used to set the priority directly. Since I'm using the Poco library, I simply set it up like this:


For my application the SCHED_FIFO policy was appropriate.

When setting real time thread priorities, be careful not to get confused between these numbers and process niceness numbers, which increase priority with a lower number. The real time thread priorities increase with a higher number when setting them in software. To further complicate matters, the priority numbers on the thread ids are shown with negative numbers when using a tool like top.

When should you use real time threads? The simple answer is when your timing requirements are more critical. In my case, I'm managing an i2c communications thread, spi communications thread, message queue manager and a multi-threaded watchdog manager. Using a real time thread on all of these is critical to improved system performance. The most noticeable performance gain came from the SPI communications thread, which is making a lot of user-space SPI calls. By making the thread real time FIFO with a high priority I saw worst case thread times drop from 250 mS to less than 30 mS. There are still kernel components which will pre-empt the user-space SPI calls, but the change in performance is significant and more than sufficient for my application.

Process Niceness

Process niceness is a user space control to determine which process is "nicer" to the other processes. A process with a higher nice value will give up it's resources for other user space processes with a lower nice value. A process would typically start with a default value of 0. As you can see, my process is not very nice! When to use this? Suppose you have a webserver running, but it's services are not important relative to a command handler running in your process. Neither are making use of realtime threads, but both are running in user space. This is where changing the niceness can be useful in order to favor the command handler.

Managing threads in linux can be pretty simple once you clear the hurdle of understanding how the policy and priority numbers work.