This chapter provides an overview of how Linux and REACT support real-time programs:
The Linux kernel has a number of features that are valuable when you are designing a real-time program. These are described in the following sections:
The default Linux scheduling algorithm is designed to ensure fairness among time-shared users. The priorities of time-shared threads are largely determined by the following:
While a time-share scheduler is effective at scheduling most standard applications, it is not suitable for real time. For deterministic scheduling, Linux provides the following POSIX real-time policies:
These policies share a real-time priority band consisting of 99 priorities. For more information about scheduling, see “Real-Time Priority Band” in Chapter 4 and the sched_setscheduler(2) man page.
Linux allows a task to lock all or part of its virtual memory into physical memory so that it cannot be paged out and so that a page fault cannot occur while it is running.
Memory locking prevents unpredictable delays caused by paging, but the locked memory is not available for the address spaces of other tasks. The system must have enough physical memory to hold the locked address space and space for a minimum of other activities.
Examples of system calls used to lock memory are mlock(2) and mlockall(2).
Normally, Linux tries to keep all CPUs busy, dispatching the next ready process to the next available CPU. Because the number of ready processes changes continuously, dispatching is a random process. A normal process cannot predict how often or when it will next be able to run. For normal programs, this does not matter as long as each process continues to run at a satisfactory average rate. However, real-time processes cannot tolerate this unpredictability. To reduce it, you can dedicate one or more CPUs to real-time work by using the following steps:
Restrict one or more CPUs from normal scheduling so that they can run only the processes that are specifically assigned to them and isolate them from the effects of scheduler load-balancing.
Assign one or more processes to run on the restricted CPUs.
A process on a dedicated CPU runs when it needs to run, delayed only by interrupt service and by kernel scheduling cycles.
In normal operations, a CPU receives frequent interrupts:
These interrupts can make the execution time of a process unpredictable. I/O interrupt control is done by /proc filesystem manipulation. For more information on controlling I/O interrupts, see “Redirect Interrupts” in Chapter 4.
You can minimize console interrupt effects with proper real-time thread placement. You should not run time-critical threads on the CPU that is servicing the system console. You can see where console interrupts are being serviced by examining the /proc/interrupts file. For example:
[root@linux root]# grep "\<serial\>" /proc/interrupts
CPU0 CPU1 CPU2 CPU3 ..
4: 28387 0 0 .. IR-IO-APIC-edge serial |
The above shows that 28,387 console driver interrupts have been serviced by CPU 0. In this case, CPUs 1 and 2 would be much better choices for running time-critical threads because they are not servicing console interrupts.
Timer processing is always performed on the CPU from which the timer was started, such as by executing a POSIX timer_settime() call. You can avoid the effects of timer processing by not allowing execution of any threads other than time-critical threads on CPUs that have been designated as such. If your time-critical threads start any timers, the timer processing will result in additional latency when the timeout occurs.
Many real-time programs must sustain a fixed frame rate. In such programs, the central design problem is that the program must complete certain activities during every frame interval.
The frame scheduler is a process execution manager that schedules activities on one or more CPUs in a predefined, cyclic order. The scheduling interval is determined by a repetitive time base, usually a hardware interrupt.
The frame scheduler makes it easy to organize a real-time program as a set of independent, cooperating threads. You concentrate on designing the activities and implementing them as threads in a clean, structured way. It is relatively easy to change the number of activities, their sequence, or the number of CPUs, even late in the project. For more information, see Chapter 5, “Using the Frame Scheduler”.
This section discusses the following:
To determine the clock source for your system, run the following:
# cat /sys/devices/system/clocksource/clocksource0/current_clocksource |
Output:
The following sections apply only to those systems that use an RTC.
 | Note: This section does not apply to systems with synchronized TSCs. See “Determining the Clock Source”. |
SGI UV 3000, SGI UV 2000, SGI UV 1000, and SGI UV 100 systems provide a systemwide clock called a real-time clock (RTC) that is accessible locally on every node. The RTC provides a raw time source that is incremented in 5-ns intervals. The CPU's local APIC timer is used for timer interrupts (the timer_create() function).
The RTC is 56 bits wide, which ensures that it will not wrap around zero unless the system has been running for more than 11.42 years. RTC values are mapped into the local memory of each node. Multiple nodes accessing the RTC value will not reduce the performance of the clock functions.
On SGI UV 100 systems, on SGI UV 1000 systems, and on SGI UV 2000 systems without synchronized TSCs, the RTC is the basis for system time. This may be obtained via the clock_gettime() function call that is implemented in conformance with the POSIX standard. The clock_gettime() function call takes an argument that describes which clock is wanted.
The following clock values are typically used:
CLOCK_REALTIME is the actual current time that you would obtain from any ordinary clock. However, CLOCK_REALTIME is set during startup and may be corrected during the operation of the system. This implies that time differences observed by an application using CLOCK_REALTIME may be affected by the initial setting or the later correction of time (via clock_settime) and therefore may not accurately reflect time that has passed for the system.
CLOCK_MONOTONIC starts at zero during bootup and is continually increasing. CLOCK_MONOTONIC will not be affected by time corrections and the initial time setup during boot. If you require a continually increasing time source that always reflects the real time that has passed for the system, use CLOCK_MONOTONIC .
The clock_gettime() function is a fastcall version that was optimized in assembler and bypasses the context switch typically necessary for a full system call. SGI recommends that you use clock_gettime() for all time needs.
CLOCK_REALTIME and CLOCK_MONOTONIC report the correct resolution.
| Note: This section does not apply to systems with synchronized TSCs. See “Determining the Clock Source”. |
In some situations, the overhead of the clock_gettime() fastcall may be too high. In that case, direct memory-mapped access to the SGI UV 3000, SGI UV 2000, SGI UV 1000, or SGI UV 100 RTC counter is useful. (See the comments in mmtimer.h.)
Like CLOCK_MONOTONIC, the RTC counter is monotonically increasing from bootup and is not affected by setting the time.
This section discusses the following:
The performance of both sockets and MPI depends on the speed of the underlying network. The network that connects nodes (systems) in an array product has a very high bandwidth.
One standard, portable way to connect processes in different computers is to use the BSD-compatible socket I/O interface. You can use sockets to communicate within the same machine, between machines on a local area network, or between machines on different continents.