Introduction
Real-time systems are at the heart of every robotics system and without it, it’s not possible to develop a reliable and safe robot application. Still there are misconceptions around the concept of real-time and even some developers don’t know exactly what real-time means, let alone what it takes to develop a real-time application. This article will explain the basics of real-time and where it is used in robotics as well as prerequisites and best practices for developing real-time applications.
We will cover the following topics:
- What is Real-time?
- Why is Real-Time Computing important in Robotics?
- How do Real-Time Operating Systems work
- Best Practices for developing Real-time applications
- Real-time in embedded programming without an operating system
What is Real-time?
Let’s start by getting rid of a common misconception right away: Real-time does not mean as fast as possible.
When you read about something that is advertised as real-time it usually means instant, very fast information delivery to the user. Real-time weather apps show you an updated weather radar every couple of minutes. Real-time stock market data gives you all the twitches of a share price every few hundred milliseconds.
However, this is not what we mean when we are talking about real-time computation. It does not mean the computation is very fast but instead, that the computation runs deterministic. If a computation is deterministic you can be sure to always get the same output for the same input. This means the computation will always take the same amount of time.
Let’s use an analogy to explain the concept of real-time. Say you are a columnist and you need to deliver a new article every day until 10 p.m. such that it can be printed in the next day’s newspaper edition. 10 p.m. presents a deadline for you or lets call it a real-time constraint. If you are too slow and hand in the article 5 minutes late, the printing press will have started without you. If you make it before 10 p.m. the article will be printed but nobody cares if you sent it 5 minutes or two hours before that. Real-time is about ensuring that you always deliver ahead of the deadline.
There are different types of real-time systems that mainly distinguish in how strict they are about the real-time constraints:
- Hard Real-time: If one deadline is missed, it is considered a severe failure of the system (i.e. your boss will fire you if you deliver after 10 p.m. one day). This type of real-time is required in high safety systems like airplanes.
- Soft Real-time: The system will tolerate if deadlines are missed at times, though it might take action to reduce those failures (you are not fired but your boss lowers the minimum amount of words your article must have to make sure you don’t miss future deadlines). In video streaming for example, the system might reduce the image quality to ensure the video plays smoothly.
- Firm Real-time: Similar to soft real-time but data that arrives after the deadline is treated as invalid and not used (you are not fired but the article is thrown away if it arrives too late).
Another term that is often used besides determinism is latency. Latency, in fact, is about how fast a system is, i.e. how much time passes between two events (for example between when the data arrives and when the computation finishes). A real-time system is making sure the latency stays bellow a defined limit.
This is the general concept of real-time computation. You might think: So far so good but what does that mean in practice? We will now have a look at some specific examples to explain why real-time is so important in Robotics.
Why is Real-Time Computing important in Robotics?
Imagine a robot arm doing assembly work in a automobile production line, lifting heavy parts and moving them from A to B. When a worker enters the workspace, the robot should stop to make sure it doesn’t hurt the human. Therefore it is important that the robot reacts to a signal (e.g. coming from a light barrier) within a certain amount of time.
Such safety systems often have critical real-time constraints. Often times they require certification from a third party instance to whom you need to prove that the real-time requirements are met. This is one more reason why it is important to know about real-time.
Robot controllers are another classical real-time application. Say we receive a position sensor value and compute the velocity by taking it’s derivative. Based on this derivative we do the computation for running a velocity controller.
Let’s look at two different imaginary scenarios. The blue bars in the upper subplot of the first figure shows the arrived sensor data. The vertical dashed lines show the point in time when the computation of the derivative is starting. In this first scenario we see that the new sensor data is always available before the computation starts.
The blue line in the lower subplot shows the computed velocity while the orange dotted line shows the imaginary output of a velocity controller. For this simple example we assume a P-Controller with the equation \[p_{out} = K * v_{set} – v_{actual}\] with \(K=2\) and the set point at \(10 rad/s\) (if you want to learn more about the different types of controllers you can check out this article). For this first scenario the controller runs smooth for the computed velocity values.
The second scenario in the next Figure shows what happens if the real-time constraints are violated. In this case the third position value arrives after the velocity computation started. If the old value is used for computing the derivative, the velocity drops to zero because it seems the position value hasn’t changed. Afterwards it jumps to 22 assuming the value that arrived too late is dropped because a newer value is already available. The resulting controller output follows the jumps in the velocity computation.
This simple example describes how non-deterministic behavior can cause huge oscillations in the controllers. These oscillations cause a bad controller performance and might even damage the hardware.
I hope these examples illustrate why real-time behavior is important and what are the consequences when real-time constraints are violated. Now let’s see what it actually takes to develop software for real-time behavior.
Prerequisite for Real-Time applications – A Real-Time operating system
There is two components to actually run an application in real-time. An operating system capable of real-time and a carefully crafted software implementation.
Let’s talk about the operating system first. Two important areas that distinguish a Real-Time Operating Systems (RTOS) from a normal operating system are scheduling and interupt/ task switching latency.
Let’s examine the first one: What is scheduling? A scheduler is an important part of today’s operating systems that decides which processes are run at which time. It allows to have multitasking behavior even if the program is only running on one CPU core and only one computation can be executed at a time. E.g. on your Windows or Linux operating system you can start multiple processes at once, do internet browsing and copy files seemingly simultaneously. The job of the scheduler is to manage all these processes that demand computation time on your PC.
There are different types of schedulers that optimize for different criteria, some are good for high throughput while others ensure a quick response time. The simplest scheduling algorithm is First come, first served (FIFO) where the processes are queued in the order they arrive and just run one after another. One scheduler that is often used in real time operating systems is priority scheduling. It prioritizes processes based on their deadlines such that the process with the earliest deadline runs first.
Now let’s talk about interrupt or task switching latency as the second important property of an RTOS. Both terms can be unified as context switching which means the CPU is doing one thing and then switching to do another thing. Schedulers in RTOSs need to be able to interrupt a running process to switch to a more important one and later come back to the original process (processes that can be interrupted and resumed later are called preemptive). Those context switches are not for free. Among other things they require to save the state of the original process to be able to come back to it later to continue its execution. The time this switch takes is called task switching latency. For real-time we want these switches to be as fast as possible.
There are different RTOSs on the market that each have their strengths and weaknesses. One popular choice for robotics applications it the RT_PREEMPT patch for Linux. This patch makes the Linux kernel real-time capable which by default it is not. To do that it uses different mechanisms to make more of the kernel code preemptive (i.e. interruptible). This choice is popular with robotics developers because they often prefer developing on Linux and that way they don’t need to switch to a different operating system for developing real-time applications.
Coming back to our real-time columnist we could compare the operating system to his writing environment. To get the job done on time it is important that he is not interrupted by unimportant tasks that he needs to do and that he can switch between the important ones like writing and editing as fast as possible.
The implementation part that we will talk about in the next section is analogous to the writing process itself. Does he have all the information and resources needed or does he first need to go off and get them? Is his phone off or does he need to react to incoming messages while writing?
Let’s talk about how to develop real-time applications which are running on a real-time operating system.
Best Practices for developing real-time applications
A real-time operating system makes sure that the underlying mechanisms like scheduling run deterministic. However, this is not enough. Also the code that is executed has to be explicitly developed for real-time. This requires a good knowledge of the system that the application is running on.
This article is not explicitly about teaching you how to write real-time applications but I want to give you a few hints and explain some best practices. This way you will get an idea what you need to consider and you will know what specifically you are talking about when discussing real-time application implementations.
Choose the right Programming Language
It is important to choose the right programming language for developing real-time applications. In robotics this usually means C or C++. With these languages you have close control over what is going on during the execution of a program. Other programming languages like python hide what is going on under the hood. This makes it very difficult to develop real-time applications with them.
Memory management
When it comes to memory management you should know where data is stored and how the program accesses it. Local variables are stored in RAM memory on the stack which can be accessed quickly. Data that needs to be fetched from the disk takes longer to access, the program has to wait for it to finish and this process is non-deterministic. There are many aspects to memory management for real-time applications. If you are interested you will find further readings in the References section.
Interaction with Hardware
Try to avoid interaction with external devices like hard drives, computer screens etc.. Access to such devices introduce unknown blocking time in your program. Also avoid input devices like mouse and keyboard (especially via USB) on your real-time machine as these input devices can lead to high priority interruptions of your program at unknown times.
Generally, a good practice for writing a real-time application is to divide it into three stages. In the first one you do non-real-time stuff like allocating memory and starting threads. The second stage is the real-time loop. When it is finished you move on to the third stage where you de-allocate resources, stop the threads etc. which, again, are processes that are not running in real-time.
If you want to learn more about developing real-time applications check out the section “Best Practices in Real-time Computing” in ROS2 design – Introduction to Real-time Systems.
Real-time in embedded programming without an operating system
Before finishing this article I want to talk about the special case of bare metal programming which means writing embedded software without an operating system. This is usually done on microcontrollers which also includes development boards like Arduino. In this case the programmer has complete control over (nearly) everything that is happening on the CPU. In the case of bare metal programming there is no scheduler and the priorities of interrupts need to be determined by the developer.
This complete control also means that the developer has to know exactly what is happening on the microcontroller and needs to think about the importance of the task, their execution time and their order of execution by himself.
The example of the velocity controller we discussed before is a typical application that is running on a microcontroller. When applications get more complex, it can be difficult to keep an overview over everything that is going on in the processing unit.
Conclusion
I hope that you got a clear idea of what real-time computation is and why it is so important for robotics applications. No matter if you will need to develop real-time applications at some point or not, being familiar with the basic concepts is always helpful. Adding Real-Time capabilities was also a big reason for the development of ROS2 in order to make it suitable for commercial applications. For reading more about how real-time is enforced in ROS2 you can start here. If you found this article helpful make sure to also check out my other Robotics tutorials and I am always happy to receive feedback in the comments 🙂
