Feature Archives | Feedback

Y2K - The Tip of the Iceberg for Embedded Systems

By Jim Ready, Chief Technical Officer

Mentor Graphics Corporation, Microtec Division

At first, I was quite skeptical about the Year 2000 problem in general, let alone the notion that embedded systems would be failing left and right when the computer clocks tick over to 2000. There are, I think, many valid reasons for my skepticism. Certainly, a major reason is technical, in that most of the popular real-time operating systems (RTOSs) supply a very simple clock interface. The clock is simply an accumulation of ticks, and the services of the operating system (OS) are based upon units of ticks having been counted and not time of day or year. Moreover, by now, the major RTOS players have all certified their systems to be Y2K-compliant, so at least those users know what the OS will do at the turn of the century (nothing bad, to be exact). On the other hand, the developers of such embedded systems certainly could have extended the system to include time-sensitive actions that could have a problem with Y2K, so it is surely proper to inspect those systems to make sure they will operate correctly when the time comes.

But after thinking about the Y2K problem, another thought came to mind. The issue certainly focused the public's attention on the fact that they depend upon a large number of embedded computers as they go about their day-to-day activities. And they realized that if many of these embedded systems did have a Y2K problem, the consequences could be severe. There are, for example, worries that airplanes will fall out of the sky because of the failure of on-board computers. Others have expressed concern about the air traffic control systems and the consequences if they were to fail. A more down-to-earth concern is that the nation's electrical power grid will go down as the year 2000 arrives. Other less severe problems that could surface include bank ATMs failing to work, grocery store checkout systems crashing and so on. It doesn't take much imagination to invent a virtually unlimited number of disaster scenarios. Of course, the common thread running through all of these scare stories is that it's the embedded computers that will get us.

Now here is the interesting part for me. The Y2K problem is an easy way for the public to understand that a bug in embedded software can potentially have really bad consequences. Companies are consequently being forced to declare that their systems will not fail and are, in fact, Y2K-compliant. However, anyone with a technical understanding of software realizes that there are countless other ways software can fail due to some significant change in the operating conditions, and moreover, that this kind of sensitivity is by no means limited to Y2K. Below is just a small sample of the many real-world problems known to plague the development of complex software applications:

Take memory leaks. As programs execute, they often acquire and release memory blocks. If the program is correctly written, for every block that is acquired another is at some point returned. There will be no loss of memory over time. However, if a program occasionally "forgets" to return a block, that memory may become unavailable to the system. Such an occurrence is called a memory leak. If a system has a slow memory leak there is a critical point where the system will run out of memory and potentially fail if it has been operating long enough. This is sort of a mini-Y2K problem. But rather than being based upon outside time, it is based upon some possibly unknown number of continuous operational hours of the system.

Now consider the problem of insufficient resource allocation. Let's say an application that uses a message-passing architecture packages data into blocks called messages and sends them from place to place for processing. The system allocates a certain number of buffers to hold the expected number of messages that will need to be processed at any given time. It is possible that there is a hidden limit in the number of messages that can be received under high load conditions where a burst of messages arrive all at once. If insufficient buffers have been allocated to accommodate this peak demand, the system might drop some incoming messages, triggering unknown consequences.

Then there is the issue of stack overflow. Stacks are used in the execution of most popular high-level languages such as C. As a program executes and makes calls to other functions, it uses a stack to hold temporary data. It is possible that the stack will occasionally and silently grow beyond the memory space allocated to it and overwrite the existing data already in the same space. As the program continues to execute, the stack retreats (again silently) from this incursion, and as far as anyone knows, with no apparent harm having been done and no traces left, save for the now corrupted data. Finally, some other part of the program, perhaps minutes or hours later, uses the data corrupted by the stack overflow, and produces incorrect results with unknown but possibly severe consequences.

It is well known that memory leaks, insufficient resource allocation, and stack overflows are commonplace problems in today's complex software applications. In fact, there are whole software tool companies whose entire business is devoted to finding problems such as these in a product before (hopefully) it is shipped. Yet no software companies are being asked to certify that their applications are "memory leak free," that they correctly allocate resources, or that they are free of stack overflow conditions. But these very same companies are required to certify that their applications are Y2K-compliant. In my experience, I strongly believe it is far more likely that an embedded system will fail due to one of the reasons outlined above, than for the very specific bug that Y2K represents.

Go figure.

JAMES F. READY
CTO AND TECHNICAL STEERING COMMITTEE CHAIRMAN
THE MICROTEC DIVISION OF MENTOR GRAPHICS CORPORATION

James F. Ready joined Microtec in December 1993 as vice president, chief technical officer. Ready, a founder of Ready Systems, served as director and vice president of Ready Systems from 1981 to the merger with Microtec Research in 1993. Before founding Ready Systems, he held engineering and marketing positions at ROLM Corporation. Ready holds a bachelor's degree from the University of Illinois and a master's degree from the University of California, Berkeley.

Ready is best known as the inventor of the VRTX RTOS and is widely credited with starting the off-the-shelf RTOS industry. He has published numerous technical articles and papers, and has lectured worldwide on the principles of real-time software design. Ready has also served as a featured speaker at many conferences and industry trade shows.