More time than you want

I was discussing with one of the other developers here some source code that comes from a very popular open source database that has been ported to QNX and that is going to be shipping with the next release of the OS. It is funny the way that developers get into this mindset that the code they are writing will be the only code in the entire universe running.

The snippet of code we were looking at looked something like the following (code changes made to protect those who should know better):

/*
** Sleep for a little while.  Return the amount of time slept.
** The argument is the number of milliseconds we want to sleep.
*/
int OSSleep(int ms){
  usleep(ms*1000);
  return ms;
}

Can you see what the flaw in the logic is here? Perhaps if we put the code into a context the problem will become clearer. The OSSleep function is being called in order to walk our way through a series of progressively longer timeouts .. based on the return value of how long this function has slept.

If the application containing this source code was the only thing running on the system, then there would be no problem. However, in most modern operating systems, QNX Neutrino counted among them, there are multiple tasks that are all running concurrently and being managed by the OS.  This means that while surely the code above will delay for N milliseconds, it could in fact be much longer than that if some other thread pre-empts this thread before or after it makes it into (or out of) the system call.

So .. what could we do to make it better?  Well we could snapshot the time on input and snapshotting it on exit and returning the difference.   This makes things incrementally better by narrowing down the window of error, but the window is still there. 

The reality is that this kind of API, and this application’s use of it, is fundamentally flawed.  What the application is trying to emulate is allowing multiple threads of execution the ability to acquire a resource.  If the resource can’t be acquired, then the thread backs off or may choose to cancel the operation all together. 

This is a perfect scenario for a condition variable … something available under Neutrino, but not necessarily under the other Unix flavours that this source was ported to.  By switching to a condition variable, we can be much more precise with respect to when we signal client applications to “try again” … which can result in a surprisingly large reduction in overall CPU consumption and system churn (due to all the extra context switches and cycles through the scheduling READY queue).

So in these days of thinking about multiple threads of execution running on separate CPU cores, we should also remind ourselves that using “time” as a basis for control needs to be carefully planned on systems where there is more than a single thread of execution.

Thomas

Advertisements

2 comments so far

  1. Mitchell Schoenbrun on

    Thomas,

    This looks like a very interesting point, but you lost me at the start with your explaination for what the application is trying to do. What does

    “The OSSleep function is being called in order to walk our way through a series of progressively longer timeouts .. based on the return value of how long this function has slept.”

    mean?

  2. Thomas on

    The calling code looked something like:

    int timeouts[] = {5, 10, 15, 30, 60 }

    for(i = 0;
    i < sizeof(timeouts) / sizeof(*timeouts);
    i++) {

    ret = OSSleep(timeouts[i])
    … do some stuff with ret …
    }

    The timeouts here were meant to max out the delay
    as the sum of all of the fields, but rather than
    jumping to the next appropriate time slot, the code assumes that the time we slept was only what
    was asked for and the return value is used later on as if it as the only time in the delay (ignoring the fact that we could be pre-empted for a long time before coming back to clients).


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: