# KIO's Thread Dispatcher - Documentation

## The Dispatcher

### Overview

#### What are threads good for?

Imagine an application, which reacts to your mouse clicks and key strokes while it saves a large file. Imagine 'power windows' with live dragging, scrolling and resizing beeing totally easy! And, in addition to that, imagine an application, which does not stop while you resize a window, drag a slider or pull down a menu. That's what threads are good for. And a priority-driven thread dispatcher is best for!

Life could be so easy.

This dispatcher is cooperative, realtime capable, purely priority driven and preempting-free.

The dispatcher knows about threads, timers, interrupts and semaphors.

#### Cooperative

The implementation of this thread dispatcher is cooperative, that is: the currently running thread must actively test whether another thread is willing to run. This is done by a simple and very fast call, which can -and should!- be spread over the entire application:

> Disp(); // Dispatcher test

All dispatcher methods (except those noted) also do this test and can therefore temporarily switch out a thread.

This cooperative implementation has advantages and drawbacks:

- + not reentrant-capable code sequences need frequently not to be guarded by semaphors
- - a malfunctioning thread which does not poll the dispatcher is hard to stop

#### Realtime

Was ist 'echtzeitfähig'? Jedes System reagiert immer mit einer gewissen Verzögerung auf Ereignisse. Die Frage ist nur, wie lang diese Verzögerung im Mittel ist und welche Antwortzeiten man garantieren kann.

Da der Dispatcher kooperativ ist, kommt es auf den Programmierer an, wie oft er einen Dispatchertest durchschnittlich einfügt und wie weit zwei Tests in seinem Code maximal auseinander liegen. Und es kommt auf die Umgebung an, in diesem Fall andere Applikationen und das MacOS.

Die Threadwechselzeit auf meinem PowerMac G4, 400MHz, altes Board, liegt unter einer Mikrosekunde. Das ist also der theoretische Grenzwert für die Reaktionszeiten des Systems, solange man keine MacOS-Funktionen aufruft. Eine mittlere Reaktionszeit von 1 Millisekunde ist aber realistisch.

Werden sehr kurze Reaktionszeiten verlangt, kann man alle Schleifenkonstrukte von C über Makros umdefinieren, so dass jeweils auch ein Dispatchertest durchgeführt wird. Letztendlich wird diese Methode nur dadurch ad adsurdum geführt, dass man spätestens bei Aufruf einer MacOS-Betriebsfunktion wieder mit langen, häufig sogar extrem langen Wartezeiten rechnen muss... werden keine MacOS-Funktionen aufgerufen, kann man so Threadwechselzeiten von unter einer Millisekunde garantieren.

#### Priority Driven

CPU-Zeit wird vom Dispatcher ausschließlich nach Priorität vergeben. Viele Leute haben damit Probleme, weil sie ihre Compiler grundsätzlich mit Systempriorität starten und dann 2 minuten lang auf ihren Computer warten müssen...

Reine Prioritätssteuerung ist die sinnvollste Methode für die Vergabe von CPU-Zeit auf Single-User-Systemen und Servern, auf Multi-User-Systemen mit Einschränkungen. Was wichtiger ist als der Rest wird als erstes getan. Benötigt wird nur eine einheitliche Konvention, um logische Dringlichkeit auf Thread-Prioritäten für den Dispatcher abzubilden.

#### Preempting-free

Weil der Dispatcher rein prioritätsgesteuert ist, und keine Zeitscheiben vergibt, müssen Zeitscheiben auch nicht vorzeitig beendet (pre-emptet) werden können. Der Dispatcher benötigt diese Krücke nicht.

#### Threads

Durch den Dispatcher kann man Aufgaben auf mehrere, quasi parallel laufende Fäden, die Threads verteilen. Eine regelmäßig wiederkehrende Aufgabe, z.B. die Animation eines Bildes oder die ununterbrochene Ausgabe von Ton, muss nicht durch regelmäßige Unterbrechung einer Hauptaufgabe erledigt werden, sondern man behandelt sie in eigenen Threads. Dadurch wird das Hauptprogramm nicht mehr unterbrochen und auch die Unterbrecheraufgaben müssen nicht jedesmal ihre Variablen laden und wieder sichern: sie bleiben in ihrem Kontext.

#### Timer

Der Dispatcher hat eine Systemuhr mit 1-Millisekunden-Auflösung. Threads können ihre Ausführung jederzeit für eine genau definierte Zeitdauer unterbrechen, sich für spätere Zeitpunkte wieder einplanen oder Interrupteinplanungen und Semaphorenanforderungen mit einem Timeout versehen.

#### Interrupts

Es gibt Interrupts, die von beliebigen Quellen, z.B. von Hardware-Interrupts jederzeit asynchron oder von Threads aus synchron getriggert werden können. Auf der anderen Seite können sich Threads für Interrupts einplanen, mit oder ohne Angabe eines Timeouts. Interrupts können zählen, so dass es möglich ist, einen Interrupt erst nach mehreren Ereignissen auszulösen, z.B. wenn die gewünschte Anzahl Zeichen von einer seriellen Schnittstelle eingelesen wurden. Oder auch nicht, dafür gibt es dann das Timeout.

#### Semaphors

Es gibt Ampeln sogenannte Semaphoren, mit denen der Zugang zu begrenzten Ressourcen kanalisiert werden kann. Der erste Thread, der eine Semaphore anfordert, bekommt sie und blockiert sie, später ankommende Threads erhalten die Semaphore erst, wenn der erste Thread sie wieder frei gibt. So kann man z.B. eine Funktion, die nicht wiedereintrittsfest ist, mit einer Semaphore versehen, so dass sie nicht von mehreren Threads gleichzeitig ausgeführt werden kann.

Außerdem können auch Semaphoren zählen: Man kann sie mehrfach freigeben oder anfordern, z.B. um so ein Modul das mehrere, aber nur begrenzt viele Threads gleichzeitig bearbeiten kann, vor Überlastung zu schützen. Und die Anforderung einer Semaphore kann ebenfalls mit einem Timeout versehen werden.

Wird eine Semaphore durch einen niedrig priorisierten Thread blockiert wenn sie von einem höher priorisierten Thread angefordert wird, so wird der niedrig priorisierte, blockierende Thread kurzfristig mit höherer Priorität ausgeführt, bis er die Semaphore wieder freigegibt.

### Integrating the dispatcher into your project

In order to use this dispatcher, you must include disp.cp in your project and disp.h in most of your files, and your application must perform the following steps:

            main()
            {
                InitMac();
                StartDispatcher();
                InitApplication();
                EventLoop();
            };

void InitMac(), void InitApplication() and void EventLoop() must be implemented by you, they are not part of the dispatcher.

void InitMac() must initialize all MacOS toolboxes, appleEvent handlers and so on.

InitApplication() must initialize your application.

EventLoop() must poll events until your application quits. This procedure is not part of the dispatcher, but there is a code snippet illustrating a required work-around for the dispatcher to run on MacOS. It is best practice to call the event polling loop from the #root thread as shown above for security reasons, though this is not really required.

#### void StartDispatcher()

This function initializes the dispatcher. When this call returns, you are running as a thread! You are not 'any' thread, but you are the #root thread: This thread's stack is the application's original stack. All other threads get their stack allocated with new(). All threads share the application's heap. The #root thread can do everything a non-threaded application can do. For all other threads there may be some limitations, but currently i do not know of any.

A system timer has been started and provides a time sheduler with millisecond [msec] resolution, which also counts the system load and execution time for each thread. Use SystemTime() to get the current system time which is measured in msec since application started. Use TimeOfDay() to get the time in msec since midnight.

#### bool DispOk()

This function returns true if the dispatcher is up and running. This function always returns true if called after StartDispatcher() returns, but it returns false if called before calling and while StartDispatcher() is executed. This is useful if you implement some of the error checking and logging macros which may need to know whether threads and dispatcher functions are available.

#### void Disp()

The central requirement to keep the dispatcher happy is to call Disp() frequently throughout your whole code! Whenether you call Disp() or any other dispatcher method, it checks for threads which became active in the meantime. In case of Disp() these are basically time-sheduled threads but maybe you have some asynchronous interrupt sources too. Call Disp() once per procedure and once per loop. But always make a rough estimation on how long the loop takes, it's not necessary to put it in every little loop.

On the other hand: as long as you do not call Disp() et.al., you can be shure that you don't get interrupted by other threads. While you manipulate data, they are in a state where data is not consistent. If these data are used by other threads too, most likely if they are global, then it will crash your application if suddenly another thread gets processing time and acesses the same data assuming it is in a consistent state. Therefore you normally surround such manipulations with a Request() and Release() of a dedicated semaphor, but in this cooperative dispatcher you don't need it, as long as you do not call Disp() or any other dispatcher method.

### System time

The system time is measured in msec since the application was launched. The system time wraps across sign border after 3.5 weeks approximately and another 3.5 weeks later it is zero again. This is no problem to the dispatcher, but this defines the hard limit for time shedules and timeout durations. And you must consider it whenever you compare times:

            if (SystemTime() >= myAlarmTime) { do this or that }     // WRONG

This is a common error! (-:< Hello to Microsoft! >:-) This code works 3.5 weeks and then it fails. Ok, on a Macintosh you'll rarely have an application running that long, but the correct way to do it is:

            if (SystemTime() - myAlarmTime >= 0) { do this or that } // Right

Always compare against a fixed value, e.g. against 0.

#### mssec, msmin, mshour and msday

These constants define the duration of a second, minute, hour and day respectively in the system-wide used time base milleseconds [msec]. Furthermore they are used for asigning priorities to threads.

            mssec    = 1000        = one second   = 1000 ms
            msmin    = mssec*60    = one minute   = 60 seconds
            mshour   = msmin*60    = one hour     = 60 minutes
            msday    = mshour*24   = one day      = 24 hours

#### long SystemTime()

This call returns the system time in msec since the application was launched. The system time wraps across sign border after 3.5 weeks approximately. Always compare times like this:

            if (SystemTime() - myAlarmTime >= 0) { do this or that }

#### long Midnight()

Returns the system time of last midnight expressed as duration in msec since the application was launched.

To resume a thread at 8:00 in the morning you can shedule it for Midnight()+8*mshour. If it's already 8:00 or later, your thread resumes immediately, else it is sheduled for 8 o'clock.

note: effect of time zone and daylight saving not yet defined.

#### long TimeOfDay()

Returns the current time expressed as duration in msec since midnight 0:00 today.

note: effect of time zone and daylight saving not yet defined.

### Polling Mac OS events

Polling Events from the Operating System is not part of the dispatcher. Howether, the Mac OS has a 'flaw'. You cannot release time to the system and other applications without polling events. These two requirements, asking for events and releasing time cannot be split.

But a priority oriented system must handle them separately, because releasing time is the lowest priority task in the system, polling events is of very high priority. If you released time on a mid-priority level, a thread of lower priority wouldn't be executed at all. Therefore releasing time is of lowest priority. If you asked for new events at a mid-priority level, you wouldn't get any while a high-priority thread is running. Therefore asking for events is of very high priority. But in the MacOS you can't combine them.

Why? The dedicated calls for polling events are WaitNextEvent(), GetNextEvent(), GetOSEvent() and EventAvail(). They all have a parameter -eventMask- to indicate which events are to be polled. And WaitNextEvent() has a parameter -sleep- which indicates how long to wait max. for the next event, if none is available immediately, releasing time while waiting. So WaitNextEvent() seems to be ideal for doing what we need: Call it with eventMask=0 and sleep>0 to release time without polling events and call it with eventMask=everyEvent and sleep=0 to ask for events.

Now to the 'flaw': There is a bug in all these calls and you lose events, namely the osEvt events (maybe others too) if you call WaitNextEvent() or any other of the above functions with an eventMask which excludes them. Your application does no longer know for shure, whether it is the foreground or in the back. Another possibility to yield time could be Delay(). But, believe it or not, Delay() does not release time. It hogs the processor until it's done!

Now to the work-around. This utilizes an interrupt, called idleIrpt and a flag, called idleFlag.

#### Irpt idleIrpt

Whenever the dispather finds no thread able and willing to run, it triggers this interrupt and looks if there is now a thread to be serviced. If there is no thread sheduled for this interrupt, it will simply hog the processor until enough time has passed and a time-sheduled thread becomes active.

The suggested method for polling events is in a loop in the main application's thread, the #root thread. This loop waits a while, asks for events and handles them and, again, waits for a while. Instead for simply waiting, you shedule this thread for the idleIrpt interrupt with a timeout. If your request times out, it's just like waiting, but if there is an interrupt, this is idle time.

If there is no idleIrpt pending, call WaitNextEvent() with sleep=0 and eventMask=everyEvent, as usual.

If the idleIrpt has been triggered, call WaitNextEvent() with sleep>0, but with eventMask=everyEvent too. So you don't lose an event and can release time if you have nothing to do and not release time while you are still busy. Don't forget to clear the interrupt afterwards.

#### int idleFlag

The time spent in WaitNextEvent() calls are accounted for the priority of the executing thread when the system timer increments a sysLoad[] cell. So spending idle time would be accounted for the priority of the thread which executes the event polling loop. To avoid this, increment idleFlag before calling WaitNextEvent() and decrement it afterwards. While idleFlag is non-zero, the system timer does not account for the current thread's priority but increments idleTime instead.

#### Event polling loop framework

See the following framework for your event polling loop. This example releases the cpu for up to 3/60 seconds (MacOS uses 1/60 sec ticks for delay parameters) if idle time is detected. And it utilizes a global event lock -gEventLock- which supresses event polling. This is required because you can run your threads while in a call-back hook of the MacOS, e.g. while menus are down. But there you can not call WaitNextEvent() without the MacOS getting desoriented!

    void PollEvents ( )
    {
        EventRecord event;
        bool f;

        while ( !done )
        {
            while (!done && gEventLock)
            {
                Wait(5);
                idleIrpt.Clear();
            }
            if (done) break;

            gEventLock++;
            if (idleIrpt.IsTriggered())
            {
                idleIrpt.Clear();
                idleFlag++;
                f = WaitNextEvent(everyEvent,&event,3,gMouseRgn);
                idleFlag--;
            }
            else
            {
                f = WaitNextEvent(everyEvent,&event,0,gMouseRgn);
            }
            gEventLock--;

            if (f)  // handle event
            {
                ...
                split thread to handle the longer taking tasks
                handle short tasks immediately
                ...
            }
            else    // no event
            {
                Shedule(idleIrpt,10);
            }
        }
    }

### System load

The dispatcher permanently monitors the cpu load of your application. These counter cells are incremented by the system timer. Therefore the sum of all cells plus idleCount increments by 1000 per second. A thread which displays system load may read and clear them.

#### long sysLoad[33]

These counter cells are incremented by the system timer according to the currently running thread's priority. They are sorted by log(2) of thread priority, that is:

            sysLoad[0]        prio 0  ..  1-1      (that is: 0 only)
            sysLoad[1]        prio 1  ..  2-1      (that is: 1 only)
            sysLoad[2]        prio 2  ..  4-1
            sysLoad[3]        prio 4  ..  8-1
            sysLoad[4]        prio 8  ..  16-1
              ...
            sysLoad[N]        prio 2^(N-1)  ..  (2^N)-1

#### long idleCount

Whenever there is no thread running when the system timer want's to increment a sysLoad[] cell, it increments this counter. This time should be released to the Mac OS and other applications. See idleFlag and idleIrpt.

#### Process time of threads

The system timer also increments a process time counter in the currently serviced thread which can be read directly from the Twsp data member ptime. As all timing in this dispatcher, it is measured in msec.

### int NumThreads() and variants

These functions count threads; either all existing or those with a special state:

int NumThreads ( ) | count all existing threads
int NumThreadsRunning ( ) | count all currently running threads
int NumThreadsTimeShed ( ) | count all time-sheduled threads
int NumThreadsSuspended( ) | count all suspended threads
int NumThreadsIrptShed ( ) | count all interrupt-sheduled threads
int NumThreadsSemaShed ( ) | count all threads blocked by a semaphor

Note: interrupt sheduled and semaphor sheduled threads are always also either suspended (if their request was without a timeout) or time-sheduled (if they requested the interrupt/semaphor with a timeout).

In addition, you can call NumThreads() with a mask defining the exact state of the threads you like to count. For the defined constants for the mask see disp_flag.