Galaxy Communicator Documentation:

The Toplevel Server Loop


The toplevel server loop in the MIT Communicator server library exploits a complex array of callback and object types. Understanding this structure is crucial to being able to do "unusual" things with the Communicator system. In this section, we'll describe the rough outline of the toplevel loop and how it relates to the dispatch functions.


Simulating a main loop

Here's a main loop which exposes the basic functionality. This code is taken from basic_double.c, in the simple_mainloop example. As the comment notes, this main loop is identical to the main loop in the Communicator library. GalIO_ServerStruct *GalSS_CmdlineSetupServer(int argc, char **argv)
This function creates and returns a server, according to the parameters provided in argc and argv. It first uses the MIT command line argument parsing library to parse the argument list and remove the general server arguments. Next, it calls the function _GalSS_InitializeDefaults, which is typically generated automatically from server information. It creates a server according to all this information, initializes signal handling, and calls _GalSS_init_server on the remaining arguments and stores the return value for later retrieval by GalIO_GetCommServerData.

voidGalSS_StartAndRunServer(GalIO_ServerStruct *server)
This function sets up the server listener and starts the timed task loop.


Argument packages

In order to allow the user to configure servers in the most extensible and flexible way possible, the Communicator infrastructure supports argument packages which can be manipulated before server instantiation.

GalIO_ServerStruct *GalSS_SetupServer(GalSS_ServerArgs *arg_pkg, int new_argc, char **new_argv)
Like GalSS_CmdnlineSetupServer, but sets up a server given the argument packagearg_pkg.

GalSS_ServerArgs *GalSS_ExtractCmdlineServerArgs(GalSS_ServerArgs *arg_pkg, int argc, char **argv, int *new_argc_ptr, char ***new_argv_ptr)
void GalSS_FreeArgPkg(GalSS_ServerArgs *arg_pkg)
In case you want to have access to all the standard server arguments, but you need to change some of the settings programmatically before you initialize the server, you can extract the server arguments using GalSS_ExtractCmdlineServerArgs and then pass the result to GalSS_SetupServer. GalSS_ExtractCmdlineServerArgs peels off the standard server arguments and returns the remainder via new_argc_ptr and new_argv_ptr. If GalSS_ExtractCmdlineServerArgs returns NULL, something's gone wrong and you should exit. Here's what the main loop would look like if you did that:

int main(int argc, char **argv)
{
  GalIO_ServerStruct *server;
  GalSS_ServerArgs *arg_pkg;
  int new_argc;
  char **new_argv;

  arg_pkg = GalSS_ExtractCmdlineServerArgs((GalSS_ServerArgs *) NULL,
                                           argc, argv, &new_argc, &new_argv);
  if (!arg_pkg) {
    GalUtil_Fatal("Failed to parse command line arguments!\n");
  }
  /* Do whatever you need to do */
  /* ... */
  server = GalSS_SetupServer(arg_pkg, new_argc, new_argv);
  if (!server) {
    GalUtil_Fatal("Failed to initialize server!\n");
  }
  GalSS_FreeArgPkg(arg_pkg);
  GalSS_StartAndRunServer(server);
  exit(0);
}

GalSS_ServerArgs *GalSS_DefaultServerArgs()
Sometimes, you may want to use the built-in argument parsing mechanism, but you might want to set some of the arguments so that the user can't modify them. In this case, you can create a default GalSS_ServerArgs * object with GalSS_DefaultServerArgs and use the functions below to set the appropriate arguments. In all these cases, the appropriate command line arguments will be disabled, and the values you provide will be used.

All these following functions can be used to modify the server args either before or after GalSS_ExtractServerArgs() is called. If called before, they disable the appropriate command line arguments, set a fixed value, and return the defaults (or a previously set value). If called after, they still disable the appropriate command line arguments (although this has no effect, since the values have already been extracted), but their primary effect is to allow the programmer to capture and overwrite information from the arguments before the server is created with GalSS_SetupServer(). See the alternate mainloop documentation for a complex example.

unsigned short GalSS_SAFixPort(GalSS_ServerArgs *arg_pkg, unsigned short port)
Sets the port to try to start the server listener on to port. 0 uses the default found in the server declaration information. Disables the -port command line argument. Returns the old value.

int GalSS_SAFixMaxConns(GalSS_ServerArgs *arg_pkg, int max_conns)
Sets the maximum number of connections permitted to max_conns. A non-positive integer uses the default found in the server declaration information, or 1 if there is no default. Disables the -maxconns command line argument. Returns the old value.

int GalSS_SAFixVerbosity(GalSS_ServerArgs *arg_pkg, int verbosity)
Sets the verbosity level. If verbosity is not -1, it is used as the verbosity level for the server. Disables the -verbosity command line argument. Returns the old value.

int GalSS_SAFixColor(GalSS_ServerArgs *arg_pkg, int color)
If color is > 0, printing will be set up for a color xterm; if 0, black and white, if < 0, it's assumed that printing is initialized elsewhere. Use 0 for this argument. Disables the -color command line argument. Returns the old value.

int GalSS_SAFixAssert(GalSS_ServerArgs *arg_pkg, int assert)
If assert is non-zero, the server initialization will fail if the declared port is not available, instead of searching for an available port.  Disables the -assert command line argument. Returns the old value.

int GalSS_SAFixValidate(GalSS_ServerArgs *arg_pkg, int validate)
If validate is non-zero, the server will check each dispatch function call against the message signature. Disables the -validate command line argument. Returns the old value.

int GalSS_SAFixLoopType(GalSS_ServerArgs *arg_pkg, int loop_type)
Sets the loop type. The loop_type argument can have one of three values:

Disables the -thread, -ttloop, -nottloop command line arguments. Returns the old value.

int GalSS_SAFixServerListenStatus(GalSS_ServerArgs *arg_pkg, int server_listen_status)
Sets the listen status (values documented in GalIO_ServerListenStatus()). If the listen status is fixed and it's a server listener, then -contact_hub and -session_id are disabled. Returns the old value.

char *GalSS_SAFixContactHubInfo(GalSS_ServerArgs *arg_pkg, char *client_pair_status, char *session_id, char **old_session_id_ptr)
Sets the Hub contact information, if the server is supposed to contact the Hub. The  client_pair_status and session_id arguments have the same form as the command line arguments described in the listener-in-Hub documentation. Disables the -contact_hub, -session_id command line arguments. Returns the old values. The returned memory is up to the caller to free. If old_session_id_ptr is NULL, the old session ID string is freed internally and not returned.

char *GalSS_SAFixServerLocations(GalSS_ServerArgs *arg_pkg, char *server_locations_file)
Sets the server location file. Disables the -server_locations_file command line argument. Returns the old value.


Manipulating the server object

There are a number of functions which can be used to manipulate the server object.

unsigned short GalIO_GetServerListenPort(GalIO_ServerStruct *scomm)
This function returns the port number that the server is actually listening on.

void *GalIO_GetServerData(GalIO_ServerStruct *scomm)
void GalIO_SetServerData(GalIO_ServerStruct *scomm, void *data, void (*free_fn)(void *))
These functions get and set the server data. This data can also be set by returning a value from _GalSS_init_server, although this behavior is strongly discouraged. The server data is also available for retrieval through individual connections using GalIO_GetCommServerData.

void *GalIO_GetCommServerData(GalIO_CommStruct *gcomm)
This function returns the value which was set by GalIO_SetServerData.

void GalIO_SetCommClientData(GalIO_CommStruct *gcomm, char *name, void *client_data)
void *GalIO_GetCommClientData(GalIO_CommStruct *gcomm, char *name)
void GalIO_SetServerClientData(GalIO_ServerStruct *server, char *name, void *client_data)
void *GalIO_GetServerClientData(GalIO_ServerStruct *server, char *name)
These functions get and set client_data associated with the dispatch function named by name. The same repository of client data information is accessible through the server or any of its connections. This functionality is useful when there's persistent data that's relevant to the dispatch function which it doesn't make sense to make global.

char *GalIO_GetServerName(GalIO_ServerStruct *scomm)
void GalIO_SetServerName(GalIO_ServerStruct *scomm, char *name)
These function get and set the name by which the server is known. Typically, this information is set in _GalSS_InitializeDefaults.

int GalIO_GetServerMaxConnections(GalIO_ServerStruct *scomm)
void GalIO_SetServerMaxConnections(GalIO_ServerStruct *scomm, int max)
These functions get and set the maximum number of connections the server may accept. Typically, the default case is handled before _GalSS_init_server is called, so you may reset it there if you choose. Note that this function can be used to reduce the number of maximum connections permitted, but it will silently fail if the number of active connections exceeds the maximum requested.

int GalIO_GetServerNumConnections(GalIO_ServerStruct *scomm)
Returns the number of connections currently connected to the server.

GalIO_CommStruct *GalIO_GetUniqueConnection(GalIO_ServerStruct *scomm)
This function will return the single active connection when there is an active connection and the maximum number of connections permitted is one. This is present mostly for backward compatibility. In releases before 2.0, there was no distinction between the server and the connection object, and only one connection at a time was permitted. In those situations, the code assumes easy access to the single permitted connection, given the server. This function provides that access.

unsigned short GalIO_GetServerDefaultPort(GalIO_ServerStruct *scomm)
void GalIO_SetServerDefaultPort(GalIO_ServerStruct *scomm, unsigned short port)
These functions get and set the default port for the server. Typically, the default case is handled before _GalSS_init_server is called, so you may reset it there if you choose.

int GalIO_ServerUsesTimedTasks(GalIO_ServerStruct *server)
This function returns 1 if the server is configured to use the timed task loop (the default case), 0 otherwise.

void GalIO_EnableDispatchFnValidation(GalIO_ServerStruct *scomm)
This function is called when the validate argument to GalSS_InitializeServerToplevel is set. All connections spawned by this server will validate each dispatch function call. Validation cannot be disabled.
 


Manipulating the connection object

When the server accepts a connection, it creates a GalIO_CommStruct object to handle that connection. This object can be accessed from the second argument of each dispatch function using the function GalSS_EnvComm. It is possible to use this connection object to access server information or to send information to the Hub. These functions, however, do not provide enough of a context to support appropriate session management. We strongly recommend using the environment-aware versions of these functions. See the documentation on adding a server and session management.

For example, the function GalIO_CommWriteFrame sends a new message to the Hub. We illustrate with a simplified variant of the double example:

Gal_Frame twice(Gal_Frame frame, void *server_data)
{
  Gal_Frame new_f = Gal_MakeFrame("main", GAL_CLAUSE);
  Gal_SetProp(new_f, ":int", Gal_IntObject(2 * Gal_GetInt(frame, ":int")));
  GalIO_CommWriteFrame(GalSS_EnvComm((GalSS_Environment *) server_data), new_f, 0);
  Gal_FreeFrame(new_f);
  return (Gal_Frame) NULL;
}
We see here that we cast the server_data back to a GalSS_Environment* to access the connection to send the message.

We exemplify the corresponding behavior for server-to-server subdialogues using a variant of the complex_twice function in the double example. Here, before the server doubles the input and submits a new token, it invokes a "multiply" message to multiply the input by some factor (set in the server which provides the "multiply" message):

Once again, we see here that we cast the server_data back to a GalSS_Environment* to access the connection to send the message.

GalIO_CommStruct *GalSS_EnvComm(GalSS_Environment *env)
Retrieves the connection object from the environment.

int GalIO_CommValidating(GalIO_CommStruct *gcomm)
Returns 1 if the connection is validating dispatch function calls, 0 otherwise.

void *GalIO_GetCommData(GalIO_CommStruct *gcomm)
void GalIO_SetCommData(GalIO_CommStruct *gcomm, void *data, void (*free_fn)(void *))
These functions get and set the data specific to a connection. If free_fn is non-NULL, it will be called on the data when the data is reset or the connection is destroyed.

char *GalIO_GetCommServerName(GalIO_CommStruct *gcomm)
This function retrieves the name by which the server is known from gcomm. This information was originally set by GalIO_SetServerName.

int GalIO_CommWriteFrame(GalIO_CommStruct *gcomm, Gal_Frame frame, int do_block)
This function writes a frame to the Hub through the gcomm connection. The type of the message is always GAL_MESSAGE_MSG_TYPE. See the section on message types.

Gal_Frame GalIO_DispatchViaHub(GalIO_CommStruct *gcomm, Gal_Frame frame, GalIO_MsgType *msg_type_ptr)
This function implements a server-to-server subdialogue with the Hub. It sends the frame and waits for a reply. The type of the reply is stored in msg_type_ptr. If the reply type is GAL_DESTROY_MSG_TYPE or GAL_MESSAGE_MSG_TYPE, this function prints a warning and returns NULL; therefore, the only legal values for *msg_type_ptr are GAL_REPLY_MSG_TYPE and GAL_ERROR_MSG_TYPE. See the section on message types.
 

Memory management

None of the frames related to these two functions are freed.

Event-driven programming

The Communicator infrastructure has a rich set of event-driven programming capabilities. Programmers can register callbacks to be run at any one of these events. This event-driven model is used to implement the Communicator main loop, as well as external main loops and programming language bindings. Using this model to its fullest, it is possible to write dispatch functions identically for the Communicator main loop and external main loops.

These event callbacks should not have any interdependencies among them. They are not guaranteed to be called in the order they are defined.
 
event description
GAL_SERVER_LISTENER_STARTUP_EVENT The server has just opened a listener on a port, either because it's listening for server connections or an outgoing broker requires it
GAL_SERVER_LISTENER_SHUTDOWN_EVENT The server is about to shut down its listener
GAL_SERVER_CLIENT_POLL_STARTUP_EVENT The server hast just started attempting to contact the Hub
GAL_SERVER_DESTRUCTION_EVENT The server is about to be destroyed
GAL_SERVER_CONNECTION_CREATION_EVENT The server has just created a new connection
GAL_CONNECTION_BROKER_OUT_STARTUP_EVENT The connection has just created an outgoing broker
GAL_CONNECTION_BROKER_IN_STARTUP_EVENT The connection has just created an incoming broker
GAL_CONNECTION_DISPATCH_FN_EVENT The connection is about to invoke a dispatch function
GAL_CONNECTION_SHUTDOWN_EVENT The connection is about to shuts down
GAL_CONNECTION_DESTRUCTION_EVENT The connection is about to be destroyed
GAL_BROKER_DATA_DONE_EVENT The broker has just determined it is done, either by receiving a termination message or via the call GalIO_BrokerDataDone.
GAL_BROKER_ABORT_EVENT The broker is about to be destroyed before determining it is done
GAL_BROKER_DESTRUCTION_EVENT The broker is about to be destroyed
GAL_BROKER_CONNECTION_EVENT The (outgoing) broker has just accepted a connection

IMPORTANT. These callbacks cannot be reentrant. If these callbacks call themselves, you'll get a deadlock in the threaded version of the library.


typedef void (*GalIO_ServerCallbackFn)(GalIO_ServerStruct *, void *);
GalIO_Callback *GalIO_AddServerCallback(GalIO_ServerStruct *scomm, int callback_event, GalIO_ServerCallbackFn fn, void *callback_data)
This function adds the callback fn to the server scomm. The possible values for callback_event are GAL_SERVER_LISTENER_STARTUP_EVENT, GAL_SERVER_LISTENER_SHUTDOWN_EVENT, GAL_SERVER_CLIENT_POLL_STARTUP_EVENT, GAL_SERVER_DESTRUCTION_EVENT. The fn is invoked with the server scomm and callback_data. The callback is returned.

typedef void (*GalIO_ServerConnectCallbackFn)(GalIO_ServerStruct *, GalIO_CommStruct *, void *);
GalIO_Callback *GalIO_AddServerConnectCallback(GalIO_ServerStruct *scomm, GalIO_ServerConnectCallbackFn connect_callback, void *callback_data)
This function adds the callback fn to the server scomm. The event is GAL_SERVER_CONNECTION_CREATION_EVENT. The fn is invoked with the server scomm, the new connection, and callback_data. The callback is returned.

void GalIO_RemoveServerCallback(GalIO_ServerStruct *scomm, GalIO_Callback *cb)
Removes the callback cb from the server scomm.

typedef void (*GalIO_ConnectionCallbackFn)(GalIO_CommStruct *, void *);
GalIO_Callback *GalIO_AddConnectionCallback(GalIO_CommStruct *gcomm, int callback_event, GalIO_ConnectionCallbackFn connect_callback, void *callback_data)
This function adds the callback fn to the connection gcomm. The possible values for callback_event are GAL_CONNECTION_SHUTDOWN_EVENT, GAL_CONNECTION_DESTRUCTION_EVENT. The fn is invoked with the connection gcomm and callback_data. The callback is returned.

typedef void (*GalIO_ConnectionBrokerCallbackFn)(GalIO_CommStruct *, GalIO_BrokerStruct *, void *);
GalIO_Callback *GalIO_AddConnectionBrokerCallback(GalIO_CommStruct *gcomm, int callback_event, GalIO_ConnectionBrokerCallbackFn connect_callback, void *callback_data)
This function adds the callback fn to the connection gcomm. The possible values for callback_event are  GAL_CONNECTION_BROKER_OUT_STARTUP_EVENT, GAL_CONNECTION_BROKER_IN_STARTUP_EVENT. The fn is invoked with the connection gcomm, the new broker, and callback_data. The callback is returned.

typedef void (*GalIO_ConnectionDispatchFnCallbackFn)(GalSS_Environment *, Gal_Frame, void *);
GalIO_Callback *GalIO_AddConnectionDispatchFnCallback(GalIO_CommStruct *gcomm, GalIO_ConnectionDispatchFnCallbackFn dispatch_callback, void *callback_data)
This function adds the callback fn to the connection gcomm. The event is GAL_CONNECTION_DISPATCH_FN_EVENT. The fn is invoked with the environment of the dispatch function call, the frame comprising the message, and callback_data. The callback is returned.

void GalIO_RemoveConnectionCallback(GalIO_CommStruct *gcomm, GalIO_Callback *cb)
Removes the callback cb from the connection gcomm.

typedef void (*GalIO_BrokerCallbackFn)(GalIO_BrokerStruct *, void *);
GalIO_Callback *GalIO_AddBrokerCallback(GalIO_BrokerStruct *b, int callback_event, GalIO_BrokerCallbackFn fn, void *callback_data)
This function adds the callback fn to the broker b. The possible values for callback_event are GAL_BROKER_DATA_DONE_EVENT, GAL_BROKER_ABORT_EVENT, GAL_BROKER_DESTRUCTION_EVENT, GAL_BROKER_CONNECTION_EVENT. The fn is invoked with the broker b and callback_data. The callback is returned.

void GalIO_RemoveBrokerCallback(GalIO_BrokerStruct *b, GalIO_Callback *cb)
Removes the callback cb from the broker b.


Listener-in-Hub support

As of version 2.1, it is possible to set up Hubs and servers so that servers contact listeners in the Hub, instead of Hubs contacting listeners in servers. This new functionality requires API support.

GalIO_CommStruct *GalIO_ContactHub(char *host, unsigned short port, GalIO_ServerStruct *scomm, char *session_id, int client_poll_flags)
This function is used to contact a Hub on the given host and port, and associate the resulting connection with the given server scomm. If the session_id argument is not NULL, the connection will use the given session ID instead of the default provided to GalSS_InitializeServerToplevel() (also accessible via the function GalIO_ServerSessionID()). This function can be used to add new connections to a server after the server starts up. The new connection will be processed according to the listener status associated with the server (see GalIO_ServerListenStatus()). If the client_poll_flags are not -1, they provide specific control over whether this connection is restarted or retried (see GalIO_ServerListenStatus).

void GalIO_OperateOnConnections(GalIO_ServerStruct *scomm, void *arg, void (*op)(GalIO_CommStruct *, void *))
This function applies the operation op to each connection associated with the server scomm. The operation is also passed arg.

int GalIO_ServerListenStatus(GalIO_ServerStruct *scomm)
The listener status is an integer whose bits correspond to various aspects of the server behavior. One set of bits control whether the server is listening for connections and/or brokers from the Hub and/or connecting as a client to the Hub; another set of bits is whether, as a Hub client, the server should retry an connection if it fails to connect; and a final set of bits control whether, as a Hub client the server should attempt to reconnect after a disconnect, do nothing, or shutdown when the last Hub disconnects. You can set this status using GalSS_SAFixServerListenStatus(). The constants and their masks are as follows:
 
Constant Mask Description
GAL_CONNECTION_LISTENER GAL_SERVER_TYPE_MASK Server listens for connections
GAL_BROKER_LISTENER GAL_SERVER_TYPE_MASK Server listens for brokers
GAL_HUB_CLIENT GAL_SERVER_TYPE_MASK Server connects to Hub
GAL_HUB_CLIENT_CONNECT_FAILURE_RETRY GAL_HUB_CLIENT_CONNECT_FAILURE_MASK Server connecting to Hub retries if it can't establish an initial connection
GAL_HUB_CLIENT_CONNECT_FAILURE_SHUTDOWN GAL_HUB_CLIENT_CONNECT_FAILURE_MASK Server connecting to Hub shuts down if it can't establish an initial connection
GAL_HUB_CLIENT_CONNECT_FAILURE_NOOP GAL_HUB_CLIENT_CONNECT_FAILURE_MASK Server connecting to Hub does nothing if it can't establish an initial connection
GAL_HUB_CLIENT_DISCONNECT_RETRY GAL_HUB_CLIENT_DISCONNECT_MASK Server connecting to Hub retries after disconnect
GAL_HUB_CLIENT_DISCONNECT_SHUTDOWN GAL_HUB_CLIENT_DISCONNECT_MASK Server connecting to Hub exits after last disconnect
GAL_HUB_CLIENT_DISCONNECT_NOOP GAL_HUB_CLIENT_DISCONNECT_MASK Server connecting to Hub does nothing after disconnect

Typically, you won't need to probe these flags directly, if you need them at all. The functions GalIO_ServerIsClient(), GalIO_ServerIsListener(), etc. will allow you to determine the server's status. Under some circumstances, you might need more complex information. For instance, if you want to know if your server will shutdown after the last disconnect, you can use the test

(GalIO_ServerListenStatus(scomm) & GAL_HUB_CLIENT_DISCONNECT_MASK) ==
   GAL_HUB_CLIENT_DISCONNECT_SHUTDOWN
And so on. The default listen status is GAL_CONNECTION_LISTENER; when the server type includes GAL_HUB_CLIENT, the default client bits are GAL_HUB_CLIENT_CONNECT_FAILURE_RETRY | GAL_HUB_CLIENT_DISCONNECT_RETRY. If you set multiple values for the client connect and disconnect statuses (which you should never do, because it's pointless, but just in case), the precedence is SHUTDOWN > NOOP > RETRY.

char *GalIO_ServerSessionID(GalIO_ServerStruct *scomm)
Returns the default session ID for the server, as determined by GalSS_InitializeServerToplevel(). The actual session ID is returned, not a copy.


Deprecated functions

These functions will still be supported, but will be removed in a future release. Any new command line arguments available to Communicator-compliant servers will not be available if you use these functions.

GalIO_ServerStruct *GalSS_CmdlineInitializeServer(int argc, char **argv)
Like GalSS_CmdlineSetupServer, but also starts up the appropriate listeners.

GalIO_ServerStruct *GalSS_InitializeServerToplevel(unsigned short server_port, int max_conns, int use_color, int do_assert, int loop_type, int validate, int verbosity, int server_listen_status, char *client_pair_string, char *session_id, int new_argc, char **new_argv)
This function was originally called internally by GalSS_CmdlineInitializeServer when the server-relevant information had been extracted from the arglist, but it became obvious that the signature of this function would have to be changed every time an argument was added to the servers; therefore, the functionality here has been superseded by argument packages. The arguments correspond to the various GalSS_SAFix* functions.

GalIO_ServerStruct *GalSS_InitializeServer(unsigned short server_port, int max_conns, int use_color, int do_assert, int use_ttloop, int validate, int new_argc, char **new_argv)
This function is an old version of GalSS_InitializeServerToplevel() which does not provide for initialization of verbosity or listener-in-Hub support. Provided for backward compability.

GalIO_ServerStruct *GalSS_InitializeServerFromServerArgs(GalSS_ServerArgs *arg_pkg, int new_argc, char **new_argv)

GalSS_ServerArgs *GalSS_ExtractServerArgs(int argc, char **argv, int *new_argc_ptr, char ***new_argv_ptr)
A version of GalSS_ExtractCmdlineServerArgs() which doesn't allow a default argument package. Provided for backward compatibility.

void GalSS_RunServer(GalIO_ServerStruct *server)
This function starts the timed task loop.


Please send comments and suggestions to:bugs-darpacomm@linus.mitre.org
Last updated December 27, 2001.

 Copyright (c) 1998 - 2001
The MITRE Corporation
ALL RIGHTS RESERVED