Galaxy Communicator Documentation:

Brokering and Audio Data


In Communicator, there are two ways of moving data from server to server. First, you can package the data into a frame and send it to another server via the Hub, using the standard frame dispatch functions. Alternatively, you can use the Hub to establish a direct connection between servers (called a broker connection), through which you can send data. This latter technique is used in the MIT architecture for transmitting audio data from the device server to the recognizer, and from the synthesizer to the device server. In this document, we describe and exemplify this brokering tool.

In some cases, it's not possible to use the MIT timed task loop. For information on how to handle brokering in those cases, look here.


Sending broker data

We've constructed an example which consists of two sides of a broker connection which transmits the contents of a file containing raw 8-bit audio data and and outputs the data to the audio device. Here's a code fragment which illustrates how the sender uses the broker connection: The important steps here are to set up the outgoing broker stream, constructing a well-formed frame for initiating a broker connection, sending the frame to the Hub (in this case, by a call to GalSS_EnvWriteFrame, not shown), writing the appropriate data to the connection, and announcing that no more data will be sent.

Step 1: setting up the outgoing broker stream

GalIO_BrokerStruct *GalIO_BrokerDataOutInit(GalIO_CommStruct *gcomm, int poll_ms, int timeout_seconds)
The outgoing broker uses the server's listener as its listener port. If the server is a Hub client and doesn't have an open listener, the broker sets up a listener for the server to host. The outgoing broker accesses the server via gcomm. The poll_ms is the time in milliseconds that the timed task loop is supposed to check the broker connection; as with GalIO_ServerInit, 0 is the default (100 ms), a positive number is an actual number of milliseconds, and < 0 means not to set up a timed task. The timeout_seconds is how long before the broker expires; -1 means never expire, 0 means use the default (which is 10 seconds). You can write to an expired broker, but it will accept no more connections; when all the data is written to the existing connections, the broker will be destroyed.

It's important to set the timeout intelligently, because outgoing brokers can now accept multiple connections. This means that they must cache the data which is written through them for subsequent subscribers. If the broker never times out, significant memory bloat might result.

Step 2: constructing a well-formed frame

The server needs to announce the availability of broker data to the Hub, and it does so by populating a frame with the appropriate information. A well-formed frame for initiating a broker connection contains the following keys: As of version 3.0, it is no longer the programmer's responsibility to construct a unique call ID for the brokers. In addition to being more convenient, much existing brokering code doesn't take into account the possibility that servers may have multiple outgoing brokers simultaneously available.

void GalIO_BrokerPopulateFrame(GalIO_BrokerStruct *b, Gal_Frame f, char *host_key, char *port_key)
Populates the frame f with the appropriate broker information, including the call ID, host and port. The host_key is the key to store the hostname in the frame, and the port_key is the key to store the port in. The call ID is stored in the key designated by GAL_BROKER_CALL_ID_FRAME_KEY.

unsigned short GalIO_GetBrokerListenPort(GalIO_BrokerStruct *b)
Returns the port number the broker b is listening on. If this value is not 0, the broker has been set up correctly.

char *GalIO_GetBrokerCallID(GalIO_BrokerStruct *b)
Returns the call ID the broker b is using. You should not typically need to use this function.

void GalIO_FrameSetBrokerCallID(Gal_Frame f, char *call_id)
Sets the unique ID for broker confirmation. You should not typically need to use this function.

char *GalIO_IPAddress(void)
This function returns the IP address of the host, for use in creating contact information for the broker connection. You should not typically need to use this function.

Step 3: writing data to the broker

Since brokers now accept multiple connections, all data which is written to an outgoing broker is cached until the broker expires. Each new client is guaranteed of getting all the data written to the broker, in the order in which it was written.

int GalIO_BrokerWriteFrame(GalIO_BrokerStruct *b, Gal_Frame frame)
int GalIO_BrokerWriteInt(GalIO_BrokerStruct *b, int i)
int GalIO_BrokerWriteFloat(GalIO_BrokerStruct *b, float f)
int GalIO_BrokerWriteList(GalIO_BrokerStruct *b, Gal_Object *elts, int n_elts)
int GalIO_BrokerWriteString(GalIO_BrokerStruct *b, char *str)
int GalIO_BrokerWriteBinary(GalIO_BrokerStruct *b, void *data, int n_bytes)
int GalIO_BrokerWriteInt16(GalIO_BrokerStruct *b, void *data, int n_ints)
int GalIO_BrokerWriteInt32(GalIO_BrokerStruct *b, void *data, int n_ints)
int GalIO_BrokerWriteInt64(GalIO_BrokerStruct *b, void *data, int n_ints)
int GalIO_BrokerWriteFloat32(GalIO_BrokerStruct *b, void *data, int n_floats)
int GalIO_BrokerWriteFloat64(GalIO_BrokerStruct *b, void *data, int n_floats)
These functions send data through the broker connection. The encoding for these elements is identical to their encoding within frames during message transport. These functions correspond in the obvious way to the Communicator object types.

int GalIO_BrokerWriteObject(GalIO_BrokerStruct *b, Gal_Object o)
Write a Gal_Object of any type through the broker connection. The encoding for these elements is identical to their encoding within frames during message transport.

Step 4: announcing the end of the data

void GalIO_BrokerDataOutDone(GalIO_BrokerStruct *b)
This function should be called on the broker structure when all the data has been sent. If you do not call this function, the incoming broker on the other end of the broker connection will never terminate.

void GalIO_ForceBrokerExpiration(GalIO_BrokerStruct *b)
This function causes the broker to expire immediately. You can use this in conjunction with GalIO_AddBrokerCallback to force a broker to expire after it's accepted a certain number of connections, for instance.

int GalIO_BrokerIsDone(GalIO_BrokerStruct *b)
Returns 1 if the broker is already marked as done, 0 otherwise.


Streaming broker data

There is a minor problem with the example we've presented here: the broker setup both establishes the broker and writes the data. In an application without threads, this means that the receiving broker won't be able to connect to the sending broker until all the data is written to the outgoing broker (because the server won't have a chance to accept connections for the outgoing broker until the broker setup returns).

For some applications, this may be acceptable, but it's almost certainly the case that you'll want to "stream" audio data, so that the receiving broker can start making use of it as soon as possible (say, to start speech recognition). In these cases, you need to exploit the timed task loop to send outbound data at periodic intervals (say, by polling the audio device and relaying the data when it's available), so that the server has a chance to accept connections.

Here's a version of the broker setup which exemplifies this strategy. This is a bit unnatural, since all the data is immediately available in a file, but it illustrates the streaming:

static Gal_Frame prepare_audio_frame(GalIO_CommStruct *gcomm, char *filename)
{
  Gal_Frame f = Gal_MakeFrame("main", GAL_CLAUSE);
  FILE *fp;
  GalIO_BrokerStruct *b;
  OutData *o = (OutData *) calloc(1, sizeof(OutData));
 
  /* In the code omitted here, the file is opened */

  /* .... */
 
  /* If we're the sending direction, we read binary data from the file,
     relaying it to the broker, until we find EOF. */
 
  b = GalIO_BrokerDataOutInit(gcomm, 0, 10);
  if (b && (GalIO_GetBrokerListenPort(b) > 0)) {
    GalIO_BrokerPopulateFrame(b, f, ":binary_host", ":binary_port");
    o->fp = fp;
    o->b = b;
    Gal_AddTask(__write_data, (void *) o, 10, 0, NULL);
  }

  return f;
}

Note that we use Gal_AddTask to set up a task to start writing the data in 10 milliseconds. Here's the task:
static void __write_data(Gal_TaskPkg *p)
{
  OutData *o = (OutData *) Gal_TaskPkgData(p);
  FILE *fp = o->fp;
  GalIO_BrokerStruct *b = o->b;
  char *buf = (char *) malloc(BLOCKSIZE * sizeof(char));
  int count;
 
  count = fread(buf, sizeof(char), BLOCKSIZE, fp);
  if (count) {
    GalIO_BrokerWriteBinary(b, buf, count);
  }
  free(buf);
  if (count == BLOCKSIZE) {
    /* Not done yet. */
    Gal_ReAddTask(p, (void *) o, 10, 0, NULL);
  } else {
    GalIO_BrokerDataOutDone(b);
    fclose(fp);
    free(o);
  }
}
We keep resetting the task until we run out of data.


Receiving broker data

Here's the other end of the connection from our example. The DataHandler structure is not part of the Communicator package; we use it for illustration: When the server receives the receive_audio message, it invokes this dispatch function. If it can find the appropriate host and port set, it sets up a broker receiving connection, makes it active, and exits. This connection takes a callback function and data to be stored in the broker structure for the callback function to use. If the broker is set up appropriately, this function will also set up other callbacks which are invoked when the broker is notified of a broker abort or that data is done (see the documentation on GalIO_AddBrokerCallback and the event-driven programming model in general for further details).

GalIO_BrokerStruct *GalSS_EnvBrokerDataInInit(GalSS_Environment *env, char *host, unsigned short port, Gal_Frame frame, GalIO_BrokerDataHandler fnptr, int poll_ms, void *refptr, void (*free_fn)(void *))
The call environment env is stored away and made available to the broker data handler fnptr via the function GalSS_BrokerGetEnvironment. The host and port are the location of the server to contact. The frame is the frame which delivered the connection message, which is forwarded to the other end of the broker connection to verify that the correct connection is being made. It must contain the :call_id key. The fnptr is a function of type GalIO_BrokerDataHandler which will be called when there is data to be read, and the caller_data is arbitrary data which will be stored in the broker structure (for instance, an object to collect the binary data, as show in the example here) and will be freed when the broker is destroyed using the caller_data_free_fn. This data can be retrieved using the function GalIO_GetBrokerData, as illustrated below. As usual, poll_ms is the number of milliseconds for the timed task; 0 means 100 ms, > 0 is a value in ms, < 0 means not to set up a timed task.

Memory management

The data passed to the callback function is not freed by the caller. It is up to the callback function itself to free the data when it's done with it.

Here's an example of a callback function: You can see how the types which are checked correspond to the types of the data which are sent. You can also see how the DataHandler object is retrieved from the broker structure. Finally, observe that the callback function frees the data it's passed after processing it.

Here's an example of a general broker callback, in this case fired when the broker is notified of an abort:

void __report_abort(GalIO_BrokerStruct *b, void *data)
{
  Gal_Frame f;
  GalSS_Environment *env = GalSS_BrokerGetEnvironment(b);

  f = Gal_MakeFrame("notify", GAL_CLAUSE);
  Gal_SetProp(f, ":notification", Gal_StringObject("Audio aborted."));
  GalSS_EnvWriteFrame(env, f, 0);
  Gal_FreeFrame(f);
}

You can see here how the callback uses GalSS_BrokerGetEnvironment to send a message with the appropriate context to the Hub.

void GalIO_SetBrokerActive(GalIO_BrokerStruct *b)
Enables the broker to process the data handler. Nothing will happen with the data until this function is called on the broker structure. This is to allow the user to control when the broker structures are activated (if, for instance, they play audio output as a side effect).

typedef void (*GalIO_BrokerDataHandler)(GalIO_BrokerStruct *broker_struct, void *object, Gal_ObjectType object_type, int object_count)
This is the type of the callback functions. The broker_struct is the broker structure which has been set up to receive the data. The object is the data itself. The type is the type of the object, and the object_count is how many elements of the data there are (this is interesting mostly in the case of array data sent by GalIO_BrokerWriteBinary, GalIO_BrokerWriteInt16, etc.).

void *GalIO_GetBrokerData(GalIO_BrokerStruct *b)
This function retrieves from the broker structure b the data which was passed in to the refptr argument to GalIO_CommBrokerDataInInit.

void GalIO_SetBrokerData(GalIO_BrokerStruct *b, void *caller_data, void (*free_fn)(void *))
If you need to update the caller data at any point, this function will free the old data using the previously set free function and update the caller data to caller_data and the free function to free_fn.

void GalIO_BrokerDataDone(GalIO_BrokerStruct *b)
This function should be called on the broker structure if the client decides it's done before all the data is read.

void GalSS_BrokerSetEnvironment(GalIO_BrokerStruct *b, GalSS_Environment *env)
Sets the environment associated with the broker b to env. You should not typically need to use this function.

GalSS_Environment *GalSS_BrokerGetEnvironment(GalIO_BrokerStruct *b)
Retrieves the environment associated with the broker b.

Gal_Frame GalIO_GetBrokerFrame(GalIO_BrokerStruct *b)
Returns the internal copy of the frame passed to GalIO_CommBrokerDataInInit. You should not typically need to use this function.

If you're dealing with multiple incoming broker objects simultaneously, you might wish to use the broker queues. Broker objects can be connected in a doubly-linked list for the purposes of ordering their processing.

GalIO_BrokerStruct *GalIO_BrokerStructQueueAppend(GalIO_BrokerStruct *b, GalIO_BrokerStruct *bqueue)
Adds the broker structure b to the end of the queue bqueue. If bqueue is NULL, b is returned, otherwise bqueue is returned.

GalIO_BrokerStruct *GalIO_BrokerStructQueuePop(GalIO_BrokerStruct *bqueue)
Removed all finished broker structure from the queue and returns the broker structure which corresponds to the new head of the active queue. The broker structure is destroyed.

GalIO_BrokerStruct *GalIO_BrokerStructDequeue(GalIO_BrokerStruct *b, GalIO_BrokerStruct *bqueue)
Removes a broker structure from the queue and returns the new queue.


Backward compatibility

These functions are provided for backward compatibility with previous releases of the Galaxy Communicator infrastructure. They are all inadequate in one way or another, and should be avoided. They will not be deprecated or removed yet, however.

GalIO_BrokerStruct *GalIO_BrokerDataInInit(char *host, unsigned short port, Gal_Frame frame, GalIO_BrokerDataHandler fnptr, void *caller_data, int poll_ms)
Like GalIO_CommBrokerDataInInit, but doesn't allow the programmer to pass in the host connection or a free function for the data.

void *GalIO_GetBrokerCallerData(GalIO_BrokerStruct *b)
Identical to GalIO_GetBrokerData but does not conform to consistent naming conventions.

typedef void (*GalIO_BrokerDataFinalizer)(GalIO_BrokerStruct *, void *caller_data)
void GalIO_BrokerSetFinalizer(GalIO_BrokerStruct *b, GalIO_BrokerDataFinalizer finalizer)
Sets a finalizer for the broker structure. This finalizer is called when the broker structure is destroyed. This can be used, for instance, to pop a broker queue and activate the next element in the queue. The finalizer is called with the contents of the refptr argument passed in to GalIO_BrokerDataInInit. This function has been superseded by the addition of a free function in GalIO_CommBrokerDataInInit and by the callback architecture described here.

GalIO_BrokerStruct *GalIO_CommBrokerDataInInit(GalIO_CommStruct *host_gcomm, char *host, unsigned short port, Gal_Frame frame, GalIO_BrokerDataHandler fnptr, int poll_ms, void *caller_data, void (*caller_data_free_fn)(void *))
This function provides almost the same functionality as GalSS_EnvBrokerDataInInit, except that the host_gcomm is the connection which hosts the callback from which the broker was created; it is provided to support the event-driven programming model. All other arguments are as they are for GalSS_EnvBrokerDataInInit.


Please send comments and suggestions to:bugs-darpacomm@linus.mitre.org
Last updated October 5, 2001.

 Copyright (c) 1998 - 2001
The MITRE Corporation
ALL RIGHTS RESERVED