In Communicator, there are two ways of moving data from server to server. First, you can package the data into a frame and send it to another server via the Hub, using the standard frame dispatch functions. Alternatively, you can use the Hub to establish a direct connection between servers (called a broker connection), through which you can send data. This latter technique is used in the MIT architecture for transmitting audio data from the device server to the recognizer, and from the synthesizer to the device server. In this document, we describe and exemplify this brokering tool.
In some cases, it's not possible to use the MIT timed task loop. For information on how to handle brokering in those cases, look here.
/* In the code omitted here, the data from the named file
is
read into buf, and total is the number
of bytes in buf */
/* .... */
/* Now that we have the audio, we write the binary data
through the broker. */
b = GalIO_BrokerDataOutInit(gcomm, 0, 10);
if (b && (GalIO_GetBrokerListenPort(b) > 0)) {
GalIO_BrokerPopulateFrame(b, f, ":binary_host",
":binary_port");
GalIO_BrokerWriteBinary(b, buf, total);
GalIO_BrokerDataOutDone(b);
}
free(buf);
return f;
}
It's important to set the timeout intelligently, because outgoing brokers can now accept multiple connections. This means that they must cache the data which is written through them for subsequent subscribers. If the broker never times out, significant memory bloat might result.
void GalIO_BrokerPopulateFrame(GalIO_BrokerStruct
*b, Gal_Frame f, char *host_key, char *port_key)
Populates the frame f with the
appropriate broker information, including the call ID, host and port. The
host_key
is the key to store the hostname in the frame, and the
port_key
is the key to store the port in. The call ID is stored in the key designated
by GAL_BROKER_CALL_ID_FRAME_KEY.
unsigned short GalIO_GetBrokerListenPort(GalIO_BrokerStruct
*b)
Returns the port number the broker b
is listening on. If this value is not 0, the broker has been set up correctly.
char *GalIO_GetBrokerCallID(GalIO_BrokerStruct
*b)
Returns the call ID the broker b
is using. You should not typically need to use this function.
void GalIO_FrameSetBrokerCallID(Gal_Frame
f,
char *call_id)
Sets the unique ID for broker confirmation.
You should not typically need to use this function.
char *GalIO_IPAddress(void)
This function returns the IP address of
the host, for use in creating contact information for the broker connection.
You should not typically need to use this function.
int GalIO_BrokerWriteFrame(GalIO_BrokerStruct
*b, Gal_Frame frame)
int GalIO_BrokerWriteInt(GalIO_BrokerStruct
*b, int i)
int GalIO_BrokerWriteFloat(GalIO_BrokerStruct
*b, float f)
int GalIO_BrokerWriteList(GalIO_BrokerStruct
*b, Gal_Object *elts, int n_elts)
int GalIO_BrokerWriteString(GalIO_BrokerStruct
*b, char *str)
int GalIO_BrokerWriteBinary(GalIO_BrokerStruct
*b, void *data, int n_bytes)
int GalIO_BrokerWriteInt16(GalIO_BrokerStruct
*b, void *data, int n_ints)
int GalIO_BrokerWriteInt32(GalIO_BrokerStruct
*b, void *data, int n_ints)
int GalIO_BrokerWriteInt64(GalIO_BrokerStruct
*b, void *data, int n_ints)
int GalIO_BrokerWriteFloat32(GalIO_BrokerStruct
*b, void *data, int n_floats)
int GalIO_BrokerWriteFloat64(GalIO_BrokerStruct
*b, void *data, int n_floats)
These functions send data through the
broker connection. The encoding for these elements is identical to their
encoding within frames during message transport. These functions correspond
in the obvious way to the Communicator
object
types.
int GalIO_BrokerWriteObject(GalIO_BrokerStruct
*b, Gal_Object o)
Write a Gal_Object of any type through
the broker connection. The encoding for these elements is identical to
their encoding within frames during message transport.
void GalIO_ForceBrokerExpiration(GalIO_BrokerStruct
*b)
This function causes the broker to expire
immediately. You can use this in conjunction with GalIO_AddBrokerCallback
to force a broker to expire after it's accepted a certain number of connections,
for instance.
int GalIO_BrokerIsDone(GalIO_BrokerStruct
*b)
Returns 1 if the broker is already marked
as done, 0 otherwise.
For some applications, this may be acceptable, but it's almost certainly the case that you'll want to "stream" audio data, so that the receiving broker can start making use of it as soon as possible (say, to start speech recognition). In these cases, you need to exploit the timed task loop to send outbound data at periodic intervals (say, by polling the audio device and relaying the data when it's available), so that the server has a chance to accept connections.
Here's a version of the broker setup which exemplifies this strategy. This is a bit unnatural, since all the data is immediately available in a file, but it illustrates the streaming:
static Gal_Frame prepare_audio_frame(GalIO_CommStruct *gcomm, char *filename)Note that we use Gal_AddTask to set up a task to start writing the data in 10 milliseconds. Here's the task:
{
Gal_Frame f = Gal_MakeFrame("main", GAL_CLAUSE);
FILE *fp;
GalIO_BrokerStruct *b;
OutData *o = (OutData *) calloc(1, sizeof(OutData));
/* In the code omitted here, the file is opened *//* .... */
/* If we're the sending direction, we read binary data from the file,
relaying it to the broker, until we find EOF. */
b = GalIO_BrokerDataOutInit(gcomm, 0, 10);
if (b && (GalIO_GetBrokerListenPort(b) > 0)) {
GalIO_BrokerPopulateFrame(b, f, ":binary_host", ":binary_port");
o->fp = fp;
o->b = b;
Gal_AddTask(__write_data, (void *) o, 10, 0, NULL);
}return f;
}
static void __write_data(Gal_TaskPkg *p)We keep resetting the task until we run out of data.
{
OutData *o = (OutData *) Gal_TaskPkgData(p);
FILE *fp = o->fp;
GalIO_BrokerStruct *b = o->b;
char *buf = (char *) malloc(BLOCKSIZE * sizeof(char));
int count;
count = fread(buf, sizeof(char), BLOCKSIZE, fp);
if (count) {
GalIO_BrokerWriteBinary(b, buf, count);
}
free(buf);
if (count == BLOCKSIZE) {
/* Not done yet. */
Gal_ReAddTask(p, (void *) o, 10, 0, NULL);
} else {
GalIO_BrokerDataOutDone(b);
fclose(fp);
free(o);
}
}
d->data_buf = (char *) NULL;
d->size = 0;
if (host && port) {
b = GalSS_EnvBrokerDataInInit((GalSS_Environment
*) server_data,
host, port, f,
env_audio_handler, 0, d, __FreeDataHandler);
if (b) {
GalIO_SetBrokerActive(b);
GalIO_AddBrokerCallback(b, GAL_BROKER_ABORT_EVENT,
__report_abort, (void *) NULL);
GalIO_AddBrokerCallback(b, GAL_BROKER_DATA_DONE_EVENT,
__report_done, (void *) NULL);
}
} else {
free(d);
}
return (Gal_Frame) NULL;
}
GalIO_BrokerStruct *GalSS_EnvBrokerDataInInit(GalSS_Environment
*env, char *host, unsigned short port, Gal_Frame frame,
GalIO_BrokerDataHandler fnptr, int poll_ms, void *refptr,
void (*free_fn)(void *))
The call environment env is stored
away and made available to the broker data handler fnptr via the
function GalSS_BrokerGetEnvironment. The host and port are
the location of the server to contact. The frame is the frame which
delivered the connection message, which is forwarded to the other end of
the broker connection to verify that the correct connection is being made.
It must contain the :call_id
key. The fnptr is a function of type
GalIO_BrokerDataHandler
which will be called when there is data to be read, and the caller_data
is arbitrary data which will be stored in the broker structure (for instance,
an object to collect the binary data, as show in the example here) and
will be freed when the broker is destroyed using the caller_data_free_fn.
This data can be retrieved using the function GalIO_GetBrokerData,
as illustrated below. As usual, poll_ms is the number of milliseconds
for the timed task; 0 means 100 ms, > 0 is a value in ms, < 0 means
not to set up a timed task.
Memory managementHere's an example of a callback function:The data passed to the callback function is not freed by the caller. It is up to the callback function itself to free the data when it's done with it.
switch (data_type) {
case GAL_BINARY:
if (d->data_buf)
d->data_buf = (char *) realloc(d->data_buf,
n_samples + d->size);
else
d->data_buf = (char *) malloc(n_samples
+ d->size);
bcopy(data, d->data_buf + d->size, n_samples);
d->size += n_samples;
free(data);
break;
default:
GalUtil_Warn("Unknown data type %s\n", Gal_ObjectTypeString(data_type));
}
}
Here's an example of a general broker callback, in this case fired when the broker is notified of an abort:
void __report_abort(GalIO_BrokerStruct *b, void *data)You can see here how the callback uses GalSS_BrokerGetEnvironment to send a message with the appropriate context to the Hub.
{
Gal_Frame f;
GalSS_Environment *env = GalSS_BrokerGetEnvironment(b);f = Gal_MakeFrame("notify", GAL_CLAUSE);
Gal_SetProp(f, ":notification", Gal_StringObject("Audio aborted."));
GalSS_EnvWriteFrame(env, f, 0);
Gal_FreeFrame(f);
}
void GalIO_SetBrokerActive(GalIO_BrokerStruct
*b)
Enables the broker to process the data
handler. Nothing will happen with the data until this function is called
on the broker structure. This is to allow the user to control when the
broker structures are activated (if, for instance, they play audio output
as a side effect).
typedef void (*GalIO_BrokerDataHandler)(GalIO_BrokerStruct
*broker_struct, void *object, Gal_ObjectType object_type,
int object_count)
This is the type of the callback functions.
The broker_struct is the broker structure which has been set up
to receive the data. The object is the data itself. The type
is the type of the object, and the object_count is how many elements
of the data there are (this is interesting mostly in the case of array
data sent by GalIO_BrokerWriteBinary,
GalIO_BrokerWriteInt16,
etc.).
void *GalIO_GetBrokerData(GalIO_BrokerStruct
*b)
This function retrieves from the broker
structure b the data which was passed in to the refptr argument
to GalIO_CommBrokerDataInInit.
void GalIO_SetBrokerData(GalIO_BrokerStruct
*b, void *caller_data, void (*free_fn)(void *))
If you need to update the caller data
at any point, this function will free the old data using the previously
set free function and update the caller data to caller_data and
the free function to free_fn.
void GalIO_BrokerDataDone(GalIO_BrokerStruct
*b)
This function should be called on the
broker structure if the client decides it's done before all the data is
read.
void GalSS_BrokerSetEnvironment(GalIO_BrokerStruct
*b, GalSS_Environment *env)
Sets the environment associated with the
broker b to env. You should not typically need to use this
function.
GalSS_Environment *GalSS_BrokerGetEnvironment(GalIO_BrokerStruct
*b)
Retrieves the environment associated with
the broker b.
Gal_Frame GalIO_GetBrokerFrame(GalIO_BrokerStruct
*b)
Returns the internal copy of the frame
passed to GalIO_CommBrokerDataInInit.
You should not typically need to use this function.
If you're dealing with multiple incoming broker objects simultaneously, you might wish to use the broker queues. Broker objects can be connected in a doubly-linked list for the purposes of ordering their processing.
GalIO_BrokerStruct *GalIO_BrokerStructQueueAppend(GalIO_BrokerStruct
*b, GalIO_BrokerStruct *bqueue)
Adds the broker structure b to
the end of the queue bqueue. If bqueue is NULL, b
is returned, otherwise bqueue is returned.
GalIO_BrokerStruct *GalIO_BrokerStructQueuePop(GalIO_BrokerStruct
*bqueue)
Removed all finished broker structure
from the queue and returns the broker structure which corresponds to the
new head of the active queue. The broker structure is destroyed.
GalIO_BrokerStruct *GalIO_BrokerStructDequeue(GalIO_BrokerStruct
*b, GalIO_BrokerStruct *bqueue)
Removes a broker structure from the queue
and returns the new queue.
GalIO_BrokerStruct *GalIO_BrokerDataInInit(char
*host, unsigned short port, Gal_Frame frame, GalIO_BrokerDataHandler
fnptr,
void *caller_data, int poll_ms)
Like GalIO_CommBrokerDataInInit,
but doesn't allow the programmer to pass in the host connection or a free
function for the data.
void *GalIO_GetBrokerCallerData(GalIO_BrokerStruct
*b)
Identical to GalIO_GetBrokerData
but does not conform to consistent naming conventions.
typedef void (*GalIO_BrokerDataFinalizer)(GalIO_BrokerStruct
*, void *caller_data)
void GalIO_BrokerSetFinalizer(GalIO_BrokerStruct
*b, GalIO_BrokerDataFinalizer finalizer)
Sets a finalizer for the broker structure.
This finalizer is called when the broker structure is destroyed. This can
be used, for instance, to pop a broker queue and activate the next element
in the queue. The finalizer is called with the contents of the refptr
argument passed in to GalIO_BrokerDataInInit. This function has been superseded
by the addition of a free function in GalIO_CommBrokerDataInInit
and by the callback architecture described here.
GalIO_BrokerStruct *GalIO_CommBrokerDataInInit(GalIO_CommStruct
*host_gcomm, char *host, unsigned short port, Gal_Frame
frame,
GalIO_BrokerDataHandler fnptr, int poll_ms, void *caller_data,
void (*caller_data_free_fn)(void *))
This function provides almost the same
functionality as GalSS_EnvBrokerDataInInit,
except that the host_gcomm is the connection which hosts the callback
from which the broker was created; it is provided to support the event-driven
programming model. All other arguments are as they are for GalSS_EnvBrokerDataInInit.
Please send comments and suggestions to:bugs-darpacomm@linus.mitre.org
Last updated October 5, 2001.
Copyright (c) 1998 - 2001
The MITRE
Corporation
ALL RIGHTS RESERVED