Galaxy Communicator Tutorial:
Setting Up a Brokered Audio Connection
Up to this point, all the communication
between servers has been mediated by the Hub. This arrangement has its
advantages: for instance, the Hub provides a central location for flow
of control and routing. However, it also has its disadvantages: in order
for a message to get from one server to another, it has to be decoded,
analyzed, and reconstructed in the Hub. While this process appears to be
quite efficient, there are circumstances in which you might prefer to send
data directly: for example, in situations involving high bandwidth or streaming,
such as audio input or output.
To support this alternative, the Galaxy
Communicator infrastructure supports a peer-to-peer connection called a
broker.
The Hub mediates the establishment of this connection, but all data which
flows through the connection goes directly from server to server. In this
lesson, we'll learn how to set up a broker connection and how it works.
Once you feel comfortable with this lesson,
you can consult the broker reference
documentation.
Recognizer,
Synthesizer and Audio
Up to this point, we've examined four servers
in the toy travel demo: the Parser, Backend and Generator servers, which
represent the simplest type of server, and the Dialogue server, which represents
the next level of complexity. The first three servers simply send responses
to dispatch functions, while the Dialogue server sends new messages to
the Hub and occasionally waits for the response. The last three servers
take advantage of all these tools, and furthermore interact with each other
via broker connections.
In addition, the Audio server also exhibits
the complexities of UI programming, which we'll explore in the next lesson.
Brokering
in four steps
The setup and operation of brokers can be
broken down into four steps. We'll describe and illustrate these steps
briefly, and then look at some code. We'll focus on the interaction between
the Synthesizer and Audio servers as an example.
Step
1: Establish the broker listener
In the first step, the server which will be
the source of the broker data establishes a listener for the broker. This
listener, in most cases, is the same listener that's listening for connections
from the Hub; the protocol is constructed so that the listener can distinguish
between the two types of connections. The server assigns each broker a
unique ID, so that the server can listen for multiple brokers simultaneously.
The server also assigns each broker a timeout; it will accept multiple
connections for the broker, until the timeout is reached.
Step
2: Notify the Hub and client
In the second step, the source server sends
a new message to the Hub which contains the broker contact information.
This information consists of the host, port and unique ID associated with
this particular broker. This message is a message like any other, and can
contain additional information (such as encoding format and sample rate
for audio).
Step
3: Establish the broker connection, transmit data
When the target server receives this information,
it establishes a connection to the source server using the host, port and
unique ID, and establishes a callback to receive the data. The source server
can now send data over the connection, and the callback will be invoked
in the target server whenever data is received.
Step
4: Process data, shut down broker listener
Finally, when the source server is done sending
data, it notifies the target server that it's done, and the broker connection
is terminated. The target server does whatever it wants to do with the
data (e.g., send audio to the audio device). When the source server's broker
reaches its timeout, it stops accepting connections for that broker; when
all the connections for the broker are done, the source server shuts down
the broker.
Now that we've seen these steps in the abstract,
let's take a look at each one of them in detail.
Brokering
details
Source
server: broker setup, message dispatch
In the first and second brokering steps, the
source server sets up a broker listener and sends a message to the Hub
announcing the broker. Here's how that happens.
There are three functions involved.
-
The function GalIO_BrokerDataOutInit
creates the broker listener. The first argument of this function is the
connection which the call environment contains (which can be retrieved
from the call environment using the function GalSS_EnvComm).
This connection, and the server it's associated with, will host the broker
and its listener. The second argument is how often (or whether) to allow
the toplevel Communicator loop to poll this broker to see if there's anything
to do; 0 means use the default poll value. The third argument is the timeout,
in seconds; after this amount of time, the broker will stop accepting new
connections.
-
The function GalIO_GetBrokerListenPort
can be used to check that the broker listener has been set up correctly.
If it returns 0, there's a problem.
-
The function GalIO_BrokerPopulateFrame
is used to insert the appropriate broker information into the message being
sent to the Hub. The host and port will be stored in the indicated keys;
the unique call ID is always stored in the :call_id
key.
After the message is populated with the appropriate
brokering information, it is sent to the Hub and freed, just like any other
message.
Hub:
broker client notification
The Hub portion of the process is the simplest,
and you can probably guess what it looks like:
PROGRAM: FromSynthesizer
RULE: :host & :port & :call_id --> Audio.Play
IN: :host :port :encoding_format :sample_rate :call_id
The Hub matches the incoming message with
the appropriate program, shown here, and checks the rule. All three keys
are present in the token: :host
and :port were specified in
the call to GalIO_BrokerPopulateFrame,
and the :call_id key is the
key which GalIO_BrokerPopulateFrame
always uses for the unique broker ID. So the rule fires, and the relevant
data is sent to the Audio.Play
dispatch function.
Target
server: set up broker client, establish callback
When the Audio server invokes the Audio.Play
dispatch function, it creates a broker client, as follows:
There are three functions involved in
setting up the client correctly.
-
The function GalSS_EnvBrokerDataInInit
creates the broker client. It requires a call environment object (in case
the callback needs the environment to write new messages to the Hub), the
host and port obtained from the message frame, the message frame itself
(to get the unique ID from), and a data callback function which is
invoked whenever the target server receives data over the broker connection
(and a few other arguments we won't discuss). If successful, this function
returns a connected broker client.
-
The function GalIO_AddBrokerCallback
allows the programmer to associate additional behavior with various events
associated with the broker (more on this later). The important aspect of
this call here is that the programmer can associate behavior with the conclusion
of a broker connection.
-
The function GalIO_SetBrokerActive
activates the broker. Under some circumstances, broker clients might need
to be evaluated in a particular order (say, if you want to guarantee that
audio output is played in the order in which the brokers are set up). One
way to control this is by ensuring that broker clients don't do anything
unless specifically requested to. As a result, all broker clients are created
inactive, and you must activate them to have them do anything. This aspect
of broker clients is a historical idiosyncracy, and we recommend that you
activate all broker clients when you create them, and use your own mechanisms
to impose any ordering constraints.
At this point, the source and target servers
have established a broker connection, and the broker client can begin to
consume data.
About broker
data
When we discussed frames
and objects, we talked about the object types that could appear in
frames, and concentrated on frames, strings, integers and lists. We suggested
at that point that there were many
other datatypes which could appear in frames, including arrays of integers
of various sizes. Any type that can appear in a frame can be sent across
a broker connection, and vice versa, but we won't be looking at frames,
string, integers and lists in our broker example; we'll be looking at arrays
of integers.
When data is written to a broker server,
it is encoded and cached in an output queue. If any broker clients are
currently connected, it is also written to the client; and when a new client
connects, the server transmits the contents of the output queue to the
new client. In other words, all clients which connect to the broker server
are guaranteed to see all the same data in the same order, no matter when
they connect. This is one of the reasons that broker servers are set up
to expire; if a broker server didn't expire, it would never be possible
to free its output queue.
The
two ends of the broker connection
On the source server side, the server writes
data to the broker, and then indicates that it's done, approximately as
follows:
void *data;
int num_samples;
GalIO_BrokerStruct *b;
SynthesisRecord *s;
/* ... */
data = PollSynthesis(s, &num_samples);
while (data) {
GalIO_BrokerWriteInt16(b, data, num_samples);
free(data);
data = PollSynthesis(s, &num_samples);
}
if (SynthesisIsDone(s)) {
GalIO_BrokerDataOutDone(b);
/* ... */
There are two functions involved in this example.
-
The function GalIO_BrokerWriteInt16
is one of a family of functions which write typed data to a broker. This
particular function writes an array of 16-bit integers to the broker. The
second argument is the array of data, and the third argument is the number
of elements in the array. This function can be called as many times as
desired, and data of different types can be mixed in the same broker connection
(although typically isn't).
-
When there's no more data to send, the function
GalIO_BrokerDataOutDone
must be called. This call causes a special message to be sent over the
broker connection which indicates that all data has been written. If this
function is not called, the broker clients will never terminate and disconnect,
and the broker server will never die.
On the target server side, the data callback
function is invoked each time a block of data is received (each block corresponds
to a single call to one of the GalIO_BrokerWrite* functions). The
data callback function looks approximately like this:
static void __AudioOutputCallback(GalIO_BrokerStruct *b,
void *data,
Gal_ObjectType data_type,
int n_samples)
{
/* ... */
switch(data_type) {
case GAL_INT_16:
printf("[Audio data to user (%d samples)]\n",
n_samples);
/* Do something with the audio */
free(data);
break;
default:
GalUtil_Warn("__AudioOutputCallback: unexpected
data type %s\n",
Gal_ObjectTypeString(data_type));
free(data);
break;
}
}
Each data callback function has the same signature.
The first argument is the broker client. The second argument is a pointer
to the data, which is interpreted according to the data
type in the third argument. So, for instance, if the data type is GAL_FRAME,
then the data is really a Gal_Frame structure; if the data type is GAL_STRING,
the data is really a char *.
In this case, all the data which the source server wrote is an array of
16-bit integers; the corresponding type is GAL_INT_16. The fourth object
is interesting only for the array and list types; it's the number of elements
in the array or list.
So when the target server receives a block
of data, it invokes the data callback, which decides what to do with the
data based on the data type. The data is allocated as new memory before
it's passed to the data callback function, so once it's done with the data,
it frees it. This happens for each block of data (that is, each GalIO_BrokerWrite* call
on the source side), until the broker client receives the termination message,
at which point it will do whatever it's supposed to do when it's done (see
the call to GalIO_AddBrokerCallback here),
and terminate.
An example
Let's watch a broker in action. This is slightly
complicated by the fact that the unit tester cannot host broker servers
or clients. So we need to use the unit tester, acting as a server, to send
a message to the Hub which will cause the Synthesizer server to produce
a broker connection. This is actually pretty simple; we'll just send a
Synthesizer.Synthesize message
to the Hub, and populate its program file with the program we saw here,
to handle the new message from the Synthesizer server.
[Brokering exercise 1]
% process_monitor $GC_HOME/tutorial/brokering/synth.config
Select "Process Control --> Restart all",
then select "Send new message" in the unit tester. Select the message
named Synthesizer.Synthesize,
and press OK. You'll see the result in the Audio pane:
[Audio client pane]
[Audio data to user (1024 samples)]
[Audio data to user (1024 samples)]
[Audio data to user (1024 samples)]
[Audio data to user (1024 samples)]
[Audio data to user (1024 samples)]
[Audio data to user (1024 samples)]
[Audio data to user (1024 samples)]
[Audio data to user (1024 samples)]
[Audio data to user (1024 samples)]
[Audio data to user (1024 samples)]
[Audio data to user (1024 samples)]
[Audio data to user (1024 samples)]
[Audio data to user (1024 samples)]
[Audio data to user (1024 samples)]
[Audio data to user (184 samples)]
[Audio data to user is finalized (14520 samples).]
Each of these printouts corresponds to a single
call to the data callback function. The Hub shows how the information was
conveyed from the source server to the target server:
[Hub pane]
------ find next op ----------------------
{c FromSynthesizer
:host "129.83.10.107"
:port 15500
:call_id "129.83.10.107:24405:0"
:sample_rate 8000
:encoding_format "linear16"
:session_id "Default"
:tidx 3 }
--------------------------------------------
found operation: Audio.Play
---- serve(Audio@<remote>:-1, token 3 op_name Play (sole provider))
Got reply from provider for Audio (id 2) : token 3
----------------[ 3]----------------------
{c FromSynthesizer
:host "129.83.10.107"
:port 15500
:call_id "129.83.10.107:24405:0"
:sample_rate 8000
:encoding_format "linear16"
:session_id "Default"
:tidx 3 }
----------------------------------------
[Destroying token 3]
The FromSynthesizer
message constructed by the Synthesizer server contains the host, port and
unique broker ID, and the program shown here
sends the appropriate message to the Audio server. Finally, the Synthesizer
pane shows the evidence of the broker client connecting, the unique ID matching,
and the transmission of the special conclusion message:
[Synthesizer pane]
__GalIO_ServerContactHandler: connected to 129.83.10.107 (sock=6)
_GalIO_BrokerListenerHandshakeHandler: reference frame matched
GalIO_BrokerDataOutCallbackHandler: done sending data.
Select "File --> Quit" to end this exercise.
Why
the dependency on :call_id?
At this point, you may be a little puzzled
about why this mechanism is so complex. In particular, there are three
keys which must be sent from source to target server, and one of the keys
(:call_id) is a "magic" value.
In addition, there's no way to tell that there's a broker request being
passed around; nothing guarantees that the :call_id
key is used only for brokers. Finally, the mechanism seems too flexible
for the normal case; while brokers support sending arbitrary data of different
types through a broker connection, in most circumstances all the data will
be the same type.
There are at least two circumstances in
which this is not just inelegant, but a real implementational problem.
For example, let's say we want to define an HTTP bridge which would
allow us to transmit Communicator interactions through a firewall over
HTTP. In order to do this in a general way, we'd have to be able to identify
broker requests automatically, because the broker data would have to be
consumed and encoded over the same HTTP connection. Since we can't
do that given the current implementation, we can't build a general HTTP bridge.
A second case is the reason the unit tester doesn't deal with brokers.
In order to establish broker connections, the unit tester, acting as a
broker client, would have to ask the user to determine whether a broker
request is present, and to identify the keys which correspond to the broker
information in the incoming message if a broker request is present. This
process might or might not be able to happen before the broker timeout
if the user has to be involved.
In fact, there's an obvious alternative.
In addition to types like GAL_STRING, etc., we could introduce a new type
- call it GAL_PROXY - which encapsulates the appropriate broker information,
and, in the simplest cases, the type of the data being sent. In the best
of circumstances, the client server would establish the broker connection
and access the broker data without the programmer even having to know about
it.
The reason we haven't implemented this
yet is, frankly, because we haven't had the time. But we have been planning
ahead. As of Galaxy Communicator 3.0, we've added support for the GAL_PROXY
type in the communication protocol. But we haven't yet had a chance to
implement the APIs which would enable access to it. We're hoping to do
that very soon.
Summary
In this lesson, you've learned about the concept
and the details of broker connections, including:
In the next lesson, we'll learn about the
final and most complex type of server: one which monitors user interaction.
Next: Creating
a UI server
Please send comments and suggestions to:
bugs-darpacomm@linus.mitre.org
Last updated September 18, 2001.
Copyright (c)
1998 - 2001
The MITRE
Corporation
ALL RIGHTS RESERVED