Galaxy Communicator Tutorial:

Setting Up a Brokered Audio Connection


Up to this point, all the communication between servers has been mediated by the Hub. This arrangement has its advantages: for instance, the Hub provides a central location for flow of control and routing. However, it also has its disadvantages: in order for a message to get from one server to another, it has to be decoded, analyzed, and reconstructed in the Hub. While this process appears to be quite efficient, there are circumstances in which you might prefer to send data directly: for example, in situations involving high bandwidth or streaming, such as audio input or output.

To support this alternative, the Galaxy Communicator infrastructure supports a peer-to-peer connection called a broker. The Hub mediates the establishment of this connection, but all data which flows through the connection goes directly from server to server. In this lesson, we'll learn how to set up a broker connection and how it works.

Once you feel comfortable with this lesson, you can consult the broker reference documentation.


Recognizer, Synthesizer and Audio

Up to this point, we've examined four servers in the toy travel demo: the Parser, Backend and Generator servers, which represent the simplest type of server, and the Dialogue server, which represents the next level of complexity. The first three servers simply send responses to dispatch functions, while the Dialogue server sends new messages to the Hub and occasionally waits for the response. The last three servers take advantage of all these tools, and furthermore interact with each other via broker connections.

In addition, the Audio server also exhibits the complexities of UI programming, which we'll explore in the next lesson.


Brokering in four steps

The setup and operation of brokers can be broken down into four steps. We'll describe and illustrate these steps briefly, and then look at some code. We'll focus on the interaction between the Synthesizer and Audio servers as an example.

Step 1: Establish the broker listener

In the first step, the server which will be the source of the broker data establishes a listener for the broker. This listener, in most cases, is the same listener that's listening for connections from the Hub; the protocol is constructed so that the listener can distinguish between the two types of connections. The server assigns each broker a unique ID, so that the server can listen for multiple brokers simultaneously. The server also assigns each broker a timeout; it will accept multiple connections for the broker, until the timeout is reached.

Step 2: Notify the Hub and client

In the second step, the source server sends a new message to the Hub which contains the broker contact information. This information consists of the host, port and unique ID associated with this particular broker. This message is a message like any other, and can contain additional information (such as encoding format and sample rate for audio).

Step 3: Establish the broker connection, transmit data

When the target server receives this information, it establishes a connection to the source server using the host, port and unique ID, and establishes a callback to receive the data. The source server can now send data over the connection, and the callback will be invoked in the target server whenever data is received.

Step 4: Process data, shut down broker listener

Finally, when the source server is done sending data, it notifies the target server that it's done, and the broker connection is terminated. The target server does whatever it wants to do with the data (e.g., send audio to the audio device). When the source server's broker reaches its timeout, it stops accepting connections for that broker; when all the connections for the broker are done, the source server shuts down the broker.
Now that we've seen these steps in the abstract, let's take a look at each one of them in detail.


Brokering details

Source server: broker setup, message dispatch

In the first and second brokering steps, the source server sets up a broker listener and sends a message to the Hub announcing the broker. Here's how that happens.

There are three functions involved. After the message is populated with the appropriate brokering information, it is sent to the Hub and freed, just like any other message.

Hub: broker client notification

The Hub portion of the process is the simplest, and you can probably guess what it looks like:
PROGRAM: FromSynthesizer

RULE: :host & :port & :call_id --> Audio.Play
IN: :host :port :encoding_format :sample_rate :call_id

The Hub matches the incoming message with the appropriate program, shown here, and checks the rule. All three keys are present in the token: :host and :port were specified in the call to GalIO_BrokerPopulateFrame, and the :call_id key is the key which GalIO_BrokerPopulateFrame always uses for the unique broker ID. So the rule fires, and the relevant data is sent to the Audio.Play dispatch function.

Target server: set up broker client, establish callback

When the Audio server invokes the Audio.Play dispatch function, it creates a broker client, as follows:

There are three functions involved in setting up the client correctly. At this point, the source and target servers have established a broker connection, and the broker client can begin to consume data.

About broker data

When we discussed frames and objects, we talked about the object types that could appear in frames, and concentrated on frames, strings, integers and lists. We suggested at that point that there were many other datatypes which could appear in frames, including arrays of integers of various sizes. Any type that can appear in a frame can be sent across a broker connection, and vice versa, but we won't be looking at frames, string, integers and lists in our broker example; we'll be looking at arrays of integers.

When data is written to a broker server, it is encoded and cached in an output queue. If any broker clients are currently connected, it is also written to the client; and when a new client connects, the server transmits the contents of the output queue to the new client. In other words, all clients which connect to the broker server are guaranteed to see all the same data in the same order, no matter when they connect. This is one of the reasons that broker servers are set up to expire; if a broker server didn't expire, it would never be possible to free its output queue.

The two ends of the broker connection

On the source server side, the server writes data to the broker, and then indicates that it's done, approximately as follows:
  void *data;
  int num_samples;
  GalIO_BrokerStruct *b;
  SynthesisRecord *s;

  /* ... */

  data = PollSynthesis(s, &num_samples);

  while (data) {
    GalIO_BrokerWriteInt16(b, data, num_samples);
    free(data);
    data = PollSynthesis(s, &num_samples);
  }
  if (SynthesisIsDone(s)) {
    GalIO_BrokerDataOutDone(b);
  /* ... */

There are two functions involved in this example. On the target server side, the data callback function is invoked each time a block of data is received (each block corresponds to a single call to one of the GalIO_BrokerWrite* functions). The data callback function looks approximately like this:
static void __AudioOutputCallback(GalIO_BrokerStruct *b,
                                  void *data,
                                  Gal_ObjectType data_type,
                                  int n_samples)
{
  /* ... */

  switch(data_type) {
  case GAL_INT_16:
    printf("[Audio data to user (%d samples)]\n", n_samples);
    /* Do something with the audio */
    free(data);
    break;
  default:
    GalUtil_Warn("__AudioOutputCallback: unexpected data type %s\n",
                 Gal_ObjectTypeString(data_type));
    free(data);
    break;
  }
}

Each data callback function has the same signature. The first argument is the broker client. The second argument is a pointer to the data, which is interpreted according to the data type in the third argument. So, for instance, if the data type is GAL_FRAME, then the data is really a Gal_Frame structure; if the data type is GAL_STRING, the data is really a char *. In this case, all the data which the source server wrote is an array of 16-bit integers; the corresponding type is GAL_INT_16. The fourth object is interesting only for the array and list types; it's the number of elements in the array or list.

So when the target server receives a block of data, it invokes the data callback, which decides what to do with the data based on the data type. The data is allocated as new memory before it's passed to the data callback function, so once it's done with the data, it frees it. This happens for each block of data (that is, each GalIO_BrokerWrite* call on the source side), until the broker client receives the termination message, at which point it will do whatever it's supposed to do when it's done (see the call to GalIO_AddBrokerCallback here), and terminate.

An example

Let's watch a broker in action. This is slightly complicated by the fact that the unit tester cannot host broker servers or clients. So we need to use the unit tester, acting as a server, to send a message to the Hub which will cause the Synthesizer server to produce a broker connection. This is actually pretty simple; we'll just send a Synthesizer.Synthesize message to the Hub, and populate its program file with the program we saw here, to handle the new message from the Synthesizer server.
[Brokering exercise 1]

% process_monitor $GC_HOME/tutorial/brokering/synth.config

Select "Process Control --> Restart all", then select "Send new message" in the unit tester. Select the message named Synthesizer.Synthesize, and press OK. You'll see the result in the Audio pane:
[Audio client pane]

[Audio data to user (1024 samples)]
[Audio data to user (1024 samples)]
[Audio data to user (1024 samples)]
[Audio data to user (1024 samples)]
[Audio data to user (1024 samples)]
[Audio data to user (1024 samples)]
[Audio data to user (1024 samples)]
[Audio data to user (1024 samples)]
[Audio data to user (1024 samples)]
[Audio data to user (1024 samples)]
[Audio data to user (1024 samples)]
[Audio data to user (1024 samples)]
[Audio data to user (1024 samples)]
[Audio data to user (1024 samples)]
[Audio data to user (184 samples)]
[Audio data to user is finalized (14520 samples).]

Each of these printouts corresponds to a single call to the data callback function. The Hub shows how the information was conveyed from the source server to the target server:
[Hub pane]

------ find next op ----------------------
{c FromSynthesizer
   :host "129.83.10.107"
   :port 15500
   :call_id "129.83.10.107:24405:0"
   :sample_rate 8000
   :encoding_format "linear16"
   :session_id "Default"
   :tidx 3 }
--------------------------------------------

found operation: Audio.Play
---- serve(Audio@<remote>:-1, token 3 op_name Play (sole provider))

Got reply from provider for Audio (id 2) : token 3

----------------[  3]----------------------
{c FromSynthesizer
   :host "129.83.10.107"
   :port 15500
   :call_id "129.83.10.107:24405:0"
   :sample_rate 8000
   :encoding_format "linear16"
   :session_id "Default"
   :tidx 3 }
----------------------------------------

        [Destroying token 3]

The FromSynthesizer message constructed by the Synthesizer server contains the host, port and unique broker ID, and the program shown here sends the appropriate message to the Audio server. Finally, the Synthesizer pane shows the evidence of the broker client connecting, the unique ID matching, and the transmission of the special conclusion message:
[Synthesizer pane]

__GalIO_ServerContactHandler: connected to 129.83.10.107 (sock=6)
_GalIO_BrokerListenerHandshakeHandler: reference frame matched
GalIO_BrokerDataOutCallbackHandler: done sending data.

Select "File --> Quit" to end this exercise.


Why the dependency on :call_id?

At this point, you may be a little puzzled about why this mechanism is so complex. In particular, there are three keys which must be sent from source to target server, and one of the keys (:call_id) is a "magic" value. In addition, there's no way to tell that there's a broker request being passed around; nothing guarantees that the :call_id key is used only for brokers. Finally, the mechanism seems too flexible for the normal case; while brokers support sending arbitrary data of different types through a broker connection, in most circumstances all the data will be the same type.

There are at least two circumstances in which this is not just inelegant, but a real implementational problem. For example, let's say we want to define an HTTP bridge which would allow us to transmit Communicator interactions through a firewall over HTTP. In order to do this in a general way, we'd have to be able to identify broker requests automatically, because the broker data would have to be consumed and encoded over the same HTTP connection. Since we can't do that given the current implementation, we can't build a general HTTP bridge. A second case is the reason the unit tester doesn't deal with brokers. In order to establish broker connections, the unit tester, acting as a broker client, would have to ask the user to determine whether a broker request is present, and to identify the keys which correspond to the broker information in the incoming message if a broker request is present. This process might or might not be able to happen before the broker timeout if the user has to be involved.

In fact, there's an obvious alternative. In addition to types like GAL_STRING, etc., we could introduce a new type - call it GAL_PROXY - which encapsulates the appropriate broker information, and, in the simplest cases, the type of the data being sent. In the best of circumstances, the client server would establish the broker connection and access the broker data without the programmer even having to know about it.

The reason we haven't implemented this yet is, frankly, because we haven't had the time. But we have been planning ahead. As of Galaxy Communicator 3.0, we've added support for the GAL_PROXY type in the communication protocol. But we haven't yet had a chance to implement the APIs which would enable access to it. We're hoping to do that very soon.


Summary

In this lesson, you've learned about the concept and the details of broker connections, including: In the next lesson, we'll learn about the final and most complex type of server: one which monitors user interaction.

Next: Creating a UI server


Please send comments and suggestions to: bugs-darpacomm@linus.mitre.org
Last updated September 18, 2001.

Copyright (c) 1998 - 2001
The MITRE Corporation
ALL RIGHTS RESERVED