The core concept in Service Broker is the conversation. Conversations introduce a new messaging paradigm which makes writing reliable distributed applications with Service Broker much easier than with most messaging systems. In this chapter you will learn what conversations are and how to use them to make asynchronous applications easier to write.

Conversations and Dialogs
A conversation is a reliable, ordered exchange of messages. There are two kinds of conversations defined in the Service Broker architecture:

• Dialog ¡ª A two-way conversation between exactly two endpoints. An endpoint is a source or destination for messages associated with a queue and may receive and send messages. A dialog is always established between an initiator endpoint and a target endpoint (see Figure 2.1). The initiator is just the endpoint that issues the BEGIN DIALOG command to start the dialog. Other than that, the initiator and target are peers. The first message always goes from the initiator to the target. Once the target receives the first message, any number of messages can be sent in either direction.
• Monolog ¡ª A one-way conversation between a single publisher endpoint and any number of subscriber endpoints. This is a reliable version of the popular publish-subscribe messaging paradigm. However, due to time constraints, monologs are not available in SQL Server 2005, though they will almost certainly be included in future versions.

  
  Figure 2.1 : Dialog

Because monologues aren¡¯t included in SQL Server 2005, the dialog is the only kind of conversation available and the terms conversation and dialog are essentially synonymous. This could lead to confusion, since you BEGIN a dialog and END a conversation.

Don¡¯t be confused by the inconsistent use of terms. The SQL Server 2005 development team retained the separate terms because when monologs are implemented, we will need to be able to differentiate among conversations, dialogs, and monologs.

This book uses dialog when referring to things that are unique to dialogs, and conversation when referring to characteristics common to both dialogs and monologs.

Conversations and Message Ordering
The most unique aspect of Service Broker conversations is that they maintain message order throughout the life of the conversation. Most messaging systems only ensure that messages are received in order if they are sent in a single transaction. A Service Broker conversation maintains message order across sending transactions, receiving transactions, and even multiple, parallel, receiving applications.

To see the value of message ordering, think about an application that streams large quantities of video over a network. Getting messages out of order might lead to crimes being solved before they were committed and other strange violations of cause and effect. In a banking application, getting deposits and withdrawals in the wrong order might lead to unjustified overdraft charges. If you write an application where message ordering is important, you will write the logic to ensure that the application can deal with out-of-order messages. The point is that with Service Broker, you don¡¯t have to¡ªyour application will always receive messages in order.

To preserve order, Service Broker includes a sequence number in every message it sends and forces the receiving application to receive messages in the same order they were sent. Some other messaging systems allow messages to be received in any desired order, but Service Broker enforces receive ordering to make the conversation paradigm work correctly.

Reliable Delivery
Service Broker cannot ensure that messages are received in order unless it also ensures they are all received. Missing messages will cause gaps in the sequence, which can¡¯t be allowed in a conversation. Service Broker ensures reliable message delivery by resending messages periodically until the message sender receives an acknowledgement. This resending and acknowledgement protocol is built into the Service Broker infrastructure so that the applications sending and receiving the messages aren¡¯t aware that it is happening.

Service Broker cannot ensure that messages are received in order unless it also ensures they are all received. Missing messages will cause gaps in the sequence, which can¡¯t be allowed in a conversation. Service Broker ensures reliable message delivery by resending messages periodically until the message sender receives an acknowledgement. This resending and acknowledgement protocol is built into the Service Broker infrastructure so that the applications sending and receiving the messages aren¡¯t aware that it is happening.

As shown in Figure 2.2, Service Broker messages to be sent over the network are placed temporarily in a queue called the transmission queue ( sys.transmission_queue ). Service Broker sends the message over the network and marks it as waiting for an acknowledgement. When Service Broker receives the message at the destination endpoint, it sends an acknowledgement back to the sender. When the sender receives the acknowledgement message, it deletes the message from the transmission queue.

If a message on the transmission queue waits too long for an acknowledgement, it is resent. The definition of ¡°too long¡± starts at a few seconds and increases every time the message is resent. For those of you who are bothered by the imprecision of this statement, as of the time this chapter was written, the timeout starts at 4 seconds and doubles after every retry until it reaches a maximum of 64 seconds. There are a few performance optimizations built into the protocol to minimize the number of messages exchanged, but this simplified description will suffice for now.

This system of timeouts and retries ensures that messages arrive at their destination in spite of server or network failures. Reliable message delivery means a Service Broker application can send a message and rely on the Service Broker infrastructure to make sure it gets to its destination. The application doesn¡¯t have to deal with network failures and retrying failed operations. Once again, Service Broker makes messaging applications easier to write.

Figure 2.2: Dialogs and Queues.

Symmetric Error Handling
Asynchronous applications are often hard to write because the application requesting a service and the application fulfilling the service request may never execute at the same time. This makes error handling especially hard, because it¡¯s possible that one endpoint might disappear because of an error condition without informing the other endpoint.

A Service Broker dialog always has two endpoints with two queues associated with them. This means that the Service Broker logic always knows how to inform both ends of a conversation when an error occurs. We call this aspect of conversation behavior symmetric error handling . Symmetric error handling is generally much easier to work with than other messaging systems that make the application monitor dead-letter queues for failed messages.

Conversation Persistence
The reliable, ordered messaging features of Service Broker conversations continue to function through network failures, power failures, and database restarts. To achieve this reliability, Service Broker maintains persistent information about each conversation in both endpoints of the conversation (see Figure 2.3).

Figure 2.3: Conversation Endpoints.

Look for the persisted information about open conversations in the sys.conversation_ endpoints table. If you ran the sample application in Chapter 1, you should still have a conversation open. Try this query:

use PublisherDB
select * from sys.conversation_endpoints

Most of the columns won¡¯t make sense yet. I¡¯ll explain a few columns now and cover the rest later.

• conversation_handle ¡ª This is a unique identifier that refers to the conversation in TSQL commands. This is the primary key for the row.
• conversation_id ¡ª This is the ¡°on the wire¡± identifier for a conversation. This identifier is included in each message sent across the network. Each message of this conversation has this unique identifier in its header so that the endpoint knows which conversation the message belongs to. At this point, you are probably wondering when to use the conversation_handle instead of another unique identifier. When both endpoints of a conversation are in the same database (as they were in the ISBN Lookup example in Chapter 1), there will be two rows in the table for the same conversation. Therefore, we need two handles as keys. The is_initiator flag indicates which endpoint is the initiator for the dialog.
• receive_sequence ¡ª This is the next sequence number that should be received at this endpoint.
• send_sequence ¡ª This is the next sequence number to be sent from this endpoint.
• receive_sequence_frag ¡ª Service Broker messages can be up to 2GB in size, but sending a message that big could tie up the network connection for a minute or more. To avoid monopolizing the network with a very large message, large messages are split into 40KB fragments and then assembled on the receiving end. The receive_sequence_frag field in the conversation endpoint keeps track of how many fragments of the next message have been received.

Because the message sequence numbers are tracked in a database table, message order in a conversation can be maintained through database restarts, recovery from backup, and failover¡ªeven if the conversation lasts for months or years.

Beginning and Ending Conversations
Before sending a Service Broker message, you must create a dialog conversation. An example of the command to create a simple dialog is shown in Listing 2.1:

BEGIN DIALOG CONVERSATION @conversationHandle
     FROM SERVICE [ISBNLookupResponseService]
     TO SERVICE 'ISBNLookupRequestService'
     ON CONTRACT [ISBNLookupContract]
     WITH LIFETIME = 600;

Listing 2.1: Creating a Dialog.

The @conversation_handle variable is used to return a unique identifier, which will be used to refer to the dialog in any future Service Broker statements that use this dialog.

Because dialogs always connect exactly two endpoints, you must specify both endpoints in the BEGIN DIALOG command. Endpoints are specified using service names. A service is a name for an endpoint. Services will be covered in more detail in Chapter 5.

The contract defines the message content of a dialog. Only messages specified in the definition of a contract may be sent on a dialog. Contracts are discussed in Chapter 5.

The LIFETIME clause specifies the number of seconds that this dialog is allowed to live. If a dialog still exists after the lifetime expires (5 minutes in this case), an error will be returned to any statements that attempt to access it and error messages are sent to both endpoints of the dialog. It is important to realize that the dialog doesn¡¯t go away when its lifetime expires. It remains in the sys.conversation_endpoints table in an error state until it is ended.

Dialog conversations are persistent database objects which exist until both ends of the conversation are ended. Conversations are ended with the following command:

END CONVERSATION @conversationHandle

Because both endpoints of the conversation must be ended, this command must be issued from both endpoints before the conversation ends completely.

If you look at the example in Chapter 1, you will notice that only one end of the conversation was ended. In Figure 2.3, you can see one conversation with a state of ¡°DI¡± (meaning ¡° disconnected inbound¡±), indicating that the opposite endpoint was ended with an END CONVERSATION command but this endpoint of the conversation hasn¡¯t been ended yet. The other conversation state is ¡°ER¡± (or ¡°error¡±) because it has been open long enough that the timeout has expired and the conversation ended with an error. If you execute an END CONVERSATION command specifying the conversation_handle for the row in the sys.conversation_endpoints table, it will go away.

Remembering to end all conversations is a very important programming practice. If you forget to do this, you will find thousands of rows in the sys.conversation_endpoints table after your application has run for awhile.

On the other hand, it is important not to end the conversation until you are done with it. The rich ordering features of conversations are not very useful if you start a new dialog for every message. After all, any messaging system can process a single message in order. To use Service Broker to its full potential, be sure to keep conversations open for as long as necessary.

Summary
Conversations are one of the fundamental concepts of Service Broker. There are two types of conversations (monologs and dialogs), but only dialogs are included in SQL Server 2005. A dialog is a reliable, two-way, ordered, persistent exchange of messages between two endpoints. The fundamental communications primitive for most messaging systems is a message. For Service Broker, the messaging primitive is the dialog.

discuss this topic to forum