Filtering and Transformation for High-Volume 
XML Message Brokering

Overview

People
Technology
Research Papers

Related Work

Software
 

 

 

Overview

In distributed environments, including Web Services, application integration, and personalized content delivery, XML will be used as the wire format in which data to be exchanged is encoded. In this emerging infrastructure, XML message brokers will serve as central exchange points for messages sent between applications and/or users. Three of the main functions of such brokers are: filtering, transformation, and routing

YFilter aims to provide fast, on-the-fly matching of XML encoded data to a large number of interest specifications, and transformation of the matching XML data based on recipient-specific requirements. Over the past two years, it has been developed into an efficient query processor on streaming data that high-capacity XML message brokering systems can be built on. 

Applications

We present three example applications where XML message brokering systems using YFilter can be deployed .

Personalized Content Delivery
Application Integration
Network Monitoring

Query Language

In YFilter, interest specifications are written in a subset of XQuery, an emerging XML query language. When these interest specifications enter an XML message broker, they become "standing queries". That is, they will be continuously applied to all incoming messages. 

Filtering-based queries can be written using path expressions. Path expressions are primarily used to address parts of an XML message. A path expression consists of a sequence of location steps. Each location step consists of an axis, a node test and zero or more predicates. An axis specifies the hierarchical relationship between the nodes, e.g. parent-child or ancestor-descendent relationship. A node test can be an element name or a wildcard operator '*'. Predicates can be applied to the text data, the attributes, or the positions of the addressed elements. They may also contain other path expressions (i.e. nested path expressions) to impose additional structural constraints.

Those queries that specify both filtering and transformation requirements can be written using a certain subset of FLWOR (for-let-where-order by-return) expressions provided by XQuery. The for clause and let clause together produce variable bindings, that are further filtered using the where clause, and finally cause results to be generated using order by and return clauses. All of these clauses use path expressions to address elements in an XML message.

XML Message Broker Architecture

We propose an architecture of XML message brokers built using YFilter, which is shown in the following figure. The primary inputs to a message broker are the queries that represent interest specifications and the XML messages. The output is a stream of XML messages to be delivered to users or applications whose interests are satisfied. These XML messages can be the original incoming messages or the customized messages transformed from the incoming ones based on the individual query specifications.


XML Message Broker Architecture

The major components of this architecture are: