Archive for the ‘Tech Blogs’ Category

Medium Posts

We have started using medium for our blogging. Please click here to see our latest posts.

XMPP Packet capture in NoSQL DataBase (BaseX)

BaseX Blog

Introduction

This ‘Blurt/Blog’ details my experiences of a recent telephony & messaging project which made use of the following technologies: BaseX database, XQJ, XQuery, Java & Spring, JAXB, Hibernate, XMPP, Openfire server.

Context

Here at Globility Ltd. we specialise in designing, developing and providing integrated communication services. I recently received an interesting client requirement to capture all XMPP (also called Jabber) packets sent or received as part of a telephony & messaging ecosystem. The packets had to be stored in their original form (for legal reasons). Naturally, the client then requested the facility to search/query this data; with output intelligently summarised into a single line of status indicators (multiple packets making up the lifecycle of a single transaction e.g. multi-party conference call). This requirement would form the heart of the clients missing piece in their audit/compliance system.

From the outset, design decisions were made to ensure the DataLogger solution would be flexible enough to capture many forms of data, and to be easily extended to query it. That would mean it could be used in many situation where the exchange of high-frequency data needs to be captured and queried/analysed.

The Server

An XMPP server would be required to provide basic messaging, presence, and routing services. My experiences with the open source server ‘Openfire’ settled the argument of which one to use (also has a pubsub service XEP-0060: Publish-Subscribe). The Openfire ‘plugin’ architecture allows you to extend and enhance the functionality of the server, so I decided to write a ‘plugin’ (leveraging the ‘out of the box’ XMPP services) which would be the main engine of the overall solution.

Packet Format (XML vs JSON)

The XMPP packets (org.xmpp.packet.Packet) had to be in XML (for existing upstream & downstream systems). JSON would have been the obvious alternative (XEP-0295: JSON Encodings for XMPP). Design decisions were made to accommodate a possible move to JSON in the future (general consensus is a move away from XML towards JSON, for most situations).

Storing Packets

I wanted to store the XMPP XML packets (of type org.xmpp.packet.Message & org.xmpp.packet.IQ) preserving their original form, and narrowed my choice down to 2 open source XML db contenders: BaseX (http://basex.org) & eXist-db (http://exist-db.org).
I settled on BaseX for various reasons:

  1. BSD license (imposing minimal restrictions on the redistribution of covered software)
  2. Allows storage of XML and JSON.
  3. Performs well with large data sets (https://mailman.uni-konstanz.de/pipermail/basex-talk/attachments/20100822/5b4d114d/attachment-0001.pdf).
  4. Has a scalable architecture allowing for future growth
  5. Has a very useful GUI (BaseX) which serves to visualise and administer the data
  6. Flexibility in modes of operation (embedded, standalone, client/server – supporting ACID safe transactions)
  7. Full set of relevant interfacing (XQuery, XQJ – Java API to execute XQuery against an XML db, REST/RESTXQ, WebDav)
  8. Lightweight, so I can have it running locally on my MAC while developing (and an Ubuntu build for production)

A major decision effecting storage size, performance and querying was should I store each packet as a separate XML document or store them into a single giant XML document. I chose the separate document per packet approach after consideration to how the data would be potentially queried.
Another major decision was should I use a standard Java interface or native BaseX API calls to handle XML DataSources. I wanted to be as database-agnostic so opted to use XQJ (http://xqj.net/). The XQJ API is to XML Databases as the JDBC API is to Relational Databases.

DataLogger Plugin

I’ve summarised the design/code of the DataLogger Openfire plugin:

  1. To create the DataLogger Plugin, Implement the org.jivesoftware.openfire.container.Plugin interface, and in the overridden initializePlugin() method, create the DataLoggerComponent that extends org.xmpp.component.AbstractComponent:

 

  1. The DataLogger Plugin (or better coding style, some helper class) has to implement the org.jivesoftware.openfire.interceptor.PacketInterceptor ( and override interceptPacket() method) to store org.xmpp.packet.IQ packets into BaseX:

 

  1. The DataLoggerComponent needs to override appropriate handlers. It must also discover all pubsub nodes on the Openfire server and subscribe to them (in order to catch org.xmpp.packet.Message packets). Finally, it need to register AdHocCommands (basis of the data querying) with the AdHocCommandManager:

@Component
public class DataLoggerComponent extends AbstractComponent implements GtmsComponent {

@Autowired
public DataLoggerComponent(DataLoggerDBParams dataLoggerDBParams) {

}
@Override
public String getDescription() {
return “Data Logger API Component”;
}
@Override
public String getName() {
return “datalogger”;
}
@Override
public String getDomain() {
return XMPPServer.getInstance().getServerInfo().getXMPPDomain();
}
@Override
protected String[] discoInfoFeatureNamespaces() {
return new String[] { DISCO_ITEMS, DATA_FORMS, GlobalConstants. ADHOC_COMMANDS,
DATALOGGER_NAMESPACE };
}
@Autowired
public void setAdHocCommandManager(AdHocCommandManager adHocCommandManager) {
this.adHocCommandManager = adHocCommandManager;
}
@Override
public IQ handleDiscoItems(IQ iq) {

}
@Override
protected IQ handleDiscoInfo(IQ iq) {

}
@Override
public IQ handleIQGet(IQ iq) {
return this.handleIQPacket(iq);
}
@Override
public IQ handleIQSet(IQ iq) {
return this.handleIQPacket(iq);
}
@Override
protected void handleIQError(IQ iq) {
this.log.debug(“IQ ERROR: ” + iq.toXML());
}
@Override
protected void handleIQResult(IQ iq) {
this.log.debug(“IQ RESULT: ” + iq.toXML());
}
private IQ handleIQPacket(IQ iq) {

}
@Override
protected void handleMessage(Message message) {
XQDataSource ds = null;
XQConnection2 xqc = null;
try {
ds = new BaseXXQDataSource();
ds.setProperty(“serverName”, this.dataLoggerDBParams.getServerName());
ds.setProperty(“port”, this.dataLoggerDBParams.getPort());
ds.setProperty(“user”, this.dataLoggerDBParams.getUser());
ds.setProperty(“password”, this.dataLoggerDBParams.getPassword());
ds.setProperty(“databaseName”, this.dataLoggerDBParams.getDatabaseName());
xqc = (XQConnection2)ds.getConnection();
XQItem item = xqc.createItemFromDocument(message.toString(), null, null);

}
class MyIQResultListener implements IQResultListener {
@Override
public void receivedAnswer(IQ iq) {
Element inIq = iq.getChildElement();
if (iq.getType() == IQ.Type.result) {
if (inIq.getNamespaceURI().equals(“http://jabber.org/protocol/disco#items”)) {
allFirstLevelNodes = getNodesFrom(iq);
}
else if (inIq.getNamespaceURI().equals(“http://jabber.org/protocol/pubsub”)) {
Element inPubSub = inIq.element(“subscriptions”);
if (inPubSub != null) {
//this is a result from a get all subscriptions request
allSubscriptions = getSubscriptionsFrom(iq);
}
else {
inPubSub = inIq.element(“subscription”);
if (inPubSub != null) {
//this is a result from a subscription request
String node = inPubSub.attributeValue(“node”);
String subscriptionStatus = inPubSub.attributeValue(“subscription”);
if (subscriptionStatus != null && subscriptionStatus.equals(“subscribed”)) {
allSubscriptions.add(node);
}
}
}
}
}
}
@Override
public void answerTimeout(String packetId) {
log.debug(“IQResultListener timed out, packetId: ” + packetId);
// notificationServiceOnline = false;
}
}
/**
* Extracts a list of Nodes from an IQ packet.
* @param packet the packet
* @return a list of Nodes
*/
private static Collection<String> getNodesFrom(IQ packet) {
Collection<String> nodes = new ArrayList<String>();
Element query = packet.getChildElement();
Iterator<Element> itemsIterator = query.elementIterator(“item”);
if (itemsIterator != null) {
while (itemsIterator.hasNext()) {
Element itemElement = itemsIterator.next();
String node = itemElement.attributeValue(“node”);
nodes.add(node.toString());
}
return nodes;
}
return null;
}
/**
* Extracts a list of Subscriptions from an IQ packet.
* @param packet the packet
* @return a list of Nodes
*/
private static Collection<String> getSubscriptionsFrom(IQ packet) {
Collection<String> subscriptions = new ArrayList<String>();
Element childElement = packet.getChildElement();
Element subscriptionsElement = childElement.element(“subscriptions”);
Iterator<Element> itemsIterator = subscriptionsElement.elementIterator(“subscription”);
if (itemsIterator != null) {
while (itemsIterator.hasNext()) {
Element itemElement = itemsIterator.next();
if (itemElement.attributeValue(“subscription”).equals(“subscribed”)) {
String node = itemElement.attributeValue(“node”);
subscriptions.add(node.toString());
}
}
return subscriptions;
}
return null;
}
@Override
public void postComponentStart() {
// enable profiler
JiveGlobals.setProperty(DataLoggerConstants.Properties.ENABLED, “TRUE”);
this.dataLoggerPubSubManager = new DataLoggerPubSubManager(this);
try {
requestAllPubSubNodes();
requestAllPubSubSubscriptions();
Collection<String> nodesNotSubscribed = difference(this.allFirstLevelNodes, this.allSubscriptions);
requestSubscriptions(nodesNotSubscribed);
this.adHocCommandManager.addCommand(new SearchDataLoggerDBCallSummary(this, this.dataLoggerDBParams));
this.adHocCommandManager.addCommand(new SearchDataLoggerDBMsgSummary(this, this.dataLoggerDBParams));
this.adHocCommandManager.addCommand(new SearchDataLoggerDBVBlastSummary(this, this.dataLoggerDBParams));
} catch (Exception e) {
log.error(GtmsLog.exceptionToString(e));
}
}
private Collection<String> difference(Collection a, Collection b) {
if (b == null || a == null) {
return a;
}
Collection diff = CollectionUtils.subtract(a, b);
return diff;
}
private void requestAllPubSubNodes() {
log.debug(“Requesting all first level nodes known to pubsub service”);
IQ iq1 = PubSubHelper.buildPubSubAllFirstLevelNodes(getJID().toString(), this.getDomain());
IQRouter iqRouter = XMPPServer.getInstance().getIQRouter();
XMPPServer.getInstance().getIQRouter().addIQResultListener(iq1.getID(), new MyIQResultListener());
iqRouter.route(iq1);
log.debug(“Sent request: ” + iq1.toXML());
}
private void requestAllPubSubSubscriptions() {
log.debug(“Requesting all datalogger pubsub subscriptions”);
IQ iq1 = PubSubHelper.buildPubSubAllSubscriptions(getJID().toString(), this.getDomain());
IQRouter iqRouter = XMPPServer.getInstance().getIQRouter();
XMPPServer.getInstance().getIQRouter().addIQResultListener(iq1.getID(), new MyIQResultListener());
iqRouter.route(iq1);
log.debug(“Sent request: ” + iq1.toXML());
}
private void requestSubscriptions(Collection<String> nodes) {
log.debug(“Requesting subscriptions to all first level unsubscribed nodes”);
IQ iq1 = null;
for (String node : nodes) {
iq1 = PubSubHelper.buildPubSubNodeSubscription(getJID().toString(), this.getDomain(), node);
IQRouter iqRouter = XMPPServer.getInstance().getIQRouter();
XMPPServer.getInstance().getIQRouter().addIQResultListener(iq1.getID(), new MyIQResultListener());
iqRouter.route(iq1);
log.debug(“Sent request: ” + iq1.toXML());
}
}
@Override
public void sendPacket(Packet packet) throws ComponentException {
this.compMan.sendPacket(this, packet);
}
@Override
public void postComponentShutdown() {

}

}

  1. In point 3) above, pay close attention to the overridden handleMessage() method. It stores org.xmpp.packet.Message packets into BaseX:

@Override
protected void handleMessage(Message message) {
XQDataSource ds = null;
XQConnection2 xqc = null;
try {
ds = new BaseXXQDataSource();
ds.setProperty(“serverName”, this.dataLoggerDBParams.getServerName());
ds.setProperty(“port”, this.dataLoggerDBParams.getPort());
ds.setProperty(“user”, this.dataLoggerDBParams.getUser());
ds.setProperty(“password”, this.dataLoggerDBParams.getPassword());
ds.setProperty(“databaseName”, this.dataLoggerDBParams.getDatabaseName());
xqc = (XQConnection2)ds.getConnection();
XQItem item = xqc.createItemFromDocument(message.toString(), null, null);
….

  1. Use multi-stage Data Forms (XEP-0004: Data Forms) as a generic way of querying the stored XML data. Any client that wants to query the data, can use them to pass in query criteria and thus, generate customisable queries. This is achieved by extending org.jivesoftware.openfire.commands.AdHocCommand (and overriding the addStageInformation(), execute() methods):

public class SearchDataLoggerDBCallSummary extends DataLoggerAdHocCommand {

@Override
protected void addStageInformation(SessionData data, Element command) {
DataForm form = new DataForm(DataForm.Type.form);
form.setTitle(“DataLogger Search: Call Summary”);
form.addInstruction(“Form to query Datalogger Database to bring back summary of calls.”);
FormField field = form.addField();
field.setType(FormField.Type.hidden);
field.setVariable(“FORM_TYPE”);
field.addValue(DATALOGGER_NAMESPACE);
field = form.addField();
field.setType(FormField.Type.text_single);
field.setLabel(“Profile (User/Group)”);
field.setVariable(“profile”);
field.setDescription(“The User or Group Profile”);
field = form.addField();
field.setType(FormField.Type.text_single);
field.setLabel(“History From (milliseconds)”);
field.setVariable(“timestamp”);
field.setDescription(“Call history from this point in time”);
field.setRequired(true);
field = form.addField();
field.setType(FormField.Type.list_single);
field.setLabel(“Sort by”);
field.setVariable(“sortby”);
field.setDescription(“Sort returned data”);
field.addValue(“number($distinct-id)”);
field.addOption(“Call ID”, “number($distinct-id)”);
field.addOption(“Timestamp”, “number($start-time3)”);
field.addOption(“State”, “$state”);
field.addOption(“Destination”, “number($called[1])”);
field.addOption(“Direction”, “$direction”);
field.addOption(“Source”, “number($caller[1])”);
field.addOption(“Duration”, “number($duration)”);
field.setRequired(true);

f// Add the form to the command
command.add(form.getElement());
}
@Override
public void execute(SessionData data, Element command) {
String profile = getValueFromData(data, “profile”);
String timestamp = getValueFromData(data, “timestamp”);
String sortby = getValueFromData(data, “sortby”);
String ascdesc = getValueFromData(data, “ascdesc”);

XQDataSource ds = null;
XQConnection2 xqc = null;
try {
ds = new BaseXXQDataSource();
ds.setProperty(“serverName”, this.dataLoggerDBParams.getServerName());
ds.setProperty(“port”, this.dataLoggerDBParams.getPort());
ds.setProperty(“user”, this.dataLoggerDBParams.getUser());
ds.setProperty(“password”, this.dataLoggerDBParams.getPassword());
ds.setProperty(“databaseName”, this.dataLoggerDBParams.getDatabaseName());
xqc = (XQConnection2)ds.getConnection();
//This code queries the xml db and prints out results

XQPreparedExpression xqp = xqc.prepareExpression(xqueryString);
// Bind a variable and execute the query
xqp.bindDouble(new QName(“start”), start, null);
xqp.bindDouble(new QName(“max”), max, null);
XQResultSequence rs = xqp.executeQuery();
while(rs.next()) {
String str = rs.getItemAsString(null);

}
}
FormField field = form.addField();
field.setLabel(DATA[8]);
field.setVariable(DATA[8]);
field.setDescription(“The size of returned result set”);
field.addValue(resCount);
xqc.close();
command.add(form.getElement());
note.addAttribute(“type”, “info”);
note.setText(“Operation finished successfully”);
logger.debug(“SearchDataLoggerDBCallSummary: End”);
} catch (Exception e) {
logger.error(GtmsLog.exceptionToString(e));
note.addAttribute(“type”, “error”);
note.setText(e.getMessage());
} finally {
if ((xqc != null) && (!xqc.isClosed())) {
try {
xqc.close();
} catch (Exception ex) {
logger.error(GtmsLog.exceptionToString(ex));
}
}
}
}
@Override
protected List<Action> getActions(SessionData data) {
return Arrays.asList(AdHocCommand.Action.complete);
}
@Override
public String getCode() {
return DataLoggerConstants.DATALOGGER_NAMESPACE + “#search-datalogger-callsummary”;
}
@Override
public String getDefaultLabel() {
return “Search datalogger call summary”;
}
@Override
protected Action getExecuteAction(SessionData data) {
return AdHocCommand.Action.complete;
}
@Override
public int getMaxStages(SessionData data) {
return 1;
}
}

  1. The following is an example of the multi-stage Data Form request & response:

————————————————————————
——————Execute the search command search-datalogger-callsummary (multi-stage)
————————————————————————
<iq from=”…” to=”datalogger…”
type=”set” id=”search1″>
<command xmlns=”http://jabber.org/protocol/commands”
node=”http://gltd.net/protocol/datalogger:01:00:00#search-datalogger-callsummary”
action=”execute” />
</iq>
—————–Server returns a new stream form. Required values are
——-marked and can be of type text-single, boolean, etc.
——-Recommend looking at Data Forms spec to see all possible types
<iq type=”result” id=”search1″ from=”datalogger…” to=”harrytest5@…”>
<command xmlns=”http://jabber.org/protocol/commands” sessionid=”…”
node=”http://gltd.net/protocol/datalogger:01:00:00#search-datalogger-callsummary”
status=”executing”>
<x xmlns=”jabber:x:data” type=”form”>
<title>DataLogger Search: Call Summary</title>
<instructions>Fill out this form to request a search of the
Datalogger Database to bring back a summary of calls.
</instructions>
<field type=”hidden” var=”FORM_TYPE”>
<value>http://gltd.net/protocol/datalogger:01:00:00</value>
</field>
<field type=”text-single” label=”Profile (User/Group)” var=”profile”>
<desc>The User or Group Profile</desc>
</field>
<field type=”text-single” label=”History From (milliseconds)”
var=”timestamp”>
<desc>Call history from this point in time</desc>
<required />
</field>
<field type=”list-single” label=”Sort by” var=”sortby”>
<desc>Sort returned data</desc>
<value>$distinct-id</value>
<option label=”Call ID”>
<value>$distinct-id</value>
</option>
<option label=”Timestamp”>
<value>$start-time3</value>
</option>
<option label=”State”>
<value>$state</value>
</option>
<option label=”Destination”>
<value>$called[1]</value>
</option>
<option label=”Direction”>
<value>$$direction</value>
</option>
<option label=”Source”>
<value>$caller[1]</value>
</option>
<required />
</field> </x>
<actions execute=”complete”>
<complete />
</actions>
</command>
</iq>
The second stage of the multi-stage Data Form would be:
Client Takes above result from 1st stage and produces the ‘set’ request below—-
<iq type=”set” id=”search2″ from=”…” to=”datalogger…”>
<command xmlns=”http://jabber.org/protocol/commands”
node=”http://gltd.net/protocol/datalogger:01:00:00#search-datalogger-callsummary”
sessionid=”…”>
<x xmlns=’jabber:x:data’ type=’submit’>
<field type=”hidden” var=”FORM_TYPE”>
<value>http://gltd.net/protocol/datalogger:01:00:00</value>
</field>
<field type=”text-single” label=”Profile (User/Group)” var=”profile”>
<value>gary_office…</value>
</field>
<field type=”text-single” label=”History From (milliseconds)” var=”timestamp”>
<value>1486790948174</value>
</field>
<field type=”list-single” label=”Sort by” var=”sortby”>
<value>$distinct-id</value>
</field>
</x>
</command>
</iq>
—————–Result from above request
<iq type=”result” id=”search2″ from=”datalogger.sirius.gltd.local”
to=”harrytest5@sirius.gltd.local/Smack”>
<command xmlns=”http://jabber.org/protocol/commands” sessionid=”xJQROWl15FO47te”
node=”http://gltd.net/protocol/datalogger:01:00:00#search-datalogger-callsummary”
status=”completed”>
<note type=”info”>Operation finished successfully</note>
<x xmlns=”jabber:x:data” type=”result”>
<reported>
<field var=”callid” type=”text-single” label=”callid” />
<field var=”count” type=”text-single” label=”count” />
<field var=”state” type=”list-single” label=”state” />
<field var=”direction” type=”text-single” label=”direction” />
<field var=”called” type=”text-single” label=”called” />
<field var=”caller” type=”text-single” label=”caller” />
<field var=”duration” type=”text-single” label=”duration” />
<field var=”timestamp” type=”text-single” label=”timestamp” />
<field var=”resultcount” type=”text-single” label=”resultcount” />
</reported>
<item>
<field var=”timestamp”>
<value>1385646662893</value>
</field>
<field var=”caller”>
<value>6004</value>
</field>
<field var=”duration”>
<value>14</value>
</field>
<field var=”count”>
<value>4</value>
</field>
<field var=”callid”>
<value>1385646172807</value>
</field>
<field var=”direction”>
<value>Outgoing</value>
</field>
<field var=”called”>
<value>770371</value>
</field>
<field var=”state”>
<value>Answered</value>
</field>
</item>
<item>
<field var=”timestamp”>
<value>1385646692313</value>
</field>
<field var=”caller”>
<value>6004</value>
</field>
<field var=”duration”>
<value>6</value>
</field>
<field var=”count”>
<value>4</value>
</field>
<field var=”callid”>
<value>1385646172808</value>
</field>
<field var=”direction”>
<value>Outgoing</value>
</field>
<field var=”called”>
<value>771338</value>
</field>
<field var=”state”>
<value>Answered</value>
</field>
</item>

<field label=”resultcount” var=”resultcount”>
<desc>The size of returned result set</desc>
<value>36</value>
</field>
</x>
</command>
</iq>

  1. A Proof of concept for unmarshalling/marshalling using JAXB was successfully completed. This will be incorporated into a future version of the DataLogger Plugin

NOTE: Belatedly, I realised that packets were being duplicated, as the pubsub Message packets were also coming in as IQ packets. I eventually switched over to using just the PacketInterceptor to capture all packets without duplication.

Development Environment (MacBook Pro)

  1. Eclipse-Kepler (Java 6, Spring, Openfire with HSQL database, Hibernate, BaseX, XQJ, JAXB)
  2. BaseXserver
  3. BaseXGUI (great for testing/running XQuery code, ). NOTE: Ensure you are not logged into basex db through the basexclient or the basexgui, otherwise you will get the following error: java.sql.SQLException: The database is already in use by another process.
  4. Git/BitBucket
  5. Grinder for testing (performance & scalability). Manual testing during development was done using a smack based XMPP client to inject Message & IQ packets
  6. Jenkins – Continuous build/test

Architecture Diagram

 

iCosts – The Legal Costs Calculator – Demo

iCosts – The Legal Costs Calculator – Demo

Lync XMPP + Video – Demo

Lync XMPP + Video – Demo

Fuze/GMTS Chat Integration Demo

Fuze/GMTS Chat Integration Demo

Lync-Openfire CSTA Plugin

CSTA Plugin Configuration

 

 

This post will go through the steps required to set up Remote Call Control between a Lync Server, and telephony plugins sitting on an Openfire server.

 

This uses our “CSTA Plugin” for Openfire.

 

 

1. Static Route

 

The first step is to configure a Static Route on the Lync server which goes to the Openfire server. This is done from the Lync Server Management Shell.

 

First, you need to define the route as a variable:

$1= New-CsStaticRoute -TCPRoute -Destination <ip of openfire server> -Port <csta sip port> -MatchUri <fqdn of the openfire server>

 

Now you need to add this variable as a Route:

Set-CsStaticRoutingConfiguration -Identity global -Route @{Add=$1}

 

The “$1″ represents the variable name used above.

 

You can run the below command to check the routes:

Get-CsStaticRoutingConfiguration

 

This will return all identities and routes configured for them.

 

 

2. Trusted Application

 

The next step is to configure the Openfire server as a Trusted Application Server.

 

Once again, this is done from the Lync Server Management Shell.

 

You first need to create a Trusted Application Pool:

New-CsTrustedApplicationPool -Identity <ip of openfire server> -Registrar <fqdn of lync server> -ComputerFqdn <ip of openfire server> -Site <site name> -TreatAsAuthenticated $true -ThrottleAsServer $true

 

Now that the pool is made, the Trusted Application must be set with this pool:

New-CsTrustedApplication -ApplicationId <any ID> -TrustedApplicationPoolFqdn <ip of openfire server> -Port <csta sip port> -EnableTcp

Set-CsTrustedApplicationPool -Identity <ip of openfire server> -OutboundOnly $True

 

Now that everything is set, you must enable the topology:

Enable-CsTopology

 

 

3. Topology Builder

 

After configuring Openfire as a Trusted Application Server, you need to make a change to it in the Topology Builder.

 

To do this, run the Lync Server Topology Builder.

 

When prompted, select “Download Topology from existing deployment”.

1. Topology Builder - Download existing

 

 

This will begin to download the topology.

 

When asked where to download the topology, you can select any location and file name, but do not change the format.

2. Topology Builder - Save location

 

Once chosen, select “Save”.

 

This will load you current deployment. Collapse the left side menu to find the folder “Trusted application servers”, which should have the just created Trusted Application Server listed.

3. Topology Builder - Trusted Application Servers

 

Right click the listing and select “Edit Properties…”.

4. Topology Builder - Edit Proprties

 

Here you need check the box “Limit service usage to selected IP addresses” and enter the IP of the Openfire Server in the field under “Primary IP address: *”.

5. Topology Builder - Primary IP

 

Once done select OK.

 

Now you need to publish back the Topology with the changes.

 

In the Topology Builder, click the “Action” menu and select “Publish Topology…”.

6. Toploogy Builder - Publish Topology

 

Click “Next” and and it will publish the topology with the changes made. It may be a good idea to save this somewhere as a backup.

 

 

4. Control Panel

 

Now that you have made the changes required to the backend Topology, you need to edit the users to allow Remote Call Control.

 

First, run the Lync Server Control Panel.

 

When this loads, log in and click the “Users” tab on the left hand side.

 

This displays all users configured for Lync.

 

Select a user and make the following changes:

> Telephony – Change this to “Remote call control

> Line URI - This is an arbitrary number in the form “tel:<number>

> Line Server URI – This is in the form “sip:<user>@<fqdn of openfire server>

 

1. Control Panel - User Changes

 

 

Once done, select Commit at the top of the window.

 

 

5. Openfire Server

 

On the Openfire side, you will need to add the CSTA plugin. This can be done in one of two ways, like any other plugin.

1. On the Openfire Admin Console go to the Plugins tab and add the plugin from there.

2. Stop the Openfire server, move the plugin into the plugins directory, and start the Openfire server back up.
Once done, the CSTA configuration page will appear in the Admin Console under the tab “Unify”.

 

If you are using any of our other telephony plugins (e.g. Etrali or Cisco), it will appear alongside those, otherwise it would be in the Unify tab on it’s own.

1. Openfire Server - Unify:CSTA tabs

 

Click on the CSTA tab to view the CSTA Properties page, where you can configure the settings for you setup.

2. Openfire Server - CSTA Offline

 

Here you need to change the listening point to match your setup.

 

The format required for this field is:

SIP://<ip of openfire server>:<sip csta port>/TCP

 

Other than this, the other settings can be left as they are.

 

Once the change is made, click “Save Properties” and then restart the MAS.

 

When you go back to the page it should be green and say “Service is starting: Awaiting RCC User Logon”.

 

This means that the plugin has started is waiting for a user to log on.

3. Openfire Server - CSTA Waiting login

 

 

6. Lync Client

 

Now you are done with the Openfire side, you are ready to log on to the Lync Client.

 

Log on as normal, and once done go to the Options menu, and to the “Phones” tab.

 

Here you need to check the box “Enable integration with your phone system”.

1. Lync Client - Enable integration

 

 

Then click the “Advanced” button and ensure that “Automatic Configuration” is checked.

2. Lync Client - Advanced

 

Log out and back in to the client, and the change will take effect.

 

Check the Openfire Admin Console now and you’ll see that “Service is starting: Awaiting RCC User Logon” has changed to “Service is online”.

3. Lync Client - CSTA Online

 

In addition, if you go to the “CSTA Users” menu on the left side, you’ll see the user logged in, the system he is using, and his “Device Name”.

4. Lync Client - CSTA Users

 

If the device name is listed, with the correct “tel:#” configured in the Control Panel, then the user will be able to dial out using their configured system.

 

 

Lync-Openfire XMPP Federation

1. Lync Topology Setup

 

On the Lync Server FE, run the Topology Builder and create a new Edge Pool.

 

a) First you enter the FQDN of the Edge Server’s Internal Interface:

1. Defining the Edge Pool - Internal NIC FQDN

 

b) On the next screen make sure you enable XMPP federation on the pool.

2. Defining the Edge Pool - XMPP Federation checked

 

c) Next you have to define the FQDN and ports of the Edge External Services. Depending on your selection in the previous screen, you will need to enter either 1 or 3 FQDNs.

3. Defining the Edge Pool - External NIC FQDN

 

d) Next you’ll define the Internal IPv4 Address of the Edge Server.

4. Defining the Edge Pool - Internal IPv4

 

e) The last part to define is the IPv4 address of the External Edge NIC.

5. Defining the Edge Pool - External IPv4

 

f) Finally, save and publish the topology.

 

 

2. Lync Control Panel

 

In the Lync Control Panel go to the “Federation and External Access” tab.

 

a) Under “External Access Policy“, edit the Global scope and make sure the following boxes are checked the click “Commit“:

> Enable communications with federated users

> Enable communications with XMPP federated users

> Enable communications with remote users

> Enable communications with public users

 

1. External Access Policy

 

b) Under “Access Edge Configuration” make sure the following boxes are checked and click “Commit“:

> Enable federation and public IM connectivity

> Enable partner domain discovery

> Enable remote user access

 

 

2. Access Edge Configuration

 

c) Under “XMPP Federated Partners” click “New” and enter the details below then click “Commit“:

> Primary Domain – The domain/FQDN of the XMPP partner

> Partner Type – Select “Federated

> TLS negotiation – Select “Not Supported

> SASL negotiation – Select “Not Supported

> Support server dial back negotiation – Select “True

 

3. XMPP Federated Partners

 

3. Lync Management Console

 

Now you need to export the configuration for the Edge Server to use.

 

a) Run the Lync Management Console and enter the command below:

1. Configuration Export

 

Now copy this export.zip to you Edge Server

 

 

4. Edge Server Setup

 

You must ensure you have two separate NICs on the Edge server, each with it’s own IP and FQDN (will require firewall settings).

You can install a new NIC either from the VM management (If it is a virtual machine) or from Windows Device Manager.

 

Make sure the XMPP server can ping this external interface, and that Edge can ping the XMPP server.

 

 

5. Installing the Edge Server

 

Run the standard Lync Deployment Wizard that comes with the Lync Server 2013 CD.

 

a) When the Wizard is run, select the option “Install or Update Lync Server System”

1. Install or Update Lync Server System

 

b) On the next screen select “Install Local Configuration Store” and then choose “Import from a file”. Click “Browse” and choose the file exported from Lync in step 3a).

2. Installing Local Configuration Store

 

c) Next to do is run the step “Set up Lync Server Components”. This will install the required components for the Edge Server.

3. Set up Lync Server Components

 

d) This step involves setting up the certificates. If you are not importing, you can request for each of the options (Internal and External).

Note that you can use and internal CA for Openfire federation, but it would need a certified CA for external Lync access.

4. Certificates

 

e) Once the certificates are set up, you can start the services from the deployment wizard.

Initially starting services from the Deployment Wizard is recommended as it will show any errors in the setup in the log (which can be accessed directly from the wizard).

Otherwise, run services.msc to view the services as shown below.

5. Edge Services

 

 

 

6. DNS Records

 

If all the services are running, you will need to set up the DNS records.

 

The main DNS record required (where the external NIC sits) is “_xmpp-server._tcp.<domain> 5269 <edge external NIC FQDN>“.

 

It may also be worth adding a service record for the XMPP server in you DNS where Lync sits.

 

a) To do this on Windows DNS Server:

> Create a new “Forward Lookup Zone” with the FQDN of the XMPP server

> Add an A record which points to the IP of the XMPP server (no need to add an FQDN, it will just use the same as the parent folder).

> Create the SRV record “_xmpp-server._tcp.<FQDN of xmpp server> 5269 <FQDN of xmpp server>

> E.g. _xmpp-server._tcp.openfire.domain.com 5269 openfire.domain.com

 

b) Secondly, the internal DNS need some new records within the local domain. These are below:

> Create the SRV record “_sipfederationtls._tcp.<internal domain> 5061 <FQDN of edge external interface>

> E.g. _sipfederationtls._tcp.test.local 5061 externaledge.test.com

> Create the SRV record “_sip._tls.<internal domain> 443 <FQDN of edge external interface>

> E.g. _sip._tls.test.local 443 externaledge.test.com 

 

 

7. Openfire Server

 

Now some changes need to be made to the Openfire server.

 

a) Go to “Server > Server Settings > Security Settings” and under “Server Connection Security” do the following, then click “Save Settings“:

> Check the radio button “Custom”

> For “Server Dialback“, check “Available

> For “TLS Method“, check “Not Available

> Check “Accept self-signed certificates. Server dialback over TLS is now available.

 

1. Security Settings

 

b) Under “Server > Server Settings > Server to Server” ensure:

> Under “Service Enabled“, “Enabled” is checked, with port “5269

> Under “Allowed to Connect“, “Anyone” is checked

 

2. Server to Server Settings

If either of these isn’t checked, check it and click “Save Settings

 

c) Next, you need to add the internal domain where lync is to the XMPP server’s hosts file, and point it to the edge server’s external IP address.

The reason is that the request from Lync comes from/goes to the user “lyncuser@internallync.test.local“, so the federation must be made with the domain internallync.test.local. Therefore the XMPP Server needs to know where this domain can be found.

 

d) If running a linux box, follow the steps below:

> Open a console to the linux machine and run the command in the next step
vi /etc/hosts

> Add the line “<external Edge NIC IP> <internal Lync user domain>

 

3. Hosts File

 

e) Finally, you may need to download a new openfire.jar with changes to the TCP Dialback protocol. This can be downloaded at the following location:

> http://www.globility.co.uk/downloads/openfire-3.8.2-gltd-0.0.1.jar

 

Note that this only works on Openfire 3.8.2.

 

 

8. Adding the contact

 

You are now ready to add the contact.

 

a) On the Lync server, select the add contact button then go to “Add a Contact Not in My Organisation > Other

1. Lync - How to add the contact

 

b) In the form, add the Openfire contact “openfireuser@openfireserver.domain.com

2. Lync - Add Contact Form

 

c) From the Openfire User’s XMPP Client, add the user “lyncuser@<internal lync domain>“.

3. Openfire - Adding the contact

 

On the Openfire Admin Console, you can check the server-to-server sessions and will see one created with the internal Lync domain.

 

 

References

 

https://www.orcsweb.com/blog/cory-granata/installing-lync-2013-edge-server/

http://ocsguy.com/2010/11/29/deploying-lync-for-xmpp/

 

Installing Mail Server on Nagios

Postfix Server Installation
First you need to install postfix;

 

When prompted, choose internet site, then enter the @ part where you wish the mail to be sent from, e.g nagios.notify. This will mean that emails received from nagios will be @nagios.notify.

 

 

Mailx Server Installation

 

Now install Nagios’s part of the mail server;

 

 

Configuring Nagios Mail Command

 

You need to edit the commands.cfg folder to take into account the new paths;

 

Replace;

 

With;

 

Note that the change is ‘/bin/mail’ to ‘/usr/bin/mailx’.

 

Now restart Nagios;

 

You should now receive email notifications of services/host states.

 

 

Enabling and Disabling Notifications for a Service

 

To disable notifications follows the steps below.

 

  1. Access the web interface;

  1. Choose Hosts or Services from the side menu (depending on which you wish to change).
  1. Select a service or host you wish to disable notifications for (note that HTTP and SSH notifications are disabled by default on localhost).
  1. On the page which opens, from the right hand section select the option ‘Disable Notifications’.
  1. This should open a page asking to confirm the host or host and service you wish to disable notifications for. Select ‘Commit’ to continue;
  1. From this page it will now show a page verifying that it has been complete. Make sure to select the option ‘Done';

 

This will return you to the homepage. Navigate back to the host/service (from 4.) and you will see at the bottom it shows Notifications in red and as ‘DISABLED’. Also the right hand menu has changed to give the option to ‘Enable’ notifications as they are currently disabled.

 

Configure Nagios

This will guide you through a basic setup of a host and some services for Nagios.

 

First thing to note is that Nagios Core has no web based UI for configuring hosts, services or commands.

 

 

Adding a Host

 

To start the configuration, add a host. You can choose the default location to create a hosts file;

 

Any new files/directories created must be added to nagios.cfg;

 

New files are added as follows;

 

Once you have created the new file, open it with your text editor;

 

A simple linux host is defined as follows;

 

The “use” part of this host definition is taken from;

 

Here you can choose the default settings for a host, e.g. contact groups, notifications enabled, retry attempts.

 

A full list of possible host definitions can be found here;

 

http://nagios.sourceforge.net/docs/3_0/objectdefinitions.html#host

 

If you wish to set a specific setting for just a few hosts, adding it to the hosts.cfg folder overrides the template.

 

 

Plugin Usage and Adding Commands

 

Commands are added through the commands.cfg file located here;

 

Adding a command requires a plugin for the command you wish to add, as well as knowledge on how to use the plugin.

 

The default locations for plugins you wish to add can be found here;

 

To know what command is required for a plugin to work, you can request help on the plugins usage;

 

Note that some plugins may require pre-requisites, which will be prompted when you attempt to use the plugin in the command line.

 

An example would be the plugin check_ping which has the following output when you use the “-h” argument;

 

This helps you understand how to define a command.

 

Now edit the commands.cfg file and look for the check_ping command definition;

 

– This is the name you use to define this command. This is used in the service definition to reference this command.

– This is defined in /usr/local/nagios/etc/objects/resources.cfg. In this case it defines the plugin directory being used (/usr/local/nagios/libexec).
“check_ping” requires the arguments -H, -w and -c.

 

– This defines the host which needs to be checked, and is common in nearly all plugins. The $HOSTADDRESS$ part is an inbuilt Nagios macro which uses the hostname defined in the service command. This won’t need to be defined in each service which uses this commands, as you will see below.

– This sets the warning value which is required for the plugin to work. The $ARG1$ is defined in the service command.

 

– This sets the critical value which is required for the plugin to work. The $ARG2$ is defined in the service command.

 

– This argument is an optional addition to the command definition. By not using $ARG3$ we are saying that every time this command is used, set “-p” to “5”.

Adding Services

 

Now you have a host and a command, you need a service!

 

This services config file can be created/found in the same place as the host file;

 

Remember to add this location to nagios.cfg if it isn’t already there.

 

Now you need to add some services to the config file for your host.

 

The service for ping is defined as follows;

 

With regards to the “check_command”, this is where you define the command to be used for this service by entering the “command_name” defined in commands.cfg.

 

– This is the “command_name”. This is always entered first.

 

After the command_name, you must enter the details for $ARG1$, $ARG2$, etc. These are separated by an exclamation mark, “!”.

 

– This the response time of the ping request. If it’s sufficient for this argument, you will receive a WARNING status.

 

– This the response time of the ping request. If it’s sufficient for this argument, you will receive a CRITICAL status.

As before, the “use” part is defined in templates.cfg.

 

A full list of service definitions can be found here;

http://nagios.sourceforge.net/docs/3_0/objectdefinitions.html#service

 

Adding a definition to services.cfg overrides the template.

You now have a new host configured with the ping service.

 

 

Hostgroups

 

If you will have, say, 10 linux servers, all requiring the “ping” command, it would be best to define a hostgroup in the host and service definitions instead of the host name.

 

A hostgroup is created as follows;

 

 

Note that when adding hostgroups/hostgroup_name to the host/service definitions, respectively, they are seperated by commas.

e.g.

These basic steps can be repeated for most plugins – the only differences come when using NRPE, NSCA, or a custom plugin, but even then the basics are essential, or when you are missing pre-requisites for the plugin to work.

When you are finished with your customization, run the command below to confirm that there are no issues with the configuration files;

Any problems are made apparent to you after running this command, telling you both the file and line where the error occurs.

 

This makes debugging errors very easy.

 

If there are no problems, you can restart nagios and head to the web page;

 

On a web browser;

http://<nagios server ip>/nagios

 

The following will be how the ping service is displayed;

 

Install Nagios on Ubuntu

The following is a guide to install Nagios Core on an Ubuntu machine.

 

Preparing for the Installation

 

First you will need to be root;

 

For convenience it is best to install ssh and vim;

 

Now you need to install some pre-requisites required for Nagios to correctly install/run;

 

Now add the user ‘nagios’ to the system;

 

Create nagcmd group to allow commands through the web;

 

 

Installing Nagios and the Plugins

 

First, make a directory to download the plugins to

 

Now you need to download Nagios and Nagios plugins;

 

Extracting Nagios so it can be compiled/installed;

 

Now you must run the install script and compile the source code;

 

Run the following command to install the web server

 

The next step is to create a password for the nagios web server login account, ‘nagiosadmin’

 

Remember this password as it is required to log on to the web interface.

 

Restart apache to apply the changes

 

Now you must extract the plugins;

 

Use the commands below to compile and install the plugins

 

Now configure Nagios to start on system start

 

Check the nagios configuration with the following command

 

Make note of this command, it is very important later on for debugging issues with config files.

 

If there are no errors, you are ready to start Nagios;

 

Now access the web interface from a web browser, navigating to
http://localhost/nagios/
or from within the network;
http://<hostname/ip>/nagios/
Congratulations, you are now done with the installation!

Please refer to our blog for a post on configuring your first host.

 

OUR BLOG

on July 5, 2018

We have started using medium for our blogging. Please click here to see our latest posts.

on February 9, 2015

BaseX Blog Introduction This 'Blurt/Blog' details my experiences of a recent

on December 4, 2013

iCosts - The Legal Costs Calculator - Demo http://youtu.be/P5pSAZzKi08

on December 4, 2013

Lync XMPP + Video - Demo http://youtu.be/aWpaxFGDwXY

Contact Us

9th Floor Capital House
40-42 Weston Street
London
SE1 3QD

sales@gltd.net 0207 100 1499 Follow us Like us

Specialists in integrated and flexible communications

Copyright © 2013 Globility Limited. All rights reserved.